Sandwiched image compression: Wrapping neural networks around a standard codec
IEEE International Conference on Image Processing (ICIP) 2021.
Improved image compression using neural pre- and post-processing.
We sandwich a standard image codec between two neural networks: a preprocessor that outputs neural codes,
and a postprocessor that reconstructs the image. The neural codes are compressed as ordinary images by the
standard codec. Using differentiable proxies for both rate and distortion, we develop a rate-distortion
optimization framework that trains the networks to generate neural codes that are efficiently compressible
as images. This architecture not only improves rate-distortion performance for ordinary RGB images, but
also enables efficient compression of alternative image types (such as normal maps of computer graphics)
using standard image codecs. Results demonstrate the effectiveness and flexibility of neural processing in
mapping a variety of input data modalities to the rigid structure of standard codecs. A surprising result
is that the rate-distortion-optimized neural processing seamlessly learns to transport color images using a
single-channel (grayscale) codec.
No hindsights yet.