DeepSIC: Deep Semantic Image Compression

arXiv:1801.09468v1 [] 29 Jan 2018

Sihui Luo
Zhejiang University
Hangzhou, China

Yezhou Yang
Arizona State University

Abstract— Incorporating semantic information into the
codecs during image compression can significantly reduce
the repetitive computation of fundamental semantic analysis
(such as object recognition) in client-side applications. The
same practice also enable the compressed code to carry the
image semantic information during storage and transmission.
In this paper, we propose a concept called Deep Semantic
Image Compression (DeepSIC) and put forward two novel
architectures that aim to reconstruct the compressed image
and generate corresponding semantic representations at the
same time. The first architecture performs semantic analysis
in the encoding process by reserving a portion of the bits
from the compressed code to store the semantic representations.
The second performs semantic analysis in the decoding step
with the feature maps that are embedded in the compressed
code. In both architectures, the feature maps are shared by
the compression and the semantic analytics modules. To validate our approaches, we conduct experiments on the publicly
available benchmarking datasets and achieve promising results.
We also provide a thorough analysis of the advantages and
disadvantages of the proposed technique.

I. I NTRODUCTION
As the era of smart cities and Internet of Things (IoT)
unfolds, the increasing number of real-world applications
require corresponding image and video transmission services
to handle compression and semantic encoding at the same
time, hitherto not addressed by the conventional systems.
Traditionally, image compression process is merely a type
of data compression that is applied to digital images to
reduce the storage and transmission cost of them. Almost
all the image compression algorithms stay at the stage of
low-level image representation in which the representation
is arrays of pixel values. For example, the most widely used
compression methods compress images through pixel-level
transformation and entropy encoding [32], [5]. However,
these conventional methods do not consider encoding the
semantic information (such as the object labels, attributes,
and scenes), beyond low-level arrays of pixel values. At the
same time, the semantic information is critical for high-level
reasoning over the image.
Within the last decade, Deep Neural Networks (DNN)
and Deep Learning (DL) have laid the foundation for going
beyond hand-crafted features in visual recognition, with a
significant performance boost in topics ranging from object
recognition [17], scene recognition [30], [35], [10], action
recognition [33], to image captioning [4], [14], [31], [3] and
visual question answering [1], [36], [19]. Recently, several
significant efforts of deep learning based image compression

Fig. 1.

Mingli Song
Zhejiang University
Hangzhou, China

General semantic image compression framework

methods have been proposed to improve the compression
performance [2], [27], [28], [7], [26], [23].
As Rippel and Bourdev [23] point out, generally speaking,
image compression is highly related to the procedure of
pattern recognition. In other words, if a system can discover
the underlying structure of the input, it can eliminate the
redundancy and represent the input more succinctly. Recent
deep learning based compression approaches discover the
structure of images by training a compression model and
then convert it to binary code [2], [27], [28], [7], [26],
[23]. Nevertheless, to the best of our knowledge, a deep
learning based image compression approach incorporating
semantic representations has not yet been explored in the
literature. These existing DL-based compression codecs, like
the conventional codecs, also only compress the images at
pixel level, and do not consider the semantics of them. Currently, when the client-side applications require the semantic
information of an image, they have to firstly reconstruct
the image from the codec and then conduct an additional
computing to obtain the semantic information.
Can a system conduct joint-optimization of the objectives
for both compression and the semantic analysis? In this
paper, we make the first attempt to approach this challenging
task which stands between the computer vision and multimedia information processing fields, by introducing the Deep
Semantic Image Compression (DeepSIC). Here, our DeepSIC framework aims to encode the semantic information in
the codecs, and thus significantly reduces the computational
resources needed for the repetitive semantic analysis on the
client side.
We depict the DeepSIC framework in Figure 1, which
aims to incorporate the semantic representation within the
codec while maintaining the ability to reconstruct visually

pleasing images. Two architectures of our proposed DeepSIC
framework are given for the joint analysis of pixel information together with the semantic representations for lossy
image compression. One is pre-semantic DeepSIC, which
integrate the semantic analysis module into the encoder
part and reserve several bits in the compressed code to
represent the semantics. The other is post-semantic DeepSIC,
which only encodes the image features during the encoding
process and conducts the semantic analysis process during
the reconstruction phase. The feature retained by decoding is
further adopted in the semantic analysis module to achieve
the semantic representation.
In summary, we make the following contributions:
• We propose a concept called Deep Semantic Image
Compression that aims to provide a novel scheme
to compress and reconstruct both the visual and the
semantic information in an image at the same time. To
our best knowledge, this is the first work to incorporate
semantics in image compression procedure.
• We put forward two novel architectures of the proposed
DeepSIC: pre-semantic DeepSIC and post-semantic
DeepSIC.
• We conduct experiments over multiple datasets to validate our framework and compare the two proposed
architectures.
The rest of this paper is organized as follows: In section II,
the conventional and DL-based image compression methods
are briefly reviewed. Section III describes the proposed
DeepSIC framework and the two architectures of DeepSIC in
detail. Our experiment results and discussions are presented
in section IV. Finally, we conclude the paper and look into
the future work in section V.
II. R ELATED WORK
Standard codecs such as JPEG [32] and JPEG2000 [21]
compress images via a pipeline which roughly breaks down
to 3 modules: transformation, quantization, and entropy
encoding. It is the mainstream pipeline for lossy image
compression codec. Although advances in the training of
neural networks have helped improving performance in existing codecs of lossy compression, recent learning-based
approaches still follow the same pipeline as well [2], [27],
[28], [7], [26], [23].
These approaches typically utilize the neural networks to
retain the visual features of the image and then convert
them into binary code through quantization. Some of these
approaches may further compress the binary code via an
entropy encoder. The feature extraction process replaces
the transformation module in JPEG, which automatically
discovers the structure of image instead of engineers manually do. Toderici [27], [28] explore various transformations
for binary feature extraction based on different types of
recurrent neural networks and compressed the binary representations with entropy encoding. [2] introduce a nonlinear
transformation in their convolutional compression network
for joint-optimization of rate and distortion, which effectively
improves the visual quality.

Many recent works utilize autoencoders to reduce the
dimensionality of images. Promising results of autoencoders
have been achieved in converting an image to compressed
code for retrieval [16]. Autoencoders also have the potential
to address the increasing need for flexible lossy compression
algorithms [26]. Other methods expand the basic autoencoder
structure and generate the binary representation of the image
by quantizing the bottleneck layer.
More recently, since Goodfellow [6] introduced Generative
Adversarial Networks(GANs) for image generation, GANs
have been demonstrated to be promising in producing fine
details of the images [13], [12]. In the compression field,
GANs are usually employed to generate reconstructed images that look natural, with the naturalness measured by a
binary classifier. The intuition is, if it is hard to distinguish
the generated images from the original, then the generated
images are ”natural” enough for humans. Some works have
achieved significant progress in generating smaller compressed code size but more visually pleasing reconstructed
images. Gregor et al. [7] introduce a homogeneous deep
generative model in latent variable image modeling. Rippel
and Bourdev’s method [23] contains an autoencoder featuring pyramidal analysis and supplement their approach with
adversarial training. They achieve real-time compression
with pleasing visual quality at a rather low bit rate.
In general, these existing DL-based compression codecs,
like the conventional codecs, also only compress the images
at pixel level, and do not consider the semantics of them.
III. D EEP S EMANTIC I MAGE C OMPRESSION
In the proposed deep semantic image compression framework (DeepSIC), the compression scheme is similar to autoencoder. The encoder-decoder image compression pipeline
commonly maps the target image through a bitrate bottleneck
with an autoencoder and train the model to minimize a loss
function penalizing it from its reconstruction result. For our
DeepSIC, this requires a careful construction of a feature
extractor and reconstruction module for the encoder and
decoder, a good selection of an appropriate optimization
objective for the joint-optimization of semantic analysis
and compression, and an entropy coding module to further
compress the fixed-sized feature map to gain codes with
variable lengths.
Figure 2 shows the two proposed DeepSIC architectures:
pre-semantic DeepSIC and post-semantic DeepSIC respectively. For pre-semantic DeepSIC, it places the semantic
analysis in the encoding process, which is implemented by
reserving a portion of the bits in the compressed code to store
semantic information. Hence the code innately reflects the
semantic information of the image. For post-semantic image
compression, the feature retained from decoding is used for
the semantic analysis module to get the class label and for
reconstruction network to synthesize the target image. Both
architectures have modules of feature extraction, entropy
coding, reconstruction from features and semantic analysis.
Here, a brief introduction of the modules in our DeepSIC
is given.

Fig. 2. Two architectures of our semantic compression network: (a) Image compression with pre semantic analysis in the encoder; (b) Image compression
with post semantic analysis in the decoder.

Feature Extraction: Features represent different types of
structure in images across scales and input channels. Feature
extraction module in image compression aims to reduce the
redundancy while maintaining max-entropy of the containing
information. We adopt Convolutional Neural Network (CNN)
as the feature extraction network. Given x as the input image,
f (x) denotes the feature extraction output.
Entropy Coding: The feature output is firstly quantized to
a lower bit precision. We then apply CABAC [20] encoding
method to lossless leverage the redundancy remained in the
data. It encodes the output of quantization y into the final
binary code sequence ŷ.
Reconstruction from Features: By entropy decoding,
we retrieve image features from the compressed code. The
inverse process of feature extraction is performed in the
decoding process to reconstruct the target image.
Semantic Analysis: Semantic analysis module is the analysis implemented on the extracted feature map. As mentioned
in section I, there are many forms of semantic analysis
such as semantic segmentation, object recognition, and image
captioning. We adopt object classification as the semantic
analysis module in this paper.
In general, we encode the input images through feature

extraction, and then quantize and code them into binary
codes for storing or transmitting to the decoder. The reconstruction network then creates an estimate of the original
input image based on the received binary code. We further
train the network with semantic analysis both in the encoder
and in the decoder (reconstruction) network. This procedure
is repeated under the loss of distortion between the original
image and the reconstruction target image, together with the
error rate of the semantic analysis. The specific descriptions
of each module are present in the subsequent subsections.
A. Feature Extraction
The output of the feature extraction module is the feature
map of an image, which contains the significant structure of
the image. CNNs that create short paths from early layers
to later layers allow feature reuse throughout the network,
and thus allow the training of very deep networks. These
CNNs can represent the image better and are demonstrated
to be good feature extractors for various computer vision
tasks [24], [18], [25], [9], [11], [29]. The strategy of reusing
the feature throughout the network helps the training of
deeper network architectures. Feature extraction model in
compression model[26] adopt similar strategy. We adopt
the operation of adding the batch-normalized output of the

Fig. 3.

The feature extraction module and the reconstruction from features module: They are both formed by a four-stage convolutional network.

previous layer to the subsequent layer in the feature networks
to . Furthermore, we also observe that the dense connections
have a regularizing effect, which reduces overfitting on tasks
with small training set sizes.
Our model extracts image features through a convolutional
network illustrated in Figure 3. Given the input image as
x ∈ RC×H×W , we denote the feature extraction output as
f (x).
Specifically, our feature extraction module consists of four
stages, each stage comprises a convolution, a subsampling,
and a batch normalization layer. Each subsequent stage
utilizes the output of the previous layers. And each stage
begins with an affine convolution to increase the receptive
field of the image. This is followed by 4 × 4, 2 × 2, or 1 × 1
downsampling to reduce the information. Each stage then
concludes by a batch normalization operation.
B. Entropy Coding
Given the extracted tensor f (x) ∈ RC×H×W , before
entropy coding the tensor, we first perform quantization. The
feature tensor is optimally quantized to a lower bit precision
B:
Q(f (x)) =

1
2B−1

 B−1

2
f (x) .

We train a classifier to predict the value of each bit from its
context feature, and then use the resulting belief distribution
to compress b.
Given y = Q(f (x)) denotes the quantized code, after
entropy encoding y into its binary representation ŷ, we
retrieve the compression code sequence.
During decoding, we decompress the code by performing
the inverse operation. Namely, we interleave between computing the context of a particular bit using the values of
previously decoded bits. The obtained context is employed
to retrieve the activation probability of the bit to be decoded.
Note that this constrains the context of each bit to only
involve features composed of bits already decoded.
C. Reconstruction From Features
The module of reconstruction from features mirrors the
structure of the feature extraction module, which is four-stage
formed as well. Each stage comprises a convolutional layer
and an upsampling layer. The output of each previous layer
is passed on to the subsequent layer through two paths, one
is the deconvolutional network, and the other is a straightforward upsampling to target size through interpolation. After
reconstruction, we obtain the output decompressed image x̂.

(1)

The quantization bin B we use here is 6 bit. After
quantization, the output is converted to a binary tensor.
The entropy of the binary code generated during feature
extraction and quantization period are not maximum because
the network is not explicitly designed to maximize entropy
in its code, and the model does not necessarily exploit visual
redundancy over a large spatial extent.
We exploit this low entropy by lossless compression via
entropy coding, to be specific, we implement an entropy coding based on the context-adaptive binary arithmetic coding
(CABAC) framework proposed by [20]. Arithmetic entropy
codes are designed to compress discrete-valued data to bit
rates closely approaching the entropy of the representation,
assuming that the probability model used to design the code
approximates the data well. We associate each bit location
in Q(f (x)) with a context, which comprises a set of features
indicating the bit value. These features are based on the
position of the bit as well as the values of neighboring bits.


x̂ = g Q−1 (Q(f (x)))

(2)

Although arithmetic entropy encoding is lossless, the
quantization will bring in some loss in accuracy, the result
of Q−1 (Q(f (x)) is not exactly the same as the output of
feature extraction. It is an approximation of f (x).
D. Semantic Analysis
As aforementioned, there are a number of semantic analysis forms. Classification task is the commonly selected way
to evaluate deep learning networks [24], [18], [25], [9],
[11]. Thus we select object classification for experiments
in this paper. The structure of our semantic analysis module
contains a sequence of convolutions following with two fully
connected layers and a softmax layer.
Figure 4 presents the structure of our semantic analysis
module. It is position-optional and can be placed in the
encoding and decoding process for the two different architectures. We denote it as h (∗) to operate on the extracted

Therefore, R can be calculated as
"
R=E

#
X

log2 Pyi (n) .

(6)

i

D measures the distortion introduced by coding and decoding. It’s calculated by the distance between the original
image and the reconstructed image. We take MSE as the
distance metric for training, thus D is defined as
Fig. 4.

The structure of semantic analysis module

h
i
2
D = E kxi − x̂i k2 .
feature map f (x). Thus the output semantic representations
are h (f (x)).
For the classification task in the semantic analysis part,
we adjust the learning rate using the related name-value
pair arguments when creating the fully connected layer.
Moreover, a softmax function is ultilized as the output unit
activation function after the last fully connected layer for the
multi-class classification.
We set the cross entropy of the classification results as
the semantic analysis loss Lsem in this module. Denote the
weight matrix of the two fully connected layer as Wf c1 and
Wf c2 respectively. Lsem is calculated as follows:
Lsem = E [sof tmax [Wf c2 ∗ (Wf c1 ∗ f (x))]]

(3)

It is worth noting that the inputs of the semantic analysis
module in the two proposed architectures are slightly different. The input feature maps of semantic analysis module
in pre-semantic DeepSIC are under floating point precision.
Differently, the input feature maps of semantic analysis
module in post-semantic DeepSIC are under fixed-point
precision due to quantization and entropy coding.
E. Joint Training of Compression and Semantic Analysis
We implement end-to-end training for the proposed DeepSIC, jointly optimize the two constraints of the semantic
analysis and image reconstruction modules. And we define
the loss as the weighted evaluation of compression ratio R,
distortion D and the loss of the semantic analysis Lsem in
Equation 4.
L = R + λ1 D + λ2 Lsem

Z

n+ 12

n− 12

pyˆi (t)dt

IV. E XPERIMENT
In this section, we present experimental results over multiple datasets to demonstrate the effectiveness of the proposed
semantic image compression.
A. Experimental Setup
Datasets For training, we jointly optimized the full set of
parameters over ILSVRC 2012 which is the subset of the
ImageNet. The ILSVRC 2012 classification dataset consists
of 1.2 million images for training, and 50,000 for validation
from 1, 000 classes. A data augmentation scheme is adopted
for training images and resize them to 128 × 128 at training
time. We report the classification accuracy on the validation
set. Performance tests on Kodak PhotoCD dataset are also
present to enable comparison with other image compression
codecs. Kodak PhotoCD dataset [5] is an uncompressed set
of images which is popularly used for testing compression
performances.
Metric To assess the visual quality of reconstructed images, we adopt Multi-Scale Structural Similarity Index
Metric (MS-SSIM) [34] and Peak Signal-to-Noise Ratio (PSNR) [8] for comparing original, uncompressed images
to compressed, degraded ones. We train our model on the
MSE metric and evaluate all reconstruction model on MSSSIM. MS-SSIM is a representative perceptual metric which
has been specifically designed to match the human visual
system. The distortion of reconstructed images quantified by
PSNR is also provided to compare our method with JPEG,
JPEG2000 and DL-based methods. Moreover, we report the
classification accuracy over validation and testing sets to
evaluate the performance of semantic analysis module.

(4)

Here, λ1 and λ2 govern the trade-offs of the three
terms. Since the quantization operation is non-differential,
the marginal density of yˆi is then given by the training of
the discrete probability masses with weights equal to the
probability mass function of yˆi , where index i runs over all
elements of the vectors, including channels, image width and
height.

Pyi (n) =

(7)

(5)

B. Implementation and Training Details
We conduct training and testing on a NVIDIA Quadro
M6000 GPU. All models are trained with 128×128 patches
sampled from the ILSVRC 2012 dataset. All network architectures are trained using the Tensorflow API, with the
Adam [15] optimizer. We set B = 32 as the batch size, and
C = 3 as the number of color channels. The extracted feature
dimensions is variable due to different subsample settings to
gain variable length of compressed code. This optimization
is performed separately for each weight, yielding separate
transforms and marginal probability models.

TABLE I
ACCURACY OVER DIFFERENT COMPRESSION RATIOS ( MEASURED BY
BPP ) ON ILSVRC VALIDATION , WITH COMPARISONS TO THE
STATE - OF - THE - ART CLASSIFICATION METHODS .
PRE - SEMANTIC

P RE -SA IS SHORT FOR
D EEP SIC AND P OST-SA IS SHORT FOR POST- SEMANTIC
D EEP SIC.

Method
Pre-SA (0.25bpp)
Post-SA (0.25bpp)
Pre-SA (0.5bpp)
Post-SA (0.5bpp)
Pre-SA (1.0bpp)
Post-SA (1.0bpp)
Pre-SA (1.5bpp)
Post-SA (1.5bpp)
VGG-16[24] (10-crops)
Yolo Darknet[22]
DenseNet-121[11] (10-crops)

Fig. 5. Examples of reconstructed image parts by different codecs (JPEG,
JPEG 2000, ours, Toderici [28] and Rippel [23]) for very low bits per
pixel (bpp) values. The uncompressed size is 24 bpp, so the examples
represent compression by around 120 and 250 times. The test images are
from the Kodak PhotoCD dataset.

Fig. 6.
Summary rate-distortion curves, computed by averaging results
over the 24 images in the Kodak test set. JPEG and JPEG 2000 results are
averaged over images compressed with identical quality settings.

We use 128 filters (size 7×7) in the first stage, each
subsampled by a factor of 4 or 2 vertically and horizontally,
and followed up with 128 filters (size 5×5) with the stride
of 2 or 1. The remaining two stages retain the number of
channels, but use filters operating across all input channels
(3×3×128), with outputs subsampled by a factor of 2 or 1 in
each dimension. The structure of the reconstruction module
mirrors the structure of the feature extraction module.
The initial learning rate is set as 0.003, with decaying
twice by a factor of 5 during training. We train each model
for a total of 8,000,000 iterations.
C. Experimental Results
Since semantic compression is a new concept, there are no
direct baseline comparisons. We conduct many experiments

Top-1 acc.
52.2%
51.6%
63.2%
61.9%
68.7%
68.8%
67.1%
68.9%
71.9%
76.5%
76.4%

Top-5 acc.
72.7%
71.4%
82.2%
81.4%
89.4%
89.9%
90.1%
89.9%
90.7%
93.3%
93.4%

and present their results as follows:
To evaluate performance of image compression quality, we
compare DeepSIC against standard commercial compression
techniques JPEG, JPEG2000, as well as recent deep-learning
based compression work [2], [28], [23]. We show results for
images in the test dataset and in every selected available
compression rate. Figure 5 shows visual examples of some
images compressed using proposed DeepSIC optimized for
a low value above 0.25 bpp, compared to JPEG, JPEG
2000 and DL-based images compressed at equal or greater
bit rates. The average Rate-Distortion curves for the luma
component of images in the Kodak PhotoCD dataset, shown
in Figure 6. Additionally, we take average MS-SSIM and
PSNR over Kodak PhotoCD dataset as the functions of the
bpp fixed for the testing images, shown in Figure 7.
To evaluate the performance of semantic analysis, we
demonstrate some examples of the results of DeepSIC with
reconstructions and their semantics, shown in Figure 8.
Comparisons of the semantic analysis result of the two
proposed architectures on classification accuracy are given in
Table I. Furthermore, as different compression ratios directly
affect the performance of compression, we compare semantic
analysis result of the proposed architectures over certain fixed
compression ratios with the mainstream classification methods in Table I. It also presents the trend of how compression
ratio affects the performance of semantic analysis.
Although we use MSE as a distortion metric for training
and incorporate semantic analysis with compression, the appearance of compressed images are substantially comparable
with JPEG and JPEG 2000, and slightly inferior to the
DL-based image compression methods. Consistent with the
appearance of these example images, we retain the semantic
representation of them through the compression procedure.
Although the performance of our method is neither the best
on the visual quality of the reconstructed images nor on the
classification accuracy, the result is still comparable with the
state-of-the-art methods.

Fig. 7. Average perceptual quality and compression ratio curves for the luma component of the images from Kodak dataset. Our DeepSIC is compared
with JPEG, JPEG2000, Ballé [2], Toderici [28] and Rippel [23]. Left: perceptual quality, measured with multi-scale structural similarity (MS-SSIM). Right:
peak signal-to-noise ratio (PSNR). We have no access to reconstructions by Rippel[23], so we carefully transcribe their results, only available in MS-SSIM,
from the graphs in their paper.

Fig. 8.
Examples of results with our DeepSIC: Images in the first row are the original images. The reconstructed images of the two architectures of
DeepSIC are shown in the second and the third row followed by the output representation of the semantic analysis module.

D. Discussion
We perform object classification as the semantic analysis
module and represent the semantics with the identifier code
of the class in this paper. Nevertheless, the semantics itself is
complicated. The length of compressed code directly affect
the size of compressed code and is far from limitless. The
problem of how to efficiently organize semantic representation of multiple objects need careful consideration and
exploration.

Yolo9000 [22] performed classification with WordTree,
a hierarchical model of visual concepts that can merge
different datasets together by mapping the classes in the
dataset to synsets in the tree. It inspires us that models like
WordTree can also be applied to hierarchically encoding the
semantics. We can set up a variety of levels to represent
the semantic information. For example, from the low-level
semantic representation, you can know “there is a cat in the
image”. While from the high-level one, you can know not

only “there’s a cat” but also “what kind of cat it is”. This
kind of schemes are more efficient to represent the semantics
of the images.
V. C ONCLUSION AND F UTURE W ORK
In this paper, we propose an image compression scheme
incorporating semantics, which we refer to as Deep Semantic
Image Compression (DeepSIC). The proposed DeepSIC aims
to reconstruct the images and generate corresponding semantic representations at the same time. We put forward two
novel architectures of it: pre-semantic DeepSIC and postsemantic DeepSIC. To validate our approach, we conduct
experiments on Kodak PhotoCD and ILSVRC datasets and
achieve promising results. We also compare the performance of the proposed two architectures of our DeepSIC.
Though incorporating semantics, the proposed DeepSIC is
still comparable with the state-of-the-art methods over the
experimental results.
This practice opens up a new thread of challenges and
has the potential to immediately impact a wide range of
applications such as semantic compression of surveillance
streams for the future smart cities, and fast post-transmission
semantic image retrieval in the Internet of Things (IoT)
applications.
Despite the challenges to explore, deep semantic image
compression is still an inspiring new direction which breaks
through the boundary of multi-media and pattern recognition.
Nevertheless, it’s unrealistic to explore all the challenges at
once. Instead, we mainly focus on the framework of deep
semantic image compression in this paper. The proposed
DeepSIC paves a promising research avenue that we plan to
further explore other possible solutions to the aforementioned
challenges.
R EFERENCES
[1] S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick,
and D. Parikh. Vqa: Visual question answering. In International
Conference on Computer Vision (ICCV), 2015.
[2] J. Ballé, V. Laparra, and E. P. Simoncelli. End-to-end optimized image
compression. arXiv preprint arXiv:1611.01704, 2016.
[3] X. Chen and C. L. Zitnick. Learning a recurrent visual representation
for image caption generation. arXiv preprint arXiv:1411.5654, 2014.
[4] J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional
networks for visual recognition and description. arXiv preprint
arXiv:1411.4389, 2014.
[5] R. Franzen. Kodak lossless true color image suite. Source: http://r0k.
us/graphics/kodak, 4, 1999.
[6] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In
International Conference on Neural Information Processing Systems,
pages 2672–2680, 2014.
[7] K. Gregor, F. Besse, D. J. Rezende, I. Danihelka, and D. Wierstra.
Towards conceptual compression. In Advances In Neural Information
Processing Systems, pages 3549–3557, 2016.
[8] P. Gupta, P. Srivastava, S. Bhardwaj, and V. Bhateja. A modified psnr
metric based on hvs for quality assessment of color images. In International Conference on Communication and Industrial Application,
pages 1–4, 2012.
[9] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image
recognition. In IEEE Conference on Computer Vision and Pattern
Recognition, pages 770–778, 2016.
[10] L. Herranz, S. Jiang, and X. Li. Scene recognition with cnns: Objects,
scales and dataset bias. In IEEE Conference on Computer Vision and
Pattern Recognition, pages 571–579, 2016.

[11] G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten. Densely
connected convolutional networks. arXiv preprint arXiv:1608.06993,
2016.
[12] D. J. Im, C. D. Kim, H. Jiang, and R. Memisevic. Generating images
with recurrent adversarial networks. arXiv preprint arXiv:1602.05110,
2016.
[13] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for realtime style transfer and super-resolution. In European Conference on
Computer Vision, pages 694–711. Springer, 2016.
[14] A. Karpathy and F.-F. Li. Deep visual-semantic alignments for
generating image descriptions. arXiv preprint arXiv:1412.2306, 2014.
[15] D. Kingma and J. Ba. Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980, 2014.
[16] A. Krizhevsky and G. E. Hinton. Using very deep autoencoders for
content-based image retrieval. In European Symposium on Artificial
Neural Networks, Bruges, Belgium, 2011.
[17] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification
with deep convolutional neural networks. In Advances in neural
information processing systems, pages 1097–1105, 2012.
[18] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks
for semantic segmentation. In IEEE Conference on Computer Vision
and Pattern Recognition, pages 3431–3440, 2015.
[19] J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical questionimage co-attention for visual question answering. arXiv preprint
arXiv:1606.00061, 2016.
[20] D. Marpe, H. Schwarz, and T. Wiegand. Context-based adaptive binary
arithmetic coding in the h. 264/avc video compression standard. IEEE
Transactions on circuits and systems for video technology, 13(7):620–
636, 2003.
[21] M. Rabbani and R. Joshi. An overview of the jpeg 2000 still image
compression standard. Signal processing: Image communication,
17(1):3–48, 2002.
[22] J. Redmon and A. Farhadi. Yolo9000: better, faster, stronger. arXiv
preprint arXiv:1612.08242, 2016.
[23] O. Rippel and L. Bourdev. Real-time adaptive image compression.
arXiv preprint arXiv:1705.05823, 2017.
[24] K. Simonyan and A. Zisserman. Very deep convolutional networks for
large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[25] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with
convolutions. In IEEE Conference on computer vision and pattern
recognition, pages 1–9, 2015.
[26] L. Theis, W. Shi, A. Cunningham, and F. Huszár. Lossy image compression with compressive autoencoders. arXiv preprint
arXiv:1703.00395, 2017.
[27] G. Toderici, S. M. O’Malley, S. J. Hwang, D. Vincent, D. Minnen,
S. Baluja, M. Covell, and R. Sukthankar. Variable rate image compression with recurrent neural networks. arXiv preprint arXiv:1511.06085,
2015.
[28] G. Toderici, D. Vincent, N. Johnston, S. J. Hwang, D. Minnen, J. Shor,
and M. Covell. Full resolution image compression with recurrent
neural networks. 2017.
[29] L. G. L. X. Tong, T. and Q. Gao. Image super-resolution using dense
skip connections. In IEEE Conference on Computer Vision and Pattern
Recognition.
[30] K. E. A. Van, de Sande, T. Gevers, and C. G. M. Snoek. Evaluating
color descriptors for object and scene recognition. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 32(9):1582–96, 2010.
[31] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A
neural image caption generator. arXiv preprint arXiv:1411.4555, 2014.
[32] G. K. Wallace. The jpeg still picture compression standard. Communications of the Acm, 38(1):xviii–xxxiv, 1992.
[33] H. Wang, A. Klser, C. Schmid, and C. L. Liu. Action recognition
by dense trajectories. In IEEE Conference on Computer Vision and
Pattern Recognition, pages 3169–3176, 2011.
[34] Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural
similarity for image quality assessment. In Signals, Systems and
Computers, 2004. Conference Record of the Thirty-Seventh Asilomar
Conference on, pages 1398–1402 Vol.2, 2003.
[35] B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In
Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q.
Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 487–495. Curran Associates, Inc., 2014.
[36] Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei. Visual7w: Grounded
question answering in images. arXiv preprint arXiv:1511.03416, 2015.

