1

Generalizable Data-free Objective for Crafting
Universal Adversarial Perturbations

arXiv:1801.08092v1 [] 24 Jan 2018

Konda Reddy Mopuri*, Aditya Ganeshan*, R. Venkatesh Babu, Senior Member, IEEE
Abstract—Machine learning models are susceptible to adversarial perturbations: small changes to input that can cause large changes
in output. It is also demonstrated that there exist input-agnostic perturbations, called universal adversarial perturbations, which can
change the inference of target model on most of the data samples. However, existing methods to craft universal perturbations are (i)
task specific, (ii) require samples from the training data distribution, and (iii) perform complex optimizations. Also, because of the data
dependence, fooling ability of the crafted perturbations is proportional to the available training data. In this paper, we present a novel,
generalizable and data-free objective for crafting universal adversarial perturbations. Independent of the underlying task, our objective
achieves fooling via corrupting the extracted features at multiple layers. Therefore, the proposed objective is generalizable to craft
image-agnostic perturbations across multiple vision tasks such as object recognition, semantic segmentation and depth estimation.
In the practical setting of black-box attacking scenario, we show that our objective outperforms the data dependent objectives to fool
the learned models. Further, via exploiting simple priors related to the data distribution, our objective remarkably boosts the fooling
ability of the crafted perturbations. Significant fooling rates achieved by our objective emphasize that the current deep learning models
are now at an increased risk, since our objective generalizes across multiple tasks without the requirement of training data for crafting
the perturbations. To encourage reproducible research, we have released the code for our proposed algorithm at GitHub† .
Index Terms—Adversarial perturbations, fooling CNNs, stability of Neural Networks, perturbations, universal, generalizable attacks,
attacks on ML systems, data-free objectives, adversarial noise.

F

1

I NTRODUCTION

S

MALL but structured perturbations to the input, called
adversarial perturbations, are shown ( [1], [2], [3]) to
significantly affect the output of machine learning systems.
Neural network based models, despite their excellent performance, are observed ( [4], [5], [6]) to be vulnerable to
adversarial attacks. Particularly, Deep Convolutional Neural
Networks (CNN) based vision models ( [7], [8], [9], [10],
[11]) can be fooled by carefully crafted quasi-imperceptible
perturbations. Multiple hypotheses attempt to explain the
existence of adversarial samples, viz. linearity of the models [5], finite training data [12], etc. More importantly, the
adversarial perturbations generalize across multiple models. That is, the perturbations crafted for one model fools
another model even if the second model has a different
architecture or trained on a different subset of training data (
[4], [5]). This property of adversarial perturbations enables
potential intruders to launch attacks without the knowledge
about the target model under attack: an attack model typically known as black-box attacking [13]. On the contrast, an
attack model where everything about the target model is
known to the attacker is called a white-box attacking. Until
recently, all the existing works assumed a threat model in
which the adversaries can directly feed input to the machine
learning system. However, Kurakin et al. [14] lately showed
that the adversarial samples can remain misclassified even
if they were constructed in physical world and observed

•

The authors are with Department of Computational and Data Sciences,
Indian Institute of Science, Bangalore, India, 560012.
E-mail: kondamopuri@iisc.ac.in, adityaganeshan@gmail.com and
venky@iisc.ac.in

* denotes equal contribution
† https://github.com/val-iisc/gd-uap

through a sensor (e.g., camera). Given that the models are
vulnerable even in physical world scenario [14], the models’ susceptibility poses serious issues about their deployability (e.g., safety concerns for autonomous driving) in
the real world. Particularly, in case of critical applications
that involve safety and security, reliable models need to
be deployed to stand against the strong adversarial attacks.
Thus, the effect of these structured perturbations has to be
studied thoroughly in order to develop dependable machine
learning systems.
Recent work by Moosavi-Dezfooli et al. [8] presented the
existence of image-agnostic perturbations, called universal
adversarial perturbations (UAP) that can fool the state-ofthe-art recognition models on most natural images. Their
method for crafting the UAPs is based on the DeepFool [7]
attacking method. It involves solving a complex optimization problem (eqn. 2) to design a perturbation. The UAP [8]
procedure utilizes a set of training images to iteratively
update the universal perturbation with an objective of
changing the predicted label upon addition, for most of the
dataset images. Similar to [8], Metzen et al. [11] proposed
UAP for semantic segmentation task. They extended the
iterative FGSM [5] attack by Kurakin et al. [14] to change
the label predicted at individual pixels and craft the perturbation. They craft the image-agnostic perturbations to
fool the system in order to predict a pre-determined target
segmentation output.
However, these approaches to craft UAPs ( [8], [11], [15])
have the following important drawbacks:
•

Data dependency: It is observed that the objective
presented by [8] to craft UAP for recognition models
requires a minimum number of training samples

2

•

•

for it to converge and craft an image-agnostic perturbation. Moreover, the fooling performance of the
resulting perturbation is proportional to the available training data (Figure 4). Similarly, the objective
for crafting image-agnostic perturbation for semantic
segmentation models proposed by [11] also requires
data. Therefore, as their optimization/objectives involve data, existing procedures can not craft perturbations when enough data is not provided.
Weaker black-box performance: Since information about
the target models is generally not available for attackers, it is practical to study the black-box attacks.
Also, black-box attacks reveal the true susceptibility
of the models, while white-box attacks provide the
upper bound on the achievable fooling. However,
the black-box attacking performance of UAP [8] is
poor compared to their white-box performance (Table 4). Note that, in [8], authors have not analyzed
the performance of their perturbations in the blackbox attack scenario. They have assumed that the
training data of the target models is known and
have not considered the case in which adversary has
access to only a different set of data. This amounts
to performing only semi white-box attacks. Black-box
attacks generally mean [13] the adversary does not
have access to (i) the target network architecture
(including the parameters), and (ii) a large training
dataset. In case of semantic segmentation, as [11]
worked with targeted attacks, they observed that the
perturbations do not generalize to other models very
well.
Task specificity: The current objectives to craft UAPs
are task specific. The objectives are typically designed to suit the underlying task at hand as the concept of fooling varies across the tasks. Particularly,
for regression tasks such as depth estimation and
crowd counting, the idea of fooling is not straight
forward as it is for recognition task.

In order to address the above shortcomings and to better
analyze the stability of the models, we present novel datafree objective to craft universal adversarial perturbations.
Our objective is to craft image-agnostic perturbations that
can fool the target model without any knowledge about the
data distribution, such as, the number of categories, type
of data (e.g., faces, objects, scenes, etc.) or the data samples
themselves. Since we do not have access to any data, instead
of an objective that reduces the confidence to the predicted
label or flip the predicted label (as in [4], [7], [8], [11]), we
propose an objective to learn perturbations that can adulterate the features extracted by the models. Our proposed
objective attempts to over-fire the neurons at multiple layers
in order to deteriorate the extracted features. During the
inference time, the added perturbations misfire the neuron
activations in order to contaminate the representations and
eventually predict wrong label.
This work extends our earlier conference paper [9]. We
make the following new contributions in this paper:
1)

Propose a novel data-free objective for crafting
image-agnostic perturbations

2)
3)

4)

We demonstrate that our objective is generalizable
across multiple vision tasks
Further, we show that apart from being data-free
objective, the proposed method can exploit minimal
prior information about the training data distribution of the target models in order to craft stronger
perturbations
We present comprehensive analysis on the proposed
objective and the crafted perturbations across three
different vision tasks covering both classification
and regression tasks.

The rest of this paper is organized as follows: section
2 presents detailed account of related works, section 3
discusses the proposed data-free objective to craft imageagnostic adversarial perturbations, section 4 demonstrates
the effectiveness of our perturbations via comprehensive
experimentation, section 5 hosts discussion on the proposed
method and finally section 6 concludes the paper.

2

R ELATED W ORKS

Szegedy et al. [4] demonstrated that despite their superior
recognition performance, neural networks are susceptible
to adversarial perturbations. Subsequently, multiple other
works [5], [6], [7], [16], [17], [18], [19] studied this interesting
and surprising property of the machine learning models.
Though it is first observed with recognition models, the
adversarial behaviour is noticed with models trained on
other tasks such as semantic segmentation [11], [20], object
detection [20] and deep reinforcement learning tasks [21],
etc. There exist multiple methods to craft these malicious
perturbations for a given data sample. For recognition tasks,
they range from performing simple gradient ascent ( [5])
on cost function to solving complex optimizations ( [4],
[7], [22]). Simple and fast methods such as FGSM [5] find
the gradient of loss function to determine the direction in
the image space to perturb the input image. An iterative
version of this attack presented in [14] achieves better fooling via performing the gradient ascent multiple times. On
the other hand, complex approaches such as [4], [7] find
minimal perturbation that can move the input across the
learned classification boundary in order to flip the predicted
label. More robust adversarial attacks have been proposed
recently that transfer to real world [14] and are invariant to
general image transformations [23].
Moreover, it is observed that the perturbations exhibit
transferability property. That means, perturbations crafted
for one model can fool other models with different architectures and trained on a disjoint training set as well ( [4], [5]).
Further, Papernot et al. [13] introduced a practical attacking
setup via model distillation to understand the black-box attacking. Black-box attacking assumes no information about
the target model and its training data. They proposed to use
a target model’s substitute to craft the perturbations.
The common underlying aspect of all these techniques is
that they are intrinsically data dependent. The perturbation
is crafted for a given data sample independently of others.
However, recent works by Moosavi-Dezfooli et al. [8] and
Metzen et al. [11] showed the existence of input-agnostic
perturbations that can fool the models over multiple images.
In [8], authors proposed an iterative procedure based on

3

3

P ROPOSED A PPROACH

In this section, we discuss the proposed data-free objective
to craft UAPs in detail.
First, we introduce the notation followed throughout the
paper. X denotes the distribution of images in Rd . f denotes
the function learned by the CNN that maps an input image
x ∼ X to its output f (x). Note that the output is task
dependent, for example, it is a label for object recognition
and segmentation map for semantic segmentation. δ denotes
the image-agnostic perturbation learned by our objective.
Note that similar to input x, δ also belongs to Rd . Though
the proposed objective is task independent, for ease of
understanding we explain the proposed approach in the
context of object recognition.
3.1

Data-free objective for fooling

The objective of our paper is to craft an image-agnostic
perturbation δ ∈ Rd that fools the CNN f for images
from the target distribution X without utilizing any samples
from it. In other words, we seek a universal adversarial
perturbation δ that significantly alters the prediction of the
CNN (f ). That is, we synthesize a δ such that

f (x + δ) 6= f (x), for x ∼ X .

(1)

8

8

In order for the δ to be an adversarial perturbation, it has
to be imperceptible when added to the images. Therefore
the pixel intensities of δ are restricted by an imperceptibility
constraint. Typically, it is realized as a max-norm constraint
in terms l or l2 norms (e.g. [5], [8], [9], [11]). In this paper,
for all our analysis we impose max-norm constraint in terms
of l norm. Thus, the aim is to find a δ such that
(2)

8

f (x + δ) 6= f (x), for x ∈ X ;
k δ k < ξ.

However, the focus of the proposed work is to craft the
image-agnostic perturbations without requiring any data
samples (not only from the training dataset X on which
the target model f is learned). The data-free nature of our
approach prohibits us from optimizing for the first part
of eqn. 2 while learning δ . That is, our approach does
not contain the data term x in the proposed objective.
Therefore, we propose to fool the CNN by contaminating
the extracted representations of the input at multiple layers
of the architecture. In other words, as opposed to the typical
“flipping the label” objective, we attempt to over-fire the

features extracted at multiple layers. That is, we craft a
perturbation δ such that it leads to additional activation
firing at each layer and thereby misleading the features
(filters) at the following layer. The accumulated effect of the
contamination eventually leads the CNN to misclassify the
input.
The perturbation essentially causes filters at a particular layer to spuriously fire and abstract out inefficacious
information. Note that in the presence of data (during
attack), in order to mislead the activations from retaining
useful discriminative information, the perturbation (δ) has
to be highly effective. Also, the imperceptibility constraint
(second part of eqn. 2) on δ makes it more challenging.
Hence without utilizing any data (x), we seek for an
image-agnostic perturbation δ that can produce maximal
spurious activations at each layer of a given CNN. In order
to craft such a δ we start with a random perturbation and
optimize for the following objective
!
K
Y
Loss = − log
kli (δ)k2 , such that k δ k < ξ.
8

Deepfool attacking [7] method to craft a universal perturbation to fool classification models. Similarly, in [11], authors
craft universal perturbations that can result in target segmentation outputs. However, both these works optimize for
different task specific objectives. Also, they require training
data to craft the image-agnostic perturbations. Unlike the
existing works, the proposed method presents a data-free
objective that can craft perturbations without the need for
any data samples. Also, we introduce a generic notion of
fooling across multiple computer vision tasks via over-firing
the neuron activations. Particularly, our objective is generalizable across various vision models in spite of differences in
terms of architectures, regularizers, underlying tasks, etc.

i=1

(3)
where li (δ) is the activation in the output tensor at layer
i when δ is fed to the network f . Note that the activations
are considered after the non-linearity (typically ReLU). K is
the total number of layers in f at which we maximize the
activations caused by perturbation δ , and ξ is the max-norm
limit on δ .
The proposed objective computes product of activation
magnitude at all the individual layers in order to simultaneously maximize the interference at all layers. We observed product resulting in stronger δ than other forms
of combining (e.g. sum) individual layer activations. This
is understandable, since product can force the individual
activations to rise in order for the loss to reduce. To avoid
working with extreme values (≈ 0), we apply log on the
product. Note that the objective is open-ended as there is no
optimum value to reach. We would ideally want δ to cause
as much strong disturbance at all the layers as possible,
while respecting the imperceptibility constraint.
3.2

Implementation Details

We begin with a target network f which is a trained CNN
whose parameters are frozen and a random perturbation
δ . We then perform the proposed optimization to update
δ for causing strong activations at multiple layers in the
given network. We typically consider all the convolution
(conv) layers before the fully connected (f c) layers. This is
because, the conv layers are generally considered to learn
required features to extract information over which a series
of f c layers perform classification. Also, we empirically
found that it is efficient to optimize at conv layers. Therefore,
we restrict the optimization to feature extraction layers. In
case of advanced architectures such as GoogLeNet [24] and
ResNet [25], we optimize at the last layers of all the inception blocks (or residual blocks) and the independent conv
layers. We observed that optimizing at these layers results
in δ with a fooling capacity similar to the one resulting from
optimizing at all layers (including the conv layers within
the inception/residual blocks). However, since optimizing

4

at only the last layers of inception/residual blocks along
with independent conv layers is slightly more efficient, we
perform the same.
Note that the optimization updates only the perturbation
δ , not the network parameters. Additionally, no image data
is involved in the optimization process. We update δ with
the gradients computed for loss in eqn. (3) iteratively till
the fooling performance of the learned δ gets saturated on
a set of validation images. In order to validate the fooling
performance of the learned δ , we compose an unrelated
substitute dataset (D). Since our objective is not to utilize
data samples from the training dataset of the target models,
we randomly select 1000 images from a substitute dataset
to serve as validation images. Note that, it is a reasonable
assumption for an attacker to have access to 1000 unrelated images. For crafting perturbations to object recognition
models trained on ILSVRC dataset [26], we choose samples
from Pascal VOC-2012 [27] dataset. Similarly, for semantic
segmentation models trained on Pascal VOC [27], [28], we
choose validation samples from ILSVRC [26], for depth
estimation models trained on KITTI dataset [29] we choose
samples from Places-205 [30] dataset.

3.3.2 Target data samples
In this subsection, we formulate our data-free objective to
utilize samples from the target distribution X and benefit
to improve the fooling ability of the crafted perturbations.
Note that in the case of data availability, we can design
direct objectives such as reducing confidence for the predicted label or changing the predicted label, etc. However,
we investigate if our data-free objective of over-firing the
activations, though is not designed to utilize data, crafts
better perturbations when data is presented to the optimization. Additionally, our objective does not utilize data
to manipulate the predicted confidences or labels. Rather,
the optimization benefits from prior information about the
data distribution such as the dynamic range, local patterns,
etc., which can be provided through the actual data samples.
Therefore, with minimal data samples we solve for the
following optimization problem

Exploiting additional priors

Though the proposed method is a data-free optimization for
crafting image-agnostic perturbations, it can exploit simple
additional priors about the data distribution X . In this
section we demonstrate how our method can utilize simple
priors such as (i) mean value and dynamic range of the
input, and (ii) target data samples.
3.3.1 Mean and dynamic range of the input
Note that the proposed optimization (eqn. (3)) does not
consider any information about X . We present only the
norm limited δ as input and maximize the resulting activations. That is, during the optimization, input to the target
CNN has a dynamic range of [−10, 10]. However, during
the inference time, input lies in [0, 255] range. Therefore,
it becomes very challenging to learn perturbations that
can affect the neuron activations in the presence of strong
(approximately, an order higher) input signal x. Hence, in
order to make the learning easier, we provide this useful
information about the data (x ∈ X ), and let the optimization
better explore the space of perturbations. Thus, we slightly
modify our objective to craft δ relative to the dynamic range
of the data. We create pseudo data d via randomly sampling
from a Gaussian distribution whose mean (µ) is equal to the
mean of training data and variance (σ) that covers 99.9%
of density to lie in [0, 255], the dynamic range of input.
Essentially, we solve for the following

Loss = − log

K
Y

!

kli (d + δ)k2 ,

i=1

8

such that kδ k

(4)

< ξ, and d ∼ N (µ, σ).

Essentially, we operate the proposed optimization in a closer
subspace to the target data distribution X . In other words,
d in eqn. (4) acts as a place holder for the actual data
and helps to learn perturbations which can over-fire the
neuron activations in the presence of the actual data. For

Loss = − log

K
Y

!

kli (x + δ)k2 ,

i=1

such that kδ k

8

3.3

each iteration of the optimization process, we sample a
new d from the normal distribution and feed it through the
target CNN. We also perform typical augmentations such as
flipping, cropping and slight rotation, etc. on the sampled d
to construct variations.

(5)

< ξ, and x ∼ X .

Note that presenting data samples to the optimization procedure is a natural extension to presenting the dynamic
range of the target data alone (section 3.3.1). In this case,
we utilize a subset of training images on which the target
CNN models are trained (similar to [8], [11]).
3.4

Improved Optimization

In this subsection, we present the improvements we propose
to our earlier optimization presented in our conference paper [9]. The proposed objective in [9] is observed to quickly
accumulate δ beyond the imposed max-norm constraint (ξ).
Because of the clipping performed after each iteration, the
updates will be futile after δ reaching the constraint. In
order to tackle this saturation, δ is re-scaled to half of its
dynamic range (i.e. [−5, 5]) in regular time intervals of
300 iterations. Though this re-scaling helps to learn better
δ , it is inefficient since it performs blind re-scaling without
verifying the scope for updating the δ . For example, as the
learning progresses, magnitude of updates decreases and
during the interval of 300 iterations, the values of δ might
not reach the extreme values of ±10. Projecting the δ by
re-scaling can badly affect the learning.
Therefore, we propose an adaptive re-scaling of δ based
on the rate of saturation (reaching the extreme values of
±10) in its pixel values. During the optimization, at each
iteration we compute the proportion (p) of the pixels in
δ that reached the max-norm limit ξ . As the learning progresses, since our objective is open ended, more number of
pixels reach the max-norm limit and because of the clipping
eventually get saturated at ξ . Hence, the rate of increase in p
decreases as δ saturates. We compute the rate of saturation,
denoted as S , of the pixels in δ after each iteration during

5

the training. For consecutive iterations, if increase in p
is not significant (less than a pre-defined threshold T h),
we perform a re-scaling to half the dynamic range. Note
that the proposed criterion for re-scaling is similar to the
typical usage of validation performance to stop training. We
observe that the proposed adaptive re-scaling consistently
leads to better learning.
3.5

Algorithmic summarization

In this subsection, for the sake of brevity we summarize the
proposed data-free objective in the form of an algorithm.
Algorithm 1 presents the proposed optimization as a series
of steps. Note that it is a generic form comprising of all
the three variations including both data-free and with prior
versions.
For ease of reference, we repeat some of the notation. Ft
is the fooling rate at iteration t, li (x) is the activation caused
at layer i of the CNN f for an input x, η is the learning rate
used for training, ∆ is the gradient of the loss with respect
to the input δ , St is the rate of saturation of pixels in the
perturbation δ at iteration t, T h is the threshold on the rate
of saturation, Ft is the fooling rate, H is the patience interval
of validation for verifying the convergence of the proposed
optimization.
Algorithm 1: Algorithm summarizing our approach
to craft image-agnostic adversarial perturbations via
data-free objective and exploiting various data priors.

1
2
3
4
5
6
7

Generalized Fooling Rate (GFR), making it independent
of the task, and dependent on the metric being used for
evaluating the model’s performance.
Consider the task of image recognition. For this task,
fooling rate for any perturbation is defined as the % of data
samples, for which the labels are changed due to the perturbation. i.e., the % of data samples for which the output label
before and after the adversarial attack are different. If the
fooling rate is x%, then for (100 − x)% of the data samples,
the label before and after the adversarial attack remains the
same. If we consider the labels of clean samples as ground
truth, and labels of perturbed samples as predicted labels,
(100 − x)% is same as the Top-1 accuracy of the model.
This leads us to introduce the following definition for the
Generalized Fooling Rate.
Let M be a metric for measuring the performance of a
model for any task, where the range of M is [0, R]. Let the
metric take two inputs ŷ and y , where ŷ is the predicted
output and y is the ground truth output, such that the
performance of the model is measured as M (ŷ, y). Let yˆδ be
the output of the model when the input is perturbed with
a perturbation δ . Then, the Generalized Fooling Rate with
respect to measure M is defined as:

R − M (yˆδ , ŷ)
(6)
.
R
This definition of Generalized Fooling rate (GFR) has the
following benefits:
GF R(M ) =

•

Data: Target CNN f , data g . Note that g = 0 for
data-free case, g = d ∼ N (µ, σ) for range prior
case, and g = x for training data samples case.
Result: Image-agnostic adversarial perturbation δ .
Randomly initialize δ0 ∼ U[−ξ, ξ]
t=0
Ft = 0
do
t←t+1
Compute li (g + δ)
K

Q
kli (g + δ)k2
Compute the loss = − log

•

i=1

8
9
10
11
12
13
14
15
16

Update1 δt : δt ← δt−1 − η∆
Compute the rate of saturation St in the δt pixels
if St < T h then
δt ← δt /2
end
Compute Ft of δt on substitute dataset D
while Ft < min. of {Ft−H , Ft−H+1 . . . Ft−1 };
i ← argmax. of {Ft−H , Ft−H+1 . . . Ft−1 }
Return δi

Fooling rate should be a measure of the change in
model’s output caused by the perturbation. Being
independent of the ground truth y , and dependant
only on yˆδ and ŷ , GFR primarily measures the change
in the output. Other methods such as the metric
value after the attack may not properly capture this.
A poorly performing model which however is very
robust to adversarial attacks will show very poor
GFR values, highlighting its robustness. For example,
GFR will be equal to 0% if there is no change in the
output, and in extreme cases it can reach 100%, for a
given task. Hence it can be compared across tasks.
GFR measures the performance of a perturbation
in terms of the damage caused to a model with
respect to a metric. This is an important feature, as
tasks such as depth estimation have multiple performance measures, where some perturbation might
cause harm only to some of the metrics while leaving
other metrics unaffected.

For all the tasks considered in this work, we report GFR
with respect to a metric, as a measure of ‘fooling’.

1 Note

that this generic update equation is only for understanding
purpose and not the exact equation implemented.

3.6 Generalized Fooling Rate (GFR) with respect to a
metric
While the notion of ‘fooling’ has been well defined for the
task of image recognition, for other tasks, how to measure the ‘fooling’ is unclear. Hence, in order to provide
an interpretable metric to measure ‘fooling’, we introduce

4

E XPERIMENTS

In this section, we present the experimental evaluation to
demonstrate the effectiveness of the proposed data-free
objective. We consider three different vision tasks to demonstrate the generalizability of our objective, namely, object recognition, semantic segmentation and unsupervised
monocular depth estimation. Note that the set of applications include both classification and regression tasks. Also,
it has both supervised and unsupervised learning setups.

6

TABLE 1
Fooling rates for the proposed data-free objective learned for object recognition on ILSVRC dataset [26]. Each row of the table shows the fooling
rates for the perturbation learned on a specific target model when attacking various other models (columns). These rates are obtained by the
proposed objective without utilizing the data samples but only the range prior (sec. 3.3.1). Diagonal rates indicate the white-box attack scenario
and off-diagonal ones represent the black-box attack scenario. Note that the highest fooling rate when attacking a given model (column) is shown
in bold.
Model
CaffeNet
VGG-F
GoogLeNet
VGG-16
VGG-19
Resnet-152

CaffeNet

VGG-F

CaffeNet

VGG-F

GoogLeNet

VGG-16

VGG-19

Resnet-152

87.02
59.89
44.70
50.05
49.11
38.41

65.97
91.91
46.09
55.66
53.45
37.20

49.40
52.24
71.44
46.59
40.90
33.22

50.46
51.65
37.95
63.08
55.73
27.76

49.92
50.63
37.90
56.04
64.67
26.52

38.57
40.72
34.56
36.84
35.81
37.3

GoogleNet

VGG-16

VGG-19

ResNet-152

Fig. 1. Universal adversarial perturbations crafted by the proposed data-free objective for multiple models trained on ILSVRC [26] dataset.
Perturbations were crafted with ξ = 10 using the minimal prior, i.e. mean and dynamic range of the input samples (sec. 3.3.1). Corresponding
target network architecture is mentioned below each perturbation. Images are best viewed in color.

We explain each of the tasks separately in the following
subsections.
For all the experiments, the ADAM [31] optimization
algorithm is used with the learning rate of 0.1. The threshold
T h for the rate of saturation S is set to 10−5 . Validation
fooling rate Ft is measured on the substitute dataset D after
every 200 iterations only when the threshold of the rate of
saturation is crossed. If it is not crossed, Ft is still measured
after every 400 iterations. Note that none of the algorithm
specific hyper-parameters are changed across tasks or across
prior scenarios.
4.1

Object recognition

We have worked with models trained on ILSVRC [26]
and Places-205 [30] datasets, viz. CaffeNet [32], VGGF [33], GoogLeNet [24], VGG-16 [34], VGG-19 [34], ResNet152 [25]. Since our approach does not involve training the
models, for all the experiments we work with available
trained models. Also, unlike UAP [8], as we do not use
training data in the data-free case (sec. 3.1 and sec. 3.3.1), no
training data is used. However, as explained earlier, we use
1000 images randomly chosen from Pascal VOC-2012 [27]
train images as validation set (D in Algorithm 1) for our
optimization. Also, in case of exploiting additional data
prior (sec. 3.3.2), we use limited data from the corresponding training set. However, for evaluating the perturbations
learned for the target models trained on ILSVRC dataset,
50000 images from the validation set are used. Similarly,
Places-205 dataset contains 20500 validation images from
205 categories, over which the fooling rates are computed.
4.1.1 Fooling performance of the data-free objective
Table 1 presents the fooling rates achieved by our objective
on various network architectures. Fooling rate is the per-

centage of test images for which our crafted perturbation δ
successfully changed the predicted label. Higher the fooling
rate, greater is the perturbation’s ability to fool and lesser
is the the classifier’s robustness. Fooling rates in Table 1
are obtained using the mean and dynamic range prior of
the training distribution (sec. 3.3.1). Each row in the table
indicates one target model employed in the learning process
and the columns indicate various models attacked using the
learned perturbations. The diagonal fooling rates indicate
the white-box attacking, where all the information about the
model is known to the attacker. The off-diagonal rates
indicate black-box attacking, where no information about the
model under attack is revealed to the attacker. However,
note that the dataset over which both the models (target
CNN and the CNN under attack) are trained is same.
Our perturbations cause a mean white-box fooling rate of
69.24% and a mean black-box fooling rate of 45.13%. Note
that, given the data-free nature of the optimization, the
fooling rates are alarmingly significant. These high fooling
rates achieved by the proposed approach can adversely
affect the real-world deployement of these models.
Figure 1 shows example image-agnostic perturbations
(δ) crafted by the proposed method. Note that the perturbations look very different for each of the target CNNs. However, the perturbations corresponding to the VGG models
look slightly similar owing to their architectural similarity. Figure 3 shows sample perturbed images (x + δ) for
VGG-19 [34] from ILSVRC [26] validation set. The top row
shows the clean and bottom row shows the corresponding
adversarial images. Note that the adversarially perturbed
images are visually indistinguishable form their corresponding clean images. All the clean images shown in the figure
are correctly classified and are successfully fooled by the
added perturbation. Below each image, the corresponding

7

TABLE 2
Fooling rates achieved by the proposed objective with and without utilizing the prior information about the training distribution of the models. For
comparison, we provide the random baseline (U [−ξ, ξ]), existing data-free [9] and data dependent [8] objectives. Note that the best fooling rate for
a given model (row) is shown in bold.
Model
CaffeNet
VGG-F
Googlenet
VGG-16
VGG-19
Resnet-152

Baseline

No prior

Range prior

Data prior

FFF [9]

UAP [8]

12.9
12.62
10.29
8.62
8.40
8.99

84.88
85.96
58.62
45.47
40.68
29.78

87.02
91.81
71.44
63.08
64.67
37.3

91.54
92.64
83.54
77.77
75.51
66.68

80.92
81.59
56.44
47.10
43.62
-

93.1
93.8
78.5
77.8
80.8
84.0

VGG16 - no data VGG16 - min. prior

VGG16 - data

Res-152-no data

Res-152-min. prior

Res-152-data

Fig. 2. Sample universal adversarial perturbations crafted by the proposed method under multiple settings for a pair of models trained on
ILSVRC [26] dataset. Perturbations were crafted with ξ = 10 for no data, minimal data prior and with data scenarios. First three perturbations
are crafted for VGG-16 [34] and the later three for ResNet-152 [25]. Corresponding setting is mentioned below each perturbation. Images are best
viewed in color.

label predicted by the model is shown. Note that the correct
labels are shown in black color and the wrong ones are
shown in red.
4.1.2

Exploiting the minimal prior

In this section, we present experimental results to demonstrate how our data-free objective can exploit the additional
prior information about the target data distribution as discussed in section 3.3. Note that we consider two cases: (i)
providing the mean and dynamic range of the data samples,
denoted as range prior (sec. 3.3.1), and (ii) utilizing minimal
data samples themselves during the optimization, denoted
as data prior ( 3.3.2).
Table 2 shows the fooling rates obtained with and without utilizing the prior information. Note that all the fooling
rates are computed for white-box attacking scenario. To
emphasize the effectiveness of the proposed objective, we
present fooling rates obtained by a random baseline perturbation. Since our learned perturbation is norm limited by ξ ,
we sample random δ from U[−ξ, ξ] and compute the fooling
rates. We denote these results in the ‘Baseline’ column of
Table 2. Also, for comparison, fooling rates obtained by our
data-free objective [9] and a data dependent objective [8] are
presented. Important observations to draw from the table
are listed below:
•
•
•

Utilizing the prior information consistently improves
the fooling ability of the crafted perturbations.
A simple range prior can boost the fooling rates on
an average by an absolute 10%.
Given the proposed objective is not designed to
utilize the data, feeding the data samples results in
an absolute 22% rise in the fooling rates. For some
of the models (e.g. GoogLeNet and VGG-16) our
method performs equally (or even outperforms) to

TABLE 3
Comparison of data-free objectives. Fooling rates achieved by the
proposed objective of maximizing the l2 norm of the activations caused
by δ versus the mean activation [9] when utilizing data samples. Note
that the proposed objective consistently outperforms the previous
objective across multiple architectures (shown in bold).
Model
CaffeNet
VGG-16
Resnet-152

•

4.1.3

Mean activation [9]

l2 norm

88.35
72.68
65.43

91.54
77.77
66.68

an objective [8] designed especially to utilize data
and flip the predicted labels.
Our improved objective (eqn.3) and the optimization
procedure (3.4) result in an absolute increase of fooling rate by 22.36% in the case of utilizing data.
Comparison of the data-free objectives

In this subsection we compare the effectiveness of the
proposed objective against the existing data-free objective
proposed in our conference publication [9]. That is, we
compare maximizing the mean versus l2 norm (energy) of
the activations caused by the perturbation δ (or x/d + δ in
case of exploiting the additional priors).
Table 3 shows the comparison of fooling rates obtained
with both the objectives (separately) in the improved optimization setup (3.4). We have chosen 3 representative
models across various generations of architectures (CaffeNet, VGG and ResNet) to compare the effectiveness of
the proposed objective. Note that the improved objective
consistently outperforms the previous one by a significant
3.18%. Similar behaviour is observed for other vision tasks
also.

8

Missile

Banana

Tank

Banjo

Skunk

French loaf

African
chamaelon

Custard Apple

Half track

Vault

Jigsaw puzzle

Normal chiton

Fig. 3. Sample original and adversarial image pairs from ILSVRC validation set generated for VGG-19. First row shows original images and
corresponding predicted labels, second row shows the corresponding perturbed images along with their predictions. Note that all of the shown
perturbed images were misclassified.

TABLE 4
Effect of data dependency on crafting the perturbations. Data
dependent objectives [8] suffer significant drop in fooling ability when
arbitrary data samples are utilized for crafting image-agnostic
perturbations. Note that, A→ B denotes that data A is used to craft
perturbations to fool models trained on data B. Also note that fooling
rates for our approach are crafted without utilizing any data samples
(denoted with ∗ ).
Model

Places-205 → ILSVRC
Ours

UAP [8]

Ours

UAP [8]

CaffeNet
GoogLenet

87.02*
71.44*

73.09
28.17

88.61*
83.37*

77.21
52.53

4.1.4

ILSVRC → Places-205

Data dependent vs. Data-free objectives

In this subsection we demonstrate the necessity of data
dependent objective [8] to have samples from the target
distribution only. That is, methods (such as [8]) that craft
perturbations with fooling objective (i.e. move samples
across the classification boundaries) require samples from
only the training data distribution during the optimization.
We show that crafting with arbitrary data samples leads to
significantly inferior fooling performance.
Table 4 shows the fooling rates of data dependent objective [8] when arbitrary data samples are utilized in place
of target samples. Experiment in which we use Places-205
data to craft perturbations for models trained on ILSVRC is
denoted as Places-205 → ILSVRC and vice versa. For both
the setups, a set of 10000 training images are used. Note
that, the rates for the proposed method are obtained without
utilizing any data and rates for data-free scenario can be
found in Table 2. Clearly the fooling rates for UAP [8] suffer
significantly, as their perturbations are strongly tied to the
target data. On the other hand, for the proposed method,
since it does not craft via optimizing a fooling objective, the
fooling performance does not decrease. Importantly, these
experiments show that the data dependent objectives are not
effective when samples from the target distribution are not

Fig. 4. Reliance of the data dependent objective UAP [8] on the size
of available training data samples. Fooling rate of the crafted perturbations monotonically increases with the number of training data samples
available during the optimization. Note that our approach utilizes no data
samples and achieves competitive fooling performance.

available. In most of the practical scenarios it is difficult to
procure the same training data. Also, as the data dependent
objectives rely on the available training data, the ability
of the crafted perturbations heavily depends on the size
of the data utilized in the crafting process. We show that
the fooling performance of UAP [8] monotonically increases
with the size of the available samples for optimization.
Figure 4 shows the fooling rates obtained by the perturbations crafted for multiple recognition models trained on
ILSVRC by UAP [8] with varying size of samples available
for optimization. We craft different UAP [8] perturbations
(using the codes provided by the authors) utilizing only 500,
1000, 2000, 4000 and 10000 data samples and evaluate their
ability to fool various models. Note that the performance
of the crafted perturbations increases drastically (shown in
different shades of blue) with the available data samples
during the optimization. For comparison, the fooling rates
obtained by the proposed data-free objective is shown in
green.

9

TABLE 5
Comparison of mean IOU obtained by various models against the proposed perturbations. Comparison with image specific adversaries [20] is also
presented. Note that ∗ denotes being image-specific and † denotes a transfer attack (black-box attacking). Note that being image specific, [20]
outperforms our perturbations, however, even our no data perturbations cause more drop in mIOU than their transfer perturbations.

Model
FCN-Alex
FCN-8s-VGG
DL-VGG
DL-RN101

Original

Baseline

No
Data

Range
prior

All
data

Data w/
less BG

ICCV 2017 [20]

46.21
65.49
62.10
74.94

45.72
64.34
61.13
73.42

15.35
42.78
36.91
56.40

10.37
39.08
35.41
58.66

10.64
33.61
44.90
37.45

8.03
28.05
27.41
39.00

3.98*
4.02*
43.96∗†
73.01∗†

TABLE 6
Generalized fooling rates achieved by the perturbations crafted by the
proposed approach under various settings. Note that for comparison,
fooling rates achieved by random perturbations are also presented.

Model
FCN-Alex
FCN-8s-VGG
DL-VGG
DL-RN101

4.2

Baseline
14.29
9.24
10.66
8.8

No
Data
80.15
49.42
55.90
37.06

Range
Prior
86.57
55.04
58.96
35.6

All
Data
85.96
61.15
44.82
58.62

Data W/
less BG
89.61
67.19
66.68
56.61

Semantic segmentation

In this subsection we demonstrate the effectiveness of our
data-free objective to craft universal adversarial perturbations for semantic segmentation. We consider four network
architectures. The first two architectures are from FCN [35]:
FCN-Alex, based on Alexnet [32], and FCN-8s-VGG, based
on the 16-layer VGGNet [34]. The last two architectures are
16-layer VGGNet based DL-VGG [36], and DL-RN101 [37],
which is a multi-scale architecture based on ResNet-101 [25].
The FCN architetures are trained on Pascal VOC-2011
dataset [28], [38], consisting 9610 training samples and the
the remaining two architectures are trained on Pascal VOC2012 dataset [27], [38], consisting 10582 training samples.
However, for testing our perturbation’s performance, we
only use the validation set provided by [35], which consist
of 736 images.
Semantic segmentation is realized as assigning a label to
each of the image pixels. That is, these models are typically
trained to perform pixel level classification into one of
21 categories (including the background) using the crossentropy loss. Performance is commonly measured in terms
of mean IOU (intersection over union) computed between
the predicted map and the ground truth. Extending the UAP
generation framework provided in [8] to segmentation is a
non-trivial task. However, our generalizable data-free algorithm can be applied for the task of semantic segmentation
without any changes.
Similar to recognition setup, we present multiple scenarios for crafting the perturbations ranging from no data to
utilizing data samples from the target distribution. An interesting observation with respect to the data samples from
Pascal VOC-2012, is that, in the 10582 training samples,
65.4% of the pixels belong to the ‘background’ category.
Due to this, when we craft perturbation using training
data samples as target distribution prior, our optimization process encounters roughly 65% pixels belonging to

‘background‘ category, and only 35% pixels belonging to
the rest 20 categories. As a result of this data imbalance,
the perturbation is not sufficiently capable to corrupt the
features of pixels belonging to the categories other than
‘background’. To handle this issue, we curate a smaller set
of 2833 training samples from Pascal VOC-2012, where each
sample has less than 50% pixels belonging to ‘background’
category. We denote this as “data with less BG”, and only
33.5% of pixels in this dataset belong to the ‘background’
category. The perturbations crafted using this dataset as
target distribution prior show a higher capability to corrupt
features of pixels belonging to the rest 20 categories. Since
mean IOU is the average of IOU for each of the 21 categories,
we further observe that perturbations crafted using “data
with less BG” cause a substantial reduction in the mean IOU
measure as well.
Table 6 shows the generalized fooling rates with respect
to the mean IOU obtained by the proposed objective under
various data priors. As explained in section 3.6, the generalized fooling rate measures the change in the performance of
a network with respect to a given metric, which in our case
is the mean IOU. Note that, similar to the recognition case,
the fooling performance monotonically increases with the
addition of data priors. This observation emphasizes that
the proposed objective, though being an indirect and nonfooling in nature, can rightly exploit the additional prior
information about the training data distribution. Also, of all
the models, generally, data with less background scenario
results in the best fooling rate. This can be attributed to the
fact that in data with less background scenario we lessen
the data-imbalance that leads to perturbations that can fool
both background and object pixels. Also, for comparison, we
present the fooling rates achieved by the random perturbation sampled from U[−ξ, ξ]. Note that similar to recognition
experiments, for all the segmentation experiments we use a
ξ value of 10.
In Table 5 we present the mean IOU metric obtained
on the perturbed images learned under various scenarios
along with original mean IOU obtained on clean images.
Comparison with baseline random noise perturbation sampled from U[−10, 10] is also provided. It is clearly observed
that the random perturbation is not effective in fooling
the segmentation models. However, the proposed objective
crafts perturbations within the same range that can significantly fool the models. We also show the mean IOU
obtained by Xie et al. [20], an image specific adversarial
perturbation crafting work. Note that since the proposed
method is an image-agnostic approach it is unfair to expect

10

FCN-Alex

FCN-8s-VGG

DL-VGG

DL-RN101

Fig. 5. Universal adversarial perturbations for semantic segmentation, crafted by the proposed data-free objective for multiple models. Perturbations
were crafted with ξ = 10 using the less BG prior. Corresponding target network architecture is mentioned below each perturbation. Images are best
viewed in color.

No prior

Range prior

less BG prior

All data prior

Fig. 6. Sample universal adversarial perturbations for semantic segmentation, crafted by the proposed method under multiple settings for FCN-8sVGG. Perturbations were crafted with ξ = 10 for no prior,range prior, data with less BG prior, and all data prior scenarios. Corresponding scenario
is mentioned below each perturbation. Images are best viewed in color.

Clean

No prior

Range Prior

Less BG prior

All data Prior

Fig. 7. Sample original and adversarial images from PASCAL-2011 dataset generated for FCN-Alex. First row shows clean and adversarial images
with various priors. Second row shows the corresponding predicted segmentation maps.

similar performance as [20]. Further, the mean IOU shown
by [20] for DL-VGG and DL-RN101 models (bottom 2 rows
of Table 5) denote the transfer performance, i.e., black-box
attacking and hence the smaller drop of the mean IOU from
that of clean images. However, they are provided as a means
of weak comparison instead of no comparison.
Finally, we end this section by providing some qualitative results. Figure 5 and 6 shows sample image-agnostic
adversarial perturbations learned by our objective for semantic segmentation. In Figure 5 we show the perturbations
learned with “data with less BG” prior for all the models.
Similar to the recognition case, these perturbations look
different across architectures. In Figure 6, we show the perturbations learned for FCN-8s-VGG model under various
scenarios ranging from no prior to full data prior. Note
that, even for a given network, the perturbations learned
with various priors look different. Figures 7 shows example

image and predicted segmentation outputs by FCN-Alex
model for perturbations crafted with various priors. Top
row shows the clean and the perturbed images. Bottom row
shows the predictions for the corresponding inputs. Note
that the type of prior utilized to craft the perturbation is
mention below the predictions. Crafted perturbations are
clearly successful in misleading the model to predict inaccurate segmentation maps. Note that the color map shown
below the predictions provides the labels.
Figure 8 shows the effect of perturbation on multiple
networks. It shows the output maps predicted by various
models for the same input perturbed with corresponding
δ learned with “data with less BG” prioir. It is interesting
to note from Figure 8 that for the same image, with UAPs
crafted using the same prior, different networks can have
very different outputs, even if their outputs for clean images
are very similar.

11

FCN-Alexnet

FCN8S-VGG16

ResNet-MSC

VGG-16

Fig. 8. Segmentation predictions of multiple models over a sample perturbed image. Perturbations were crafted with ξ = 10 using the “data with
less BG” prior. The first row shows the perturbed input image, the second shows the segmentation output of clean sample image, and the third
shows the segmentation output of the perturbed sample image. The corresponding target network architecture is mentioned below each column.
Images are best viewed in color.

4.3

Depth estimation

Recent works such as [39], [40], [41] show an increase in use
of convolutional networks for regression-based Computer
Vision task. A natural question to ask is whether they are
as susceptible to universal adversarial attacks, as CNNs
used for classification. In this section, by crafting UAPs
for convolutional networks performing regression, we show
that they are equally susceptible to universal adversarial
attacks. To the best of our knowledge, we are the first
to provide an algorithm for crafting universal adversarial
attacks for convolutional networks performing regression.
Many recent works like [39], [42], [43] perform depth
estimation using convolutional network. In [39], the authors introduce Monodepth, an encoder-decoder architecture which regresses the depth of given monocular input image. We craft UAP using the proposed method for
the two variants of Monodepth, Monodepth-VGG and
Monodepth-Resnet50. The network is trained using KITTI
dataset [29]. In its raw form, the dataset contains 42, 382
rectified stereo pairs from 61 scenes, with a typical image
being 1242×375 pixels in size. We show results on the eigen

split, introduced in [44], which consist of 23488 images for
training and validation, and 697 images for test. We use
the same crop size as suggested by the authors of [44] and
evaluate at the input image resolution.
As in the case of object recognition, UAPs crafted by the
proposed method for monodepth also show the potential to
exploit priors about the data distribution. We consider three
cases, (i) providing no priors, (ii)range prior, and (ii) data
prior. For providing data priors, we randomly pick 10000
image samples from the KITTI train dataset. To attain complete independence of target data in the training procedure,
while crafting the perturbation, we perform validation on a
set of 1000 randomly picked images from Places-205 dataset.
The optimization procedure followed is the same as in the
case of the previous two task.
Table 7 and 8 show the performance of MonodepthResnet50, and Monodepth-VGG respectively under the
presence of the various UAPs crafted by the proposed
method. As can be observed from the tables, the crafted
UAPs have a strong impact on the performance of the
networks. For both the variants of monodepth, UAPs crafted

12

Fig. 9. Sample original and adversarial image pairs from KITTI dataset generated for Monodepth-VGG. First row shows clean and perturbed
images with various priors. Second row shows the corresponding predicted depth maps.
TABLE 7
Performance of the crafted perturbations for Monodepth-Resnet50 using various metrics for evaluating depth estimation on the eigen test-split of
KITTI datset. Results are presented for various scenarios such as clean data (Normal), perturbations learned with no prior, various priors and the
train set mean. Note that the best fooling results for each scenario are shown in bold. The evaluation of trainset mean performance has been taken
from [45]. Note that the first four metrics are error based (higher means better fooling, indicated as ↑) and the later three are precision based
(lower is better fooling, indicated as ↓).
Perturbation
Normal
Baseline
No prior
Range prior
Data prior
Train set mean

Abs Rel (↑)

Sq Rel (↑)

RMSE (↑)

RMSE log (↑)

δ < 1.25 (↓)

δ < 1.252 (↓)

δ < 1.253 (↓)

0.133
0.1339
0.201
0.319
0.380
0.361

1.148
1.1591
1.810
3.292
10.278
4.826

5.549
5.576
6.603
9.064
10.976
8.102

0.230
0.231
0.352
0.640
0.402
0.377

0.829
0.827
0.688
0.460
0.708
0.638

0.935
0.934
0.840
0.611
0.842
0.804

0.970
0.969
0.908
0.717
0.900
0.894

TABLE 8
Performance of the crafted perturbations for Monodepth-VGG using various metrics for evaluating depth estimation on the eigen test-split of KITTI
datset. Results are presented for various scenarios such as clean data (Normal), perturbations learned with no prior, various priors and the train
set mean. Note that the best fooling results for each scenario are shown in bold. The evaluation of trainset mean performance has been taken
from [45]. Note that the first four metrics are error based (higher means better fooling, indicated as ↑) and the later three are precision based
(lower is better fooling, indicated as ↓).
Perturbation
Normal
Baseline
No prior
Range prior
Data prior
Train set mean

Abs Rel (↑)

Sq Rel (↑)

RMSE (↑)

RMSE log (↑)

δ < 1.25 (↓)

δ < 1.252 (↓)

δ < 1.253 (↓)

0.148
0.149
0.192
0.212
0.355
0.361

1.344
1.353
1.802
2.073
9.612
4.826

5.927
5.949
6.626
6.994
10.592
8.102

0.247
0.248
0.325
0.364
0.390
0.377

0.803
0.800
0.704
0.658
0.714
0.638

0.922
0.920
0.861
0.825
0.850
0.804

0.964
0.963
0.929
0.906
0.908
0.894

with range prior, bring down the accuracy with the threshold of 1.25 units (δ < 1.25) by 25.7% on an average.
With data priors, the crafted UAPs are able to increase the
Sq Rel (an error metric) to almost 10 times the original
performance. Under the impact of the crafted UAPs, the
network’s performance drops below that of the baseline,
which uses the train set mean as the prediction for all image
pixels.
Table 9 shows the Generalized Fooling Rates with respect
to the accuracy with the threshold of 1.25 units (δ < 1.25).
A surprising observation from the table is that the performance of the range prior perturbations on both the networks
surpasses that of the data prior perturbations. This might
lead us to understand that the range prior perturbations
are stronger. However the Generalized Fooling Rate (GFR)
measures the change in a network’s outcome, with respect to
a metric. As observed from Table 7 and 8, the data prior perturbations have a much harsher effect on the error metrics
like Root Mean Square Error (RMSE), whereas range prior
perturbations have a harsher effect on precision metrics like
δ < 1.25. Hence, ‘which perturbation is better?’, is rather
a metric dependant question and conclusions based on a

TABLE 9
Generalized Fooling rate with respect to δ < 1.25 metric for the task of
depth estimation

Model
Monodepth-VGG
Monodepth-Resnet50

Baseline

No
data

Range
prior

Data
prior

0.4%
2%

15.3%
21.3%

22.7%
47.6%

21.3%
24.3%

single metric would only partially reflect the truth.
Figures 9 and 11 show some qualitative examples, showing the perturbed input and the change in output. Figure 9
shows the input-output pair for Monodepth-VGG, where
the input is perturbed by the various kinds of UAPs crafted.
Figure 11 compares the input-output pair for MonodepthVGG and Monodepth-Resnet50 using the range-prior UAP.
As show by figures 9 and 11, for both versions of Monodepth, quasi-imperceptible change in the input can cause a
large change in the output.

13

No data Prior

Range Prior

Data Prior

Fig. 10. Universal adversarial perturbations for depth estimation task crafted using our proposed algorithm.Top row shows the perturbations learned
for Monodepth-Resnet50 and the bottom row shows that for Monodepth-VGG. Perturbations were crafted with ξ = 10 for no prior,range prior, and
all data prior scenarios. Corresponding scenario is mentioned below each column. Images are best viewed in color.

Fig. 11. Depth predictions of both models over a sample perturbed image. Perturbations were crafted with ξ = 10 using the range prior.The top row
corresponds to Monodepth-Resnet50 and the bottom row corresponds to Monodepth-VGG. The first column shows the perturbed input image,
the second shows the depth output of clean sample image, and the third shows the depth output of the perturbed sample image. Images are best
viewed in color.

the learned representations at each layer?” That is, Are the
features learned by the CNNs robust to small changes in the
input ? or Can a small change in the input layer leads to a
huge change in its output ? As mentioned earlier ‘fooling’
refers to instability of the CNN in terms of its output,
independent of the task at hand.

Fig. 12. Percentage relative change in the extracted representations
caused by our crafted perturbations at multiple layers of VGG-16. Note
that as we do deeper into the net, the amount of perturbation increases
drastically which results in eventual misclassification of the input.

5

D ISCUSSION

As demonstrated by our experimentation in section 4, it
is evident that the proposed algorithm is able to craft
perturbations which achieve ‘fooling’ for a variety of computer vision tasks. An important question to ask about the
Convolutional neural Networks (CNNs), is “How stable are

Earlier works (e.g., [4], [5], [8]) have proven that the
CNNs can be fooled by small but learned noise. Most of
these works (e.g., [5]) rely on data to find these directions
in which the input can be slightly perturbed in order to
fool the CNN. These approaches try to find via simple
methods such as gradient ascent, the directions in which
the network’s confidence falls. Recent works by MoosaviDezfooli et al. [7], [8] find the nearest classification boundary
for a given sample and perturb it by moving across the
boundary thereby achieve the fooling. They also establish
that it is possible to learn a single perturbation which can
work for most of the input images. Thus existing works
(independent of the underlying task) exploit data samples,
model’s confidence and the classification boundaries in the
input space in order to craft the adversarial perturbations,
either image specific or agnostic.
Unlike the existing works, we depict this as a stability
issue with the learned representations by CNNs. We attempt
to learn the optimal perturbation in the input space that
can cause maximal change in the output of the network.

14

TABLE 10
Relative Shift in Classification Layer’s input and the fooling rate, for
VGG-16 in the task of classification. The relative shift has been
evaluation on 1000 random images from ILSVRC validation set, while
the fooling rate is evaluated on the entire ILSVRC validation set.

Perturbation

Rel. Shift in input to
f c8 (classification) layer

Fooling rate

Baseline
No Prior
Range Prior
All data Prior

0.0006
0.867
1.142
3.169

8.62
45.47
63.08
77.77

We achieve this via learning perturbations that can result
in maximal change in the activations at all the intermediate
layers of the architecture. As an example, we consider VGG16 CNN trained for object recognition to illustrate the working of our objective. Figure 12 shows the percentage relative
kl (x+δ)−li (x)k2 ×100
change in the feature activations ( i
) at
kli (x)k2
various layers in the architecture. Also, the relative change
is shown for variants of our method that utilize the prior
information about the training data distribution. Note that
the percentage relative change in the feature activations
due to the addition of the learned perturbation increases
monotonically as we go deeper in the network. Because
of this accumulated perturbation in the projection of the
input, our learned adversaries are able to fool the CNNs
independent of the task at hand. This phenomenon explains
the fooling achieved by our objective. Also, Figure 12 shows
that with the utilization of the data priors, the relative
perturbation further increases. Hence, our objective achieves
better fooling when the prior information is provided during the learning. Note that, for comparison, the baseline
perturbation of random noise with equal max-norm as the
learned perturbations is provided. The perturbation caused
by the random noise is almost negligible at all the layers
of CNN, which explains its robustness to random noise
attacks.
Finally, we relate the relative change caused by our
perturbations at the input to classification layer (f c8 or softmax) to the fooling rates achieved for various perturbations.
Table 10 shows the relative shift in the feature activations
i (x)k2
) that are input to the classification layer and
( kli (x+δ)−l
kli (x)k2
the corresponding fooling rates for various perturbations.
Note that they are highly correlated, which explains why
and how the proposed objective can fool the CNNs trained
across multiple vision tasks.

crafted perturbations can pose substantial threat to the deep
learned systems in terms of black-box attacking.
Further, we show that our objective can exploit minimal
priors about the target data distribution to craft stronger
perturbations. For example, providing simple information
such as the mean and dynamic range of the images to
the proposed objective would craft significantly stronger
perturbations. Though the proposed objective is data-free
in nature, it can craft stronger perturbations when data is
utilized.
More importantly, we introduced the idea of generalizable objectives to craft image-agnostic perturbations. It is
already established that the representations learned by deep
models are susceptible. On top of it, the existence of generic
objectives to fool “any” learning based vision model independent of the underlying task can pose critical concerns
about the model deployment. Therefore, it is an important
research direction to be focused by the community in order
to build reliable machine learning based systems.

R EFERENCES
[1]

[2]

[3]

[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]

6

C ONCLUSION

In this paper, we proposed a novel data-free objective to
craft image-agnostic (universal) adversarial perturbations
(UAP). More importantly, we show that the proposed objective is generalizable not only across multiple CNN architectures but also across diverse computer vision tasks.
We demonstrated that our seemingly simple objective of
injecting maximal “adversarial” energy into the learned
representations (subject to the imperceptibility constraint)
is effective to fool both the classification and regression
models. Significant transfer performances achieved by our

[12]
[13]

[14]
[15]
[16]

B. Biggio, G. Fumera, and F. Roli, “Pattern recognition systems
under attack: Design issues and research challenges,” International
Journal of Pattern Recognition and Artificial Intelligence, vol. 28,
no. 07, 2014.
B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov,
G. Giacinto, and F. Roli, “Evasion attacks against machine learning
at test time,” in Joint European Conference on Machine Learning and
Knowledge Discovery in Databases, 2013, pp. 387–402.
L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein, and J. D.
Tygar, “Adversarial machine learning,” in Proceedings of the 4th
ACM Workshop on Security and Artificial Intelligence, ser. AISec ’11,
2011.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,”
arXiv preprint arXiv:1312.6199, 2013.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and
harnessing adversarial examples,” arXiv preprint arXiv:1412.6572,
2014.
A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial machine
learning at scale,” arXiv preprint arXiv:1611.01236, 2016.
S. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: A
simple and accurate method to fool deep neural networks,” in
IEEE Computer Vision and Pattern Recognition (CVPR), 2016.
S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal adversarial perturbations,” in IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2017.
K. R. Mopuri, U. Garg, and R. V. Babu, “Fast feature fool: A data
independent approach to universal adversarial perturbations,” in
Proceedings of the British Machine Vision Conference (BMVC), 2017.
C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, “Adversarial examples for semantic segmentation and object detection,”
arXiv preprint arXiv:1703.08603, 2017.
J. H. Metzen, M. C. Kumar, T. Brox, and V. Fischer, “Universal
adversarial perturbations against semantic image segmentation,”
in International Conference on Computer Vision (ICCV), 2017.
Y. Bengio, “Learning deep architectures for AI,” Found. Trends
Mach. Learn., vol. 2, no. 1, Jan. 2009.
N. Papernot, P. D. McDaniel, I. J. Goodfellow, S. Jha, Z. B.
Celik, and A. Swami, “Practical black-box attacks against deep
learning systems using adversarial examples,” arXiv preprint
arXiv:1602.02697, 2016.
A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples
in the physical world,” arXiv preprint arXiv:1607.02533, 2016.
C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. L. Yuille, “Adversarial examples for semantic segmentation and object detection,”
arXiv preprint arXiv:1703.08603, 2017.
A. Fawzi, O. Fawzi, and P. Frossard, “Analysis of classifiers’ robustness to adversarial perturbations,” arXiv preprint
arXiv:1502.02590, 2015.

15

Fig. 13. Sample failure case for the object recognition using VGG-16 model. Top row shows multiple clean images from ILSVRC validation set.
Bottom row shows the adversarial images generated by adding the perturbation crafted utilizing the no data prior. Note that for all the shown images
the perturbation fails to change the predicted label.

Clean

No data

Range prior

Data with less BG

All data

Fig. 14. Sample failure case for the semantic segmentation using the FCN-8s-VGG model. Top row shows the clean and corresponding perturbed
images for no prior to various priors. Bottom row shows the predicted segmentation maps. Note that the people segments are undisturbed by the
addition of various perturbations.

Clean

No data

Range prior

Data prior

Fig. 15. Sample failure case for the depth estimation using Monodepth-VGG model. Note that, top row shows clean image and the corresponding
perturbed images with no data case and various prior cases. Bottom row shows the corresponding depth predictions.

[17] A. Fawzi, S. Moosavi-Dezfooli, and P. Frossard, “Robustness of
classifiers: from adversarial to random noise,” in Advances in
Neural Information Processing Systems (NIPS), 2016.
[18] A. Nguyen, J. Yosinski, and J. Clune, “Deep neural networks
are easily fooled: High confidence predictions for unrecognizable
images,” in Computer Vision and Pattern Recognition (CVPR), 2015.
[19] A. Rozsa, E. M. Rudd, and T. E. Boult, “Adversarial diversity and
hard positive generation,” in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (CVPR) Workshops, 2016,
pp. 25–32.
[20] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, “Adversarial examples for semantic segmentation and object detection,”

in International Conference on Computer Vision (ICCV), 2017.
[21] V. Behzadan and A. Munir, “Vulnerability of deep reinforcement learning to policy induction attacks,” arXiv preprint
arXiv:1701:04143, 2017.
[22] O. Bastani, Y. Ioannou, L. Lampropoulos, D. Vytiniotis, A. Nori,
and A. Criminisi, “Measuring neural net robustness with constraints,” in Advances in Neural Information Processing Systems
(NIPS), 2016.
[23] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok, “Synthesizing robust adversarial examples,” arXiv preprint arXiv:1707.07397, 2017.
[24] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with

16

convolutions,” arXiv preprint arXiv:1409.4842, 2014.
[25] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” arXiv preprint arXiv:1512.03385, 2015.
[26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,
Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and
L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,”
International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp.
211–252, 2015.
[27] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and
A. Zisserman, “The PASCAL Visual Object Classes Challenge 2012
(VOC2012) Results.”
[28] M. Everingham, L. Van-Gool, C. K. I. Williams, J. Winn, and
A. Zisseman, “The PASCAL Visual Object Classes Challenge 2007
(VOC2011) Results.”
[29] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous
driving? the kitti vision benchmark suite,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
[30] B. Zhou, A. Khosla, À. Lapedriza, A. Torralba, and A. Oliva,
“Places: An image database for deep scene understanding,” arXiv
preprint arXiv:1610.02055, 2016.
[31] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint: arXiv:1412.6980, 2014.
[32] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick,
S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture
for fast feature embedding,” arXiv preprint arXiv:1408.5093, 2014.
[33] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, “Return
of the devil in the details: Delving deep into convolutional nets,”
in Proceedings of the British Machine Vision Conference (BMVC), 2014.
[34] K. Simonyan and A. Zisserman, “Very deep convolutional
networks for large-scale image recognition,” arXiv preprint
arXiv:abs/1409.1556, 2014.
[35] E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE transactions on pattern
analysis and machine intelligence (PAMI), vol. 39, no. 4, 2017.
[36] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L.
Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” in
International Conference on Learning Representations (ICLR), 2015.
[37] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L.
Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” arXiv
preprint arXiv:1606.00915, 2016.
[38] B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, and J. Malik,
“Semantic contours from inverse detectors,” in IEEE International
Conference on Computer Vision (ICCV), 2011.
[39] C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised
monocular depth estimation with left-right consistency,” in CVPR,
2017.
[40] D. B. Sam, S. Surya, and R. V. Babu, “Switching convolutional
neural network for crowd counting,” in 2017 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2017.
[41] A. Kendall and R. Cipolla, “Geometric loss functions for camera
pose regression with deep learning,” in 2017 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), July 2017, pp.
6555–6564.
[42] S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser,
“Semantic scene completion from a single depth image,” in 2017
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
July 2017, pp. 190–198.
[43] D. Xu, E. Ricci, W. Ouyang, X. Wang, and N. Sebe, “Multi-scale
continuous crfs as sequential deep networks for monocular depth
estimation,” in 2017 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), July 2017, pp. 161–169.
[44] D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from
a single image using a multi-scale deep network,” in Advances in
Neural Information Processing Systems (NIPS), 2014, pp. 2366–2374.
[45] C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised
monocular depth estimation with left-right consistency,” in IEEE
conference on Computer Vision and Pattern Recognition (CVPR), 2017.

