Published as a conference paper at ICLR 2018

arXiv:1705.07904v3 [cs.LG] 22 Feb 2018

S EMANTICALLY D ECOMPOSING THE L ATENT S PACES
OF G ENERATIVE A DVERSARIAL N ETWORKS
Chris Donahue
Department of Music
University of California, San Diego
cdonahue@ucsd.edu

Zachary C. Lipton
Carnegie Mellon University
Amazon AI
zlipton@cmu.edu

Akshay Balsubramani
Department of Genetics
Stanford University
abalsubr@stanford.edu

Julian McAuley
Department of Computer Science
University of California, San Diego
jmcauley@eng.ucsd.edu

A BSTRACT
We propose a new algorithm for training generative adversarial networks that jointly
learns latent codes for both identities (e.g. individual humans) and observations
(e.g. specific photographs). By fixing the identity portion of the latent codes, we
can generate diverse images of the same subject, and by fixing the observation
portion, we can traverse the manifold of subjects while maintaining contingent
aspects such as lighting and pose. Our algorithm features a pairwise training
scheme in which each sample from the generator consists of two images with a
common identity code. Corresponding samples from the real dataset consist of two
distinct photographs of the same subject. In order to fool the discriminator, the
generator must produce pairs that are photorealistic, distinct, and appear to depict
the same individual. We augment both the DCGAN and BEGAN approaches with
Siamese discriminators to facilitate pairwise training. Experiments with human
judges and an off-the-shelf face verification system demonstrate our algorithm’s
ability to generate convincing, identity-matched photographs.

1

I NTRODUCTION

In many domains, a suitable generative process might consist of several stages. To generate a
photograph of a product, we might wish to first sample from the space of products, and then from
the space of photographs of that product. Given such disentangled representations in a multistage
generative process, an online retailer might diversify its catalog, depicting products in a wider variety
of settings. A retailer could also flip the process, imagining new products in a fixed setting. Datasets
for such domains often contain many labeled identities with fewer observations of each (e.g. a
collection of face portraits with thousands of people and ten photos of each). While we may know the
identity of the subject in each photograph, we may not know the contingent aspects of the observation
(such as lighting, pose and background). This kind of data is ubiquitous; given a set of commonalities,
we might want to incorporate this structure into our latent representations.
Generative adversarial networks (GANs) learn mappings from latent codes z in some low-dimensional
space Z to points in the space of natural data X (Goodfellow et al., 2014). They achieve this
power through an adversarial training scheme pitting a generative model G : Z 7→ X against a
discriminative model D : X 7→ [0, 1] in a minimax game. While GANs are popular, owing to their
ability to generate high-fidelity images, they do not, in their original form, explicitly disentangle the
latent factors according to known commonalities.
In this paper, we propose Semantically Decomposed GANs (SD-GANs), which encourage a specified portion of the latent space to correspond to a known source of variation.1,2 The technique
1
2

Web demo: https://chrisdonahue.github.io/sdgan
Source code: https://github.com/chrisdonahue/sdgan

1

Published as a conference paper at ICLR 2018

Figure 1: Generated samples from SD-BEGAN. Each of the four rows has the same identity code zI
and each of the fourteen columns has the same observation code zO .
decomposes the latent code Z into one portion ZI corresponding to identity, and the remaining
portion ZO corresponding to the other contingent aspects of observations. SD-GANs learn through a
pairwise training scheme in which each sample from the real dataset consists of two distinct images
with a common identity. Each sample from the generator consists of a pair of images with common
zI ∈ ZI but differing zO ∈ ZO . In order to fool the discriminator, the generator must not only
produce diverse and photorealistic images, but also images that depict the same identity when zI is
fixed. For SD-GANs, we modify the discriminator so that it can determine whether a pair of samples
constitutes a match.
As a case study, we experiment with a dataset of face photographs, demonstrating that SD-GANs
can generate contrasting images of the same subject (Figure 1; interactive web demo in footnote
on previous page). The generator learns that certain properties are free to vary across observations
but not identity. For example, SD-GANs learn that pose, facial expression, hirsuteness, grayscale
vs. color, and lighting can all vary across different photographs of the same individual. On the
other hand, the aspects that are more salient for facial verification remain consistent as we vary the
observation code zO . We also train SD-GANs on a dataset of product images, containing multiple
photographs of each product from various perspectives (Figure 4).
We demonstrate that SD-GANs trained on faces generate stylistically-contrasting, identity-matched
image pairs that human annotators and a state-of-the-art face verification algorithm recognize as
depicting the same subject. On measures of identity coherence and image diversity, SD-GANs
perform comparably to a recent conditional GAN method (Odena et al., 2017); SD-GANs can also
imagine new identities, while conditional GANs are limited to generating existing identities from the
training data.

2

S EMANTICALLY D ECOMPOSED G ENERATIVE A DVERSARIAL N ETWORKS

Before introducing our algorithm, we briefly review the prerequisite concepts.
2.1

GAN PRELIMINARIES

GANs leverage the discriminative power of neural networks to learn generative models. The generative model G ingests latent codes z, sampled from some known prior PZ , and produces G(z), a
sample of an implicit distribution PG . The learning process consists of a minimax game between G,
parameterized by θG , and a discriminative model D, parameterized by θD . In the original formulation,
the discriminative model tries to maximize log likelihood, yielding
min max V (G, D) = Ex∼PR [log D(x)] + Ez∼PZ [log(1 − D(G(z)))].
G

D

(1)

Training proceeds as follows: For k iterations, sample one minibatch from the real distribution PR
and one from the distribution of generated images PG , updating discriminator weights θD to increase
V (G, D) by stochastic gradient ascent. Then sample a minibatch from PZ , updating θG to decrease
V (G, D) by stochastic gradient descent.
2

Published as a conference paper at ICLR 2018

Algorithm 1 Semantically Decomposed GAN Training
1: for n in 1:NumberOfIterations do
2:
for m in 1:MinibatchSize do
3:
Sample one identity vector zI ∼ Uniform([−1, 1]dI ).
4:
Sample two observation vectors z1O , z2O ∼ Uniform([−1, 1]dO ).
5:
z1 ← [zI ; z1O ], z2 ← [zI ; z2O ].
6:
Generate pair of images G(z1 ), G(z2 ), adding them to the minibatch with label 0 (fake).
7:
8:
9:
10:
11:
12:
13:

for m in 1:MinibatchSize do
Sample one identity i ∈ I uniformly at random from the real data set.
Sample two images of i without replacement x1 , x2 ∼ PR (x|I = i).
Add the pair to the minibatch, assigning label 1 (real).
Update discriminator weights by θD ← θD + ∇θD V (G, D) using its stochastic gradient.
Sample another minibatch of identity-matched latent vectors z1 , z2 .
Update generator weights by stochastic gradient descent θG ← θG − ∇θG V (G, D).

Zhao et al. (2017b) propose energy-based GANs (EBGANs), in which the discriminator can be
viewed as an energy function. Specifically, they devise a discriminator consisting of an autoencoder:
D(x) = Dd (De (x)). In the minimax game, the discriminator’s weights are updated to minimize
the reconstruction error L(x) = ||x − D(x)|| for real data, while maximizing the error L(G(z))
for the generator. More recently, Berthelot et al. (2017) extend this work, introducing Boundary
Equilibrium GANs (BEGANs), which optimize the Wasserstein distance (reminiscent of Wasserstein
GANs (Arjovsky et al., 2017)) between autoencoder loss distributions, yielding the formulation:
VBEGAN (G, D) = L(x) − L(G(z)).

(2)

Additionally, they introduce a method for stabilizing training. Positing that training becomes unstable
when the discriminator cannot distinguish between real and generated images, they introduce a new
hyperparameter γ, updating the value function on each iteration to maintain a desired ratio between
the two reconstruction errors: E[L(G(z))] = γ E[L(x)]. The BEGAN model produces what appear
to us, subjectively, to be the sharpest images of faces yet generated by a GAN. In this work, we adapt
both the DCGAN (Radford et al., 2016) and BEGAN algorithms to the SD-GAN training scheme.
2.2

SD-GAN FORMULATION

Consider the data’s identity as a random variable I in a discrete index set I. We seek to learn a latent
representation that conveniently decomposes the variation in the real data into two parts: 1) due to I,
and 2) due to the other factors of variation in the data, packaged as a random variable O. Ideally, the
decomposition of the variation in the data into I and O should correspond exactly to a decomposition
of the latent space Z = ZI × ZO . This would permit convenient interpolation and other operations
on the inferred subspaces ZI and ZO .
A conventional GAN samples I, O from their joint distribution. Such a GAN’s generative model
samples directly from an unstructured prior over the latent space. It does not disentangle the variation
in O and I, for instance by modeling conditional distributions PG (O | I = i), but only models their
average with respect to the prior on I.
Our SD-GAN method learns such a latent space decomposition, partitioning the coordinates of Z
into two parts representing the subspaces, so that any z ∈ Z can be written as the concatenation
[zI ; zO ] of its identity representation zI ∈ RdI = ZI and its contingent aspect representation
zO ∈ RdO = ZO . SD-GANs achieve this through a pairwise training scheme in which each sample
from the real data consists of x1 , x2 ∼ PR (x | I = i), a pair of images with a common identity i ∈ I.
Each sample from the generator consists of G(z1 ), G(z2 ) ∼ PG (z | ZI = zI ), a pair of images
generated from a common identity vector zI ∈ ZI but i.i.d. observation vectors z1O , z2O ∈ ZO . We
assign identity-matched pairs from PR the label 1 and zI -matched pairs from PG the label 0. The
discriminator can thus learn to reject pairs for either of two primary reasons: 1) not photorealistic or
2) not plausibly depicting the same subject. See Algorithm 1 for SD-GAN training pseudocode.
3

Published as a conference paper at ICLR 2018

(a) DCGAN

(b) SD-DCGAN

(c) BEGAN

(d) SD-BEGAN

Figure 2: SD-GAN architectures and vanilla counterparts. Our SD-GAN models incorporate a decomposed latent space and Siamese discriminators. Dashed lines indicate shared weights. Discriminators
also observe real samples in addition to those from the generator (not pictured for simplicity).
2.3

SD-GAN DISCRIMINATOR ARCHITECTURE

With SD-GANs, there is no need to alter the architecture of the generator. However, the discriminator
must now act upon two images, producing a single output. Moreover, the effects of the two input
images x1 , x2 on the output score are not independent. Two images might be otherwise photorealistic
but deserve rejection because they clearly depict different identities. To this end, we devise two
novel discriminator architectures to adapt DCGAN and BEGAN respectively. In both cases, we first
separately encode each image using the same convolutional neural network De (Figure 2). We choose
this Siamese setup (Bromley, 1994; Chopra et al., 2005) as our problem is symmetrical in the images,
and thus it’s sensible to share weights between the encoders.
To adapt DCGAN, we stack the feature maps De (x1 ) and De (x2 ) along the channel axis, applying
one additional strided convolution. This allows the network to further aggregate information from the
two images before flattening and fully connecting to a sigmoid output. For BEGAN, because the
discriminator is an autoencoder, our architecture is more complicated. After encoding each image,
we concatenate the representations [De (x1 ); De (x2 )] ∈ R2(dI +dO ) and apply one fully connected
bottleneck layer R2(dI +dO ) ⇒ RdI +2dO with linear activation. In alignment with BEGAN, the
SD-BEGAN bottleneck has the same dimensionality as the tuple of latent codes (zI , z1O , z2O ) that
generated the pair of images. Following the bottleneck, we apply a second FC layer RdI +2dO ⇒
R2(dI +dO ) , taking the first dI + dO components of its output to be the input to the first decoder and
the second dI + dO components to be the input to the second decoder. The shared intermediate layer
gives SD-BEGAN a mechanism to push apart matched and unmatched pairs. We specify our exact
architectures in full detail in Appendix E.

3

E XPERIMENTS

We experimentally validate SD-GANs using two datasets: 1) the MS-Celeb-1M dataset of celebrity
face images (Guo et al., 2016) and 2) a dataset of shoe images collected from Amazon (McAuley
et al., 2015). Both datasets contain a large number of identities (people and shoes, respectively) with
multiple observations of each. The “in-the-wild” nature of the celebrity face images offers a richer
test bed for our method as both identities and contingent factors are significant sources of variation.
In contrast, Amazon’s shoe images tend to vary only with camera perspective for a given product,
making this data useful for sanity-checking our approach.
Faces From the aligned face images in the MS-Celeb-1M dataset, we select 12,500 celebrities at
random and 8 associated images of each, resizing them to 64x64 pixels. We split the celebrities into
subsets of 10,000 (training), 1,250 (validation) and 1,250 (test). The dataset has a small number
of duplicate images and some label noise (images matched to the wrong celebrity). We detect and
4

Published as a conference paper at ICLR 2018

Figure 3:
Generated samples
SD-DCGAN model trained on faces.

from

Figure 4:
Generated samples from
SD-DCGAN model trained on shoes.

remove duplicates by hashing the images, but we do not rid the data of label noise. We scale the pixel
values to [−1, 1], performing no additional preprocessing or data augmentation.
Shoes Synthesizing novel product images is another promising domain for our method. In our
shoes dataset, product photographs are captured against white backgrounds and primarily differ in
orientation and distance. Accordingly, we expect that SD-GAN training will allocate the observation
latent space to capture these aspects. We choose to study shoes as a prototypical example of a
category of product images. The Amazon dataset contains around 3,000 unique products with the
category “Shoe” and multiple product images. We use the same 80%, 10%, 10% split and again hash
the images to ensure that the splits are disjoint. There are 6.2 photos of each product on average.
3.1

T RAINING DETAILS

We train SD-DCGANs on both of our datasets for 500,000 iterations using batches of 16 identitymatched pairs. To optimize SD-DCGAN, we use the Adam optimizer (Kingma & Ba, 2015) with
hyperparameters α = 2e−4, β1 = 0.5, β2 = 0.999 as recommended by Radford et al. (2016). We
also consider a non-Siamese discriminator that simply stacks the channels of the pair of real or fake
images before encoding (SD-DCGAN-SC).
As in (Radford et al., 2016), we sample latent vectors z ∼ Uniform([−1, 1]100 ). For SD-GANs, we
partition the latent codes according to zI ∈ RdI , zO ∈ R100−dI using values of dI = [25, 50, 75].
Our algorithm can be trivially applied with k-wise training (vs. pairwise). To explore the effects of
using k > 2, we also experiment with an SD-DCGAN where we sample k = 4 instances each from
PG (z | ZI = zI ) for some zI ∈ ZI and from PR (x | I = i) for some i ∈ I. For all experiments,
unless otherwise stated, we use dI = 50 and k = 2.
We also train an SD-BEGAN on both of our datasets. The increased complexity of the SD-BEGAN
model significantly increases training time, limiting our ability to perform more-exhaustive hyperparameter validation (as we do for SD-DCGAN). We use the Adam optimizer with the default
hyperparameters from (Kingma & Ba, 2015) for our SD-BEGAN experiments. While results from
our SD-DCGAN k = 4 model are compelling, an experiment with a k = 4 variant of SD-BEGAN
resulted in early mode collapse (Appendix F); hence, we excluded SD-BEGAN k = 4 from our
evaluation.
We also compare to a DCGAN architecture trained using the auxiliary classifier GAN (AC-GAN)
method (Odena et al., 2017). AC-GAN differs from SD-GAN in two key ways: 1) random identity
codes zI are replaced by a one-hot embedding over all the identities in the training set (matrix of size
10000x50); 2) the AC-GAN method encourages that generated photos depict the proper identity by
tasking its discriminator with predicting the identity of the generated or real image. Unlike SD-GANs,
the AC-DCGAN model cannot imagine new identities; when generating from AC-DCGAN (for our
quantitative comparisons to SD-GANs), we must sample a random identity from those existing in the
training data.
5

Published as a conference paper at ICLR 2018

Table 1: Evaluation of 10k pairs from MS-Celeb-1M (real data) and generative models; half have
matched identities, half do not. The identity verification metrics demonstrate that FaceNet (FN) and
human annotators on Mechanical Turk (MT) verify generated data similarly to real data. The sample
diversity metrics ensure that generated samples are statistically distinct in pixel space. Data generated
by our best model (SD-BEGAN) performs comparably to real data. * 1k pairs, † 200 pairs.
Identity Verification
Dataset

3.2

Mem

Sample Diversity

Judge

AUC

Acc.

FAR

ID-Div

All-Div

MS-Celeb-1M
AC-DCGAN
SD-DCGAN
SD-DCGAN-SC
SD-DCGAN k=4
SD-DCGAN dI =25
SD-DCGAN dI =75
SD-BEGAN

131 MB
57 MB
47 MB
75 MB
57 MB
57 MB
68 MB

FN
FN
FN
FN
FN
FN
FN
FN

.913
.927
.823
.831
.852
.835
.816
.928

.867
.851
.749
.757
.776
.764
.743
.857

.045
.083
.201
.180
.227
.222
.268
.110

.621
.497
.521
.560
.523
.526
.517
.588

.699
.666
.609
.637
.614
.615
.601
.673

†MS-Celeb-1M
*MS-Celeb-1M
*AC-DCGAN
*SD-DCGAN k=4
*SD-BEGAN

131 MB
75 MB
68 MB

Us
MT
MT
MT
MT

-

.850
.759
.765
.688
.723

.110
.035
.090
.147
.096

.621
.621
.497
.523
.588

.699
.699
.666
.614
.673

E VALUATION

The evaluation of generative models is a fraught topic. Quantitative measures of sample quality can
be poorly correlated with each other (Theis et al., 2016). Accordingly, we design an evaluation to
match conceivable uses of our algorithm. Because we hope to produce diverse samples that humans
deem to depict the same person, we evaluate the identity coherence of SD-GANs and baselines using
both a pretrained face verification model and crowd-sourced human judgments obtained through
Amazon’s Mechanical Turk platform.
3.2.1

Q UANTITATIVE

Recent advancements in face verification using deep convolutional neural networks (Schroff et al.,
2015; Parkhi et al., 2015; Wen et al., 2016) have yielded accuracy rivaling humans. For our evaluation,
we procure FaceNet, a publicly-available face verifier based on the Inception-ResNet architecture
(Szegedy et al., 2017). The FaceNet model was pretrained on the CASIA-WebFace dataset (Yi et al.,
2014) and achieves 98.6% accuracy on the LFW benchmark (Huang et al., 2012).3
FaceNet ingests normalized, 160x160 color images and produces an embedding f (x) ∈ R128 .
The training objective for FaceNet is to learn embeddings that minimize the L2 distance between
matched pairs of faces and maximize the distance for mismatched pairs. Accordingly, the embedding
space yields a function for measuring the similarity between two faces x1 and x2 : D(x1 , x2 ) =
||f (x1 ) − f (x2 )||22 . Given two images, x1 and x2 , we label them as a match if D(x1 , x2 ) ≤ τv
where τv is the accuracy-maximizing threshold on a class-balanced set of pairs from MS-Celeb-1M
validation data. We use the same threshold for evaluating both real and synthetic data with FaceNet.
We compare the performance of FaceNet on pairs of images from the MS-Celeb-1M test set against
generated samples from our trained SD-GAN models and AC-DCGAN baseline. To match FaceNet’s
training data, we preprocess all images by resizing from 64x64 to 160x160, normalizing each image
individually. We prepare 10,000 pairs from MS-Celeb-1M, half identity-matched and half unmatched.
From each generative model, we generate 5,000 pairs each with z1I = z2I and 5,000 pairs with
z1I 6= z2I . For each sample, we draw observation vectors zO randomly.
We also want to ensure that identity-matched images produced by the generative models are diverse.
To this end, we propose an intra-identity sample diversity (ID-Div) metric. The multi-scale structural
similarity (MS-SSIM) (Wang et al., 2004) metric reports the similarity of two images on a scale
from 0 (no resemblance) to 1 (identical images). We report 1 minus the mean MS-SSIM for all pairs
3

“20170214-092102” pretrained model from https://github.com/davidsandberg/facenet

6

Published as a conference paper at ICLR 2018

of identity-matched images as ID-Div. To measure the overall sample diversity (All-Div), we also
compute 1 minus the mean similarity of 10k pairs with random identities.
In Table 1, we report the area under the receiver operating characteristic curve (AUC), accuracy, and
false accept rate (FAR) of FaceNet (at threshold τv ) on the real and generated data. We also report our
proposed diversity statistics. FaceNet verifies pairs from the real data with 87% accuracy compared
to 86% on pairs from our SD-BEGAN model. Though this is comparable to the accuracy achieved
on pairs from the AC-DCGAN baseline, our model produces samples that are more diverse in pixel
space (as measured by ID-Div and All-Div). FaceNet has a higher but comparable FAR for pairs
from SD-GANs than those from AC-DCGAN; this indicates that SD-GANs may produce images that
are less semantically diverse on average than AC-DCGAN.
We also report the combined memory footprint of G and D for all methods in Table 1. For conditional
GAN approaches, the number of parameters grows linearly with the number of identities in the
training data. Especially in the case of the AC-GAN, where the discriminator computes a softmax
over all identities, linear scaling may be prohibitive. While our 10k-identity subset of MS-Celeb-1M
requires a 131MB AC-DCGAN model, an AC-DCGAN for all 1M identities would be over 8GB,
with more than 97% of the parameters devoted to the weights in the discriminator’s softmax layer. In
contrast, the complexity of SD-GAN is constant in the number of identities.
3.2.2

Q UALITATIVE

In addition to validating that identity-matched SD-GAN samples are verified by FaceNet, we also
demonstrate that humans are similarly convinced through experiments using Mechanical Turk. For
these experiments, we use balanced subsets of 1,000 pairs from MS-Celeb-1M and the most promising
generative methods from our FaceNet evaluation. We ask human annotators to determine if each
pair depicts the “same person” or “different people”. Annotators are presented with batches of ten
pairs at a time. Each pair is presented to three distinct annotators and predictions are determined by
majority vote. Additionally, to provide a benchmark for assessing the quality of the Mechanical Turk
ensembles, we (the authors) manually judged 200 pairs from MS-Celeb-1M. Results are in Table 1.
For all datasets, human annotators on Mechanical Turk answered “same person” less frequently than
FaceNet when the latter uses the accuracy-maximizing threshold τv . Even on real data, balanced
so that 50% of pairs are identity-matched, annotators report “same person” only 28% of the time
(compared to the 41% of FaceNet). While annotators achieve higher accuracy on pairs from ACDCGAN than pairs from SD-BEGAN, they also answer “same person” 16% more often for ACDCGAN pairs than real data. In contrast, annotators answer “same person” at the same rate for
SD-BEGAN pairs as real data. This may be attributable to the lower sample diversity produced by
AC-DCGAN. Samples from SD-DCGAN and SD-BEGAN are shown in Figures 3 and 1 respectively.

4

R ELATED WORK

Style transfer and novel view synthesis are active research areas. Early attempts to disentangle style
and content manifolds used factored tensor representations (Tenenbaum & Freeman, 1997; Vasilescu
& Terzopoulos, 2002; Elgammal & Lee, 2004; Tang et al., 2013), applying their results to face image
synthesis. More recent work focuses on learning hierarchical feature representations using deep
convolutional neural networks to separate identity and pose manifolds for faces (Zhu et al., 2013;
Reed et al., 2014; Zhu et al., 2014; Yang et al., 2015; Kulkarni et al., 2015; Oord et al., 2016; Yan
et al., 2016) and products (Dosovitskiy et al., 2015). Gatys et al. (2016) use features of a convolutional
network, pretrained for image recognition, as a means for discovering content and style vectors.
Since their introduction (Goodfellow et al., 2014), GANs have been used to generate increasingly highquality images (Radford et al., 2016; Zhao et al., 2017b; Berthelot et al., 2017). Conditional GANs
(cGANs), introduced by Mirza & Osindero (2014), extend GANs to generate class-conditional data.
Odena et al. (2017) propose auxiliary classifier GANs, combining cGANs with a semi-supervised
discriminator (Springenberg, 2015). Recently, cGANs have been used to ingest text (Reed et al.,
2016) and full-resolution images (Isola et al., 2017; Liu et al., 2017; Zhu et al., 2017) as conditioning
information, addressing a variety of image-to-image translation and style transfer tasks. Chen et al.
(2016) devise an information-theoretic extension to GANs in which they maximize the mutual
information between a subset of latent variables and the generated data. Their unsupervised method
7

Published as a conference paper at ICLR 2018

Figure 5: Linear interpolation of zI (identity) and zO (observation) for three pairs using SD-BEGAN
generator. In each matrix, rows share zI and columns share zO .
appears to disentangle some intuitive factors of variation, but these factors may not correspond to
those explicitly disentangled by SD-GANs.
Several related papers use GANs for novel view synthesis of faces. Tran et al. (2017); Huang et al.
(2017); Yin et al. (2017a;b); Zhao et al. (2017a) all address synthesis of different body/facial poses
conditioned on an input image (representing identity) and a fixed number of pose labels. Antipov
et al. (2017) propose conditional GANs for synthesizing artificially-aged faces conditioned on both
a face image and an age vector. These approaches all require explicit conditioning on the relevant
factor (such as rotation, lighting and age) in addition to an identity image. In contrast, SD-GANs can
model these contingent factors implicitly (without supervision).
Mathieu et al. (2016) combine GANs with a traditional reconstruction loss to disentangle identity.
While their approach trains with an encoder-decoder generator, they enforce a variational bound
on the encoder embedding, enabling them to sample from the decoder without an input image.
Experiments with their method only address small (28x28) grayscale face images, and their training
procedure is complex to reproduce. In contrast, our work offers a simpler approach and can synthesize
higher-resolution, color photographs.
One might think of our work as offering the generative view of the Siamese networks often favored
for learning similarity metrics (Bromley, 1994; Chopra et al., 2005). Such approaches are used for
discriminative tasks like face or signature verification that share the many classes with few examples
structure that we study here. In our work, we adopt a Siamese architecture in order to enable the
discriminator to differentiate between matched and unmatched pairs. Recent work by Liu & Tuzel
(2016) propose a GAN architecture with weight sharing across multiple generators and discriminators,
but with a different problem formulation and objective from ours.

5

D ISCUSSION

Our evaluation demonstrates that SD-GANs can disentangle those factors of variation corresponding
to identity from the rest. Moreover, with SD-GANs we can sample never-before-seen identities, a
benefit not shared by conditional GANs. In Figure 3, we demonstrate that by varying the observation
vector zO , SD-GANs can change the color of clothing, add or remove sunnies, or change facial
pose. They can also perturb the lighting, color saturation, and contrast of an image, all while keeping
the apparent identity fixed. We note, subjectively, that samples from SD-DCGAN tend to appear
less photorealistic than those from SD-BEGAN. Given a generator trained with SD-GAN, we can
independently interpolate along the identity and observation manifolds (Figure 5).
On the shoe dataset, we find that the SD-DCGAN model produces convincing results. As desired, manipulating zI while keeping zO fixed yields distinct shoes in consistent poses (Figure 4). The identity
code zI appears to capture the broad categories of shoes (sneakers, flip-flops, boots, etc.). Surprisingly,
neither original BEGAN nor SD-BEGAN can produce diverse shoe images (Appendix G).
In this paper, we presented SD-GANs, a new algorithm capable of disentangling factors of variation
according to known commonalities. We see several promising directions for future work. One logical
extension is to disentangle latent factors corresponding to more than one known commonality. We
also plan to apply our approach in other domains such as identity-conditioned speech synthesis.
8

Published as a conference paper at ICLR 2018

ACKNOWLEDGEMENTS
The authors would like to thank Anima Anandkumar, John Berkowitz and Miller Puckette for their
helpful feedback on this work. This work used the Extreme Science and Engineering Discovery
Environment (XSEDE), which is supported by National Science Foundation grant number ACI1053575 (Towns et al., 2014). GPUs used in this research were donated by the NVIDIA Corporation.

R EFERENCES
Grigory Antipov, Moez Baccouche, and Jean-Luc Dugelay. Face aging with conditional generative
adversarial networks. In ICIP, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein GAN. In ICML, 2017.
David Berthelot, Tom Schumm, and Luke Metz. BEGAN: Boundary equilibrium generative adversarial networks. arXiv:1703.10717, 2017.
Jane Bromley. Signature verification using a "Siamese" time delay neural network. In NIPS, 1994.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. InfoGAN:
Interpretable representation learning by information maximizing generative adversarial nets. In
NIPS, 2016.
Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with
application to face verification. In CVPR, 2005.
Alexey Dosovitskiy, Jost Tobias Springenberg, and Thomas Brox. Learning to generate chairs with
convolutional neural networks. In CVPR, 2015.
Ahmed Elgammal and Chan-Su Lee. Separating style and content on a nonlinear manifold. In CVPR,
2004.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional
neural networks. In CVPR, 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair,
Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In NIPS, 2014.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. MS-Celeb-1M: A dataset
and benchmark for large scale face recognition. In ECCV, 2016.
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Neural networks for machine learning lecture
6a: Overview of mini–batch gradient descent.
Gary B. Huang, Marwan Mattar, Honglak Lee, and Erik Learned-Miller. Learning to align from
scratch. In NIPS, 2012.
Rui Huang, Shu Zhang, Tianyu Li, and Ran He. Beyond face rotation: Global and local perception
gan for photorealistic and identity preserving frontal view synthesis. In ICCV, 2017.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with
conditional adversarial networks. In CVPR, 2017.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Tejas D Kulkarni, William F Whitney, Pushmeet Kohli, and Josh Tenenbaum. Deep convolutional
inverse graphics network. In NIPS, 2015.
Zachary C Lipton and Subarna Tripathi. Precise recovery of latent vectors from generative adversarial
networks. ICLR Workshop Track, 2017.
Ming-Yu Liu and Oncel Tuzel. Coupled generative adversarial networks. In NIPS, 2016.
Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In
ICML, 2017.
9

Published as a conference paper at ICLR 2018

Michael F Mathieu, Junbo Jake Zhao, Junbo Zhao, Aditya Ramesh, Pablo Sprechmann, and Yann
LeCun. Disentangling factors of variation in deep representation using adversarial training. In
NIPS, 2016.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based
recommendations on styles and substitutes. In SIGIR, 2015.
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv:1411.1784, 2014.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary
classifier GANs. In ICML, 2017.
Aaron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Koray
Kavukcuoglu. Conditional image generation with pixelcnn decoders. In NIPS, 2016.
Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. In BMVC, 2015.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context
encoders: Feature learning by inpainting. In CVPR, 2016.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep
convolutional generative adversarial networks. In ICLR, 2016.
Scott Reed, Kihyuk Sohn, Yuting Zhang, and Honglak Lee. Learning to disentangle factors of
variation with manifold interaction. In ICML, 2014.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee.
Generative adversarial text to image synthesis. In ICML, 2016.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A unified embedding for face
recognition and clustering. In CVPR, 2015.
Jost Tobias Springenberg. Unsupervised and semi-supervised learning with categorical generative
adversarial networks. In ICLR, 2015.
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. Inception-v4, Inception-ResNet
and the impact of residual connections on learning. In AAAI, 2017.
Yichuan Tang, Ruslan Salakhutdinov, and Geoffrey Hinton. Tensor analyzers. In ICML, 2013.
Joshua B Tenenbaum and William T Freeman. Separating style and content. In NIPS, 1997.
Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative
models. In ICLR, 2016.
John Towns, Timothy Cockerill, Maytal Dahan, Ian Foster, Kelly Gaither, Andrew Grimshaw, Victor
Hazlewood, Scott Lathrop, Dave Lifka, Gregory D Peterson, et al. XSEDE: Accelerating scientific
discovery. Computing in Science & Engineering, 2014.
Luan Tran, Xi Yin, and Xiaoming Liu. Disentangled representation learning GAN for pose-invariant
face recognition. In CVPR, 2017.
M Vasilescu and Demetri Terzopoulos. Multilinear analysis of image ensembles: Tensorfaces. In
ECCV, 2002.
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality
assessment. In Asilomar Conference on Signals, Systems and Computers, 2004.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach
for deep face recognition. In ECCV, 2016.
Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. Attribute2Image: Conditional image
generation from visual attributes. In ECCV, 2016.
Jimei Yang, Scott E Reed, Ming-Hsuan Yang, and Honglak Lee. Weakly-supervised disentangling
with recurrent transformations for 3D view synthesis. In NIPS, 2015.
10

Published as a conference paper at ICLR 2018

Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. Learning face representation from scratch.
arXiv:1411.7923, 2014.
Weidong Yin, Yanwei Fu, Leonid Sigal, and Xiangyang Xue. Semi-latent GAN: Learning to generate
and modify facial images from attributes. arXiv:1704.02166, 2017a.
Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. Towards large-pose
face frontalization in the wild. In ICCV, 2017b.
Bo Zhao, Xiao Wu, Zhi-Qi Cheng, Hao Liu, and Jiashi Feng. Multi-view image generation from a
single-view. arXiv:1704.04886, 2017a.
Junbo Zhao, Michael Mathieu, and Yann LeCun. Energy-based generative adversarial network. In
ICLR, 2017b.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation
using cycle-consistent adversarial networks. In ICCV, 2017.
Zhenyao Zhu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning identity-preserving face
space. In ICCV, 2013.
Zhenyao Zhu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Multi-view perceptron: a deep model
for learning face identity and view representations. In NIPS, 2014.

11

Published as a conference paper at ICLR 2018

A

E STIMATING LATENT CODES

Figure 6: Linear interpolation of both identity (vertical) and observation (horizontal) on latent codes
recovered for unseen images. All rows have the same identity vector (zI ) and all columns have the
same observation vector (zO ).
We estimate latent vectors for unseen images and demonstrate that the disentangled representations
of SD-GANs can be used to depict the estimated identity with different contingent factors. In order to
find a latent vector ẑ such that G(ẑ) (pretrained G) is similar to an unseen image x, we can minimize
the distance between x and G(ẑ): minẑ ||G(ẑ) − x||22 (Lipton & Tripathi, 2017).
In Figure 6, we depict estimation and linear interpolation across both subspaces for two pairs of
images using SD-BEGAN. We also display the corresponding source images being estimated. For
both pairs, ẑI (identity) is consistent in each row and ẑO (observation) is consistent in each column.

B

PAIRWISE DISCRIMINATION OF EMBEDDINGS AND ENCODINGS

In Section 3.1, we describe an AC-GAN (Odena et al., 2017) baseline which uses an embedding
matrix over real identities as latent identity codes (G : i, zO 7→ x̂). In place of random identity
vectors, we tried combining this identity representation with pairwise discrimination (in the style of
SD-GAN). In this experiment, the discriminator receives either either two real images with the same
identity (x1i , x2i ), or a real image with label i and synthetic image with label i (x1i , G(i, zO )). All
other hyperparameters are the same as in our SD-DCGAN experiment (Section 3.1). We show results
in Figure 7.

Figure 7: Generator with a one-hot identity embedding trained against a pairwise discriminator. Each
row shares an identity vector and each column shares an observation vector. Random sample of 4
real images of the corresponding identity on the right.
In Appendix C, we detail a modification of the DR-GAN (Tran et al., 2017) method which uses an
encoding network Ge to transform images to identity representations (Gd : Ge (x), zO 7→ x̂). We
also tried combining this encoder-decoder approach with pairwise discrimination. The discriminator
12

Published as a conference paper at ICLR 2018

receives either two real images with the same identity (x1i , x2i ), or (x1i , Gd (Ge (x1i ), zO ). We show
results in Figure 8.

Figure 8: Generator with an encoder-decoder architecture trained against a pairwise discriminator.
Each row shares an identity vector and each column shares an observation vector. Input image on the
right.
While these experiments are exploratory and not part of our principle investigation, we find the results
to be qualitatively promising. We are not the first to propose pairwise discrimination with pairs of
(real, real) or (real, fake) images in GANs (Pathak et al., 2016; Isola et al., 2017).

C

E XPLORATORY EXPERIMENT WITH DR-GAN S

Tran et al. (2017) propose Disentangled Representation learning-GAN (DR-GAN), an approach
to face frontalization with similar setup to our SD-GAN algorithm. The (single-image) DR-GAN
generator G (composition of Ge and Gd ) accepts an input image x, a pose code c, and a noise vector z.
The DR-GAN discriminator receives either x or x̂ = Gd (Ge (x), c, z). In the style of (Springenberg,
2015), the discriminator is tasked with determining not only if the image is real or fake, but also
classifying the pose c, suggesting a disentangled representation to the generator. Through their
experiments, they demonstrate that DR-GAN can explicitly disentangle pose and illumination (c)
from the rest of the latent space (Ge (x); z).

Figure 9: Generated samples from cGAN trained only to disentangle identity. Each row shares an
identity vector and each column shares an observation vector; input image on the right.
In addition to our AC-DCGAN baseline (Odena et al., 2017), we tried modifying DR-GAN to
only disentangle identity (rather than both identity and pose in the original paper). We used the
DCGAN (Radford et al., 2016) discriminator architecture (Table 4) as Ge , linearly projecting the
final convolutional layer to Ge (x) ∈ R50 (in alignment with our SD-GAN experiments). We altered
the discriminator to predict the identity of x or x̂, rather than pose information (which is unknown
in our experimental setup). With these modifications, Ge (x) is analogous to zI in the SD-GAN
generator, and z is analogous to zO . Furthermore, this setup is identical to the AC-DCGAN baseline
13

Published as a conference paper at ICLR 2018

except that the embedding matrix is replaced by an encoding network Ge . Unfortunately, we found
that the generator quickly learned to produce a single output image x̂ for each input x regardless
of observation code z (Figure 9). Accordingly, we excluded this experiment from our evaluation
(Section 3.2).

D

I MAGINING IDENTITIES WITH AC-GAN

Figure 10: AC-DCGAN generation with random identity vectors that sum to one. Each row shares an
identity vector and each column shares an observation vector.

Figure 11: AC-DCGAN generation with one-hot identity vectors. Each row shares an identity vector
and each column shares an observation vector.
As stated in Section 3.1, AC-GANs Odena et al. (2017) provide no obvious way to imagine new identities. For our evaluation (Section 3.2), the AC-GAN generator receives identity input zI ∈ [0, 1]10000 :
a one-hot over all identities. One possible approach to imagining new identities would be to query a
P10000
trained AC-GAN generator with a random vector zI such that i=1 zI [i] = 1. We found that
this strategy produced little identity variety (Figure 10) compared to the normal one-hot strategy
(Figure 11) and excluded it from our evaluation.

E

A RCHITECTURE D ESCRIPTIONS

We list here the full architectural details for our SD-DCGAN and SD-BEGAN models. In these
descriptions, k is the number of images that the generator produces and discriminator observes per
identity (usually 2 for pairwise training), and dI is the number of dimensions in the latent space ZI
(identity). In our experiments, dimensionality of ZO is always 100 − dI . As a concrete example, the
bottleneck layer of the SD-BEGAN discriminator autoencoder (“fc2” in Table 6) with k = 2, dI = 50
has output dimensionality 150.
We emphasize that generators are parameterized by k in the tables only for clarity and symmetry with
the discriminators. Implementations need not modify the generator; instead, k can be collapsed into
the batch size.
For the stacked-channels versions of these discriminators, we simply change the number of input
image channels from 3 to 3k and set k = 1 wherever k appears in the table.
14

Published as a conference paper at ICLR 2018

Table 2: Input abstraction for both SD-DCGAN and SD-BEGAN generators during training (where
zO is always different for every pair or set of k)
Operation
[zi ; zo ]
dup zi
concat

Input Shape
[(dI ,);(k,100-dI )]
[(dI ,);(k,100-dI )]
[(k,dI );(k,100-dI )]

Kernel Size

Output Shape
[(dI ,);(k,100-dI )]
[(k,dI );(k,100-dI )]
(k,100)

Table 3: SD-DCGAN generator architecture
Operation
z
fc1
reshape
bnorm
relu
upconv1
bnorm
relu
upconv2
bnorm
relu
upconv3
bnorm
relu
upconv4
tanh

Input Shape
(k,100)
(k,8192)
(k,8192)
(k,4,4,512)
(k,4,4,512)
(k,4,4,512)
(k,8,8,256)
(k,8,8,256)
(k,8,8,256)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,32,32,64)
(k,32,32,64)
(k,32,32,64)
(k,64,64,3)

Kernel Size
(100,8192)

(5,5,512,256)
(5,5,256,128)
(5,5,128,64)
(5,5,64,3)

Output Shape
(k,100)
(k,8192)
(k,4,4,512)
(k,4,4,512)
(k,4,4,512)
(k,8,8,256)
(k,8,8,256)
(k,8,8,256)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,32,32,64)
(k,32,32,64)
(k,32,32,64)
(k,64,64,3)
(k,64,64,3)

Table 4: SD-DCGAN discriminator architecture
Operation
x or G(z)
downconv1
lrelu(a=0.2)
downconv2
bnorm
lrelu(a=0.2)
downconv3
bnorm
lrelu(a=0.2)
downconv4
stackchannels
downconv5
flatten
fc1
sigmoid

Input Shape
(k,64,64,3)
(k,64,64,3)
(k,32,32,64)
(k,32,32,64)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,8,8,256)
(k,8,8,256)
(k,8,8,256)
(k,4,4,512)
(4,4,512k)
(2,2,512)
(2048,)
(1,)

Kernel Size
(5,5,3,64)
(5,5,64,128)
(5,5,128,256)
(5,5,256,512)
(3,3,512k,512)
(2048,1)

15

Output Shape
(k,64,64,3)
(k,32,32,64)
(k,32,32,64)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,8,8,256)
(k,8,8,256)
(k,8,8,256)
(k,4,4,512)
(4,4,512k)
(2,2,512)
(2048,)
(1,)
(1,)

Published as a conference paper at ICLR 2018

Table 5: SD-BEGAN generator architecture
Operation
z
fc1
reshape
conv2d
elu
conv2d
elu
upsample2
conv2d
elu
conv2d
elu
upsample2
conv2d
elu
conv2d
elu
upsample2
conv2d
elu
conv2d
elu
conv2d

Input Shape
(k,100)
(k,100,)
(k,100,8192)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)

Kernel Size
(100,8192)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,3)

Output Shape
(k,100)
(k,100,8192)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,8,8,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,16,16,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,32,32,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,3)

Table 6: SD-BEGAN discriminator autoencoder architecture. The decoder portion is equivalent to,
but does not share weights with, the SD-BEGAN generator architecture (Table 5).
Operation
x or G(z)
conv2d
elu
conv2d
elu
conv2d
elu
downconv2d
elu
conv2d
elu
conv2d
elu
downconv2d
elu
conv2d
elu
conv2d
elu
downconv2d
elu
conv2d
elu
conv2d
elu
flatten
fc1
concat
fc2
fc3
split
G (Table 5)

Input Shape
(k,64,64,3)
(k,64,64,3)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,32768)
(k,100)
(100k,)
(dI +(100-dI )k,)
(100k,)
(k,100)

Kernel Size
(3,3,3,128)
(3,3,128,128)
(3,3,128,128)
(3,3,128,256)
(3,3,256,256)
(3,3,256,256)
(3,3,256,384)
(3,3,384,384)
(3,3,384,384)
(3,3,384,512)
(3,3,512,512)
(3,3,512,512)
(32768,100)
(100k,dI +(100-dI )k,)
(dI +(100-dI )k,100k,)

16

Output Shape
(k,64,64,3)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,64,64,128)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,32,32,256)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,16,16,384)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,8,8,512)
(k,32768)
(k,100)
(100k,)
(dI +(100-dI )k,)
(100k,)
(k,100)
(k,64,64,3)

Published as a conference paper at ICLR 2018

F

FACE S AMPLES

We present samples from each model reported in Table 1 for qualitative comparison. In each matrix,
zI is the same across all images in a row and zO is the same across all images in a column. We draw
identity and observation vectors randomly for these samples.

Figure 12: Generated samples from AC-DCGAN (four sample of real photos of ID on right)

Figure 13: Generated samples from SD-DCGAN

Figure 14: Generated samples from SD-DCGAN with stacked-channel discriminator
17

Published as a conference paper at ICLR 2018

Figure 15: Generated samples from SD-DCGAN with k = 4

Figure 16: Generated samples from SD-DCGAN with dI = 25

Figure 17: Generated samples from SD-DCGAN with dI = 75

Figure 18: Generated samples from SD-DCGAN trained with the Wasserstein GAN loss (Arjovsky
et al., 2017). This model was optimized using RMS-prop (Hinton et al.) with α = 5e−5. In our
evaluation (Section 3.2), FaceNet had an AUC of .770 and an accuracy of 68.5% (at τv ) on data
generated by this model. We excluded it from Table 1 for brevity.
18

Published as a conference paper at ICLR 2018

Figure 19: Generated samples from SD-BEGAN

Figure 20: Generated samples from SD-BEGAN with k = 4, demonstrating mode collapse

G

S HOE S AMPLES

We present samples from an SD-DCGAN and SD-BEGAN trained on our shoes dataset.

Figure 21: Generated samples from SD-DCGAN

Figure 22: Generated samples from SD-BEGAN

19

