arXiv:1712.09216v1 [] 26 Dec 2017

Large-Scale 3D Scene Classification With Multi-View Volumetric
CNN
Dror Aiger
Google Inc.

Brett Allen
Google Inc.

Aleksey Golovinskiy
Google Inc.

aigerd@google.com

brettallen@google.com

agolovin@google.com

December 27, 2017

Abstract

riety of commercial products (an overview of stereo reconstruction methods can be found in [13]). Approaches
targeting high quality typically use “dense stereo” reconstruction. These usually define a “photoconsistency” volume as a scalar function on 3D space whose value increases with the similarity of projected images. A high
photoconsistency value at a 3D position suggests the existence of a surface. We propose using the projected images that serve as input to such photoconsistency volumes
to classify a 3D volume. We further develop a step that
merges the classifications of these volumes into a consistent 2D result for the case of aerial imagery.
The following properties are desirable in a practical,
large-scale classification system:

We introduce a method to classify imagery using a convolutional neural network (CNN) on multi-view image projections. The power of our method comes from using projections of multiple images at multiple depth planes near
the reconstructed surface. This enables classification of
categories whose salient aspect is appearance change under different viewpoints, such as water, trees, and other
materials with complex reflection/light response properties. Our method does not require boundary labelling
in images and works on pixel-level classification with
a small (few pixels) context, which simplifies the creation of a training set. We demonstrate this application
on large-scale aerial imagery collections, and extend the
per-pixel classification to robustly create a consistent 2D
classification which can be used to fill the gaps in nonreconstructible water regions. We also apply our method
to classify tree regions. In both cases, the training data can
quickly be generated using a small number of manuallycreated polygons on a map. We show that even with a
very simple and standard network our CNN outperforms
the state-of-the-art image classification, the Inception-V3
model retrained from a large collection of aerial images.

1

• Scale. It must generalize well, to avoid significantly
increasing the size of training set with increased use.
• Ease of curating / creating training sets. We seek to
avoid needing a huge, manually-labelled set of examples as is typical for CNN-based labeling / segmentation methods.
• Robustness to reconstruction / photoconsistency
computation. The system must continue to work as
the underlying stereo pipeline changes.
• Very high accuracy. High numerical classification
rates may be misleading, since even a small error rate
can lead to a preponderance of visible artifacts.

Introduction

Methods to reconstruct 3D geometry from images have
become increasingly effective at creating photorealistic
3D scenes, both in the research community and in a va-

While our approach is general to any volume and set
of posed images, we consider it in the context of (a)
1

(c)

(d)

Figure 1: Aliasing in stereo reconstruction causes water
regions to be very noisy (a). We apply our water classification technique to each source image’s depth map and its
projections, and aggregate the result into a 2D mask (b).
Using this mask, we can replace the water with a smooth
surface (c) and texture this mesh (d).

(a)

1 - photoconsistency

(b)

1 - photoconsistency

(a)

The problem offers many challenges, which we examine through the lens of the above case studies. One approach might be to explicitly classify source images. Visually, water is deceptively difficult to classify (see Figure 4 for a few examples of water vs non-water patches),
so a classifier operating explicitly on source images will
have a hard time. Our method overcomes this problem
by learning from the interactions between projected image features rather than just the images. This is similar to
how photoconsistency volumes are created, but we avoid
having to design the similarity measure and choose parameters such as window sizes.

altitude

altitude

(b)

(c)

Figure 2: (a) In wavy water, we can discover image patches with high correlation through pure chance.
For example, observe the plane-sweep photoconsistency
graphs for the land and water depth slices marked with red
crosses are shown in (b) and (c) respectively. The characteristics of these graphs are not sufficient to distinguish
water from land.

depthmap creation, in which a source pixel corresponds to
an optic ray through the volume, and (b) aerial imagery,
where many applications desire as a final result a 2D classified raster. We examine the challenges of such a system
and how our approach addresses them in the context of
two specific applications.
First, we target the classification of water. While dense
stereo methods can create high-resolution geometry under appropriate circumstances, it is difficult to judge when
they fail, and failure leads to egregious errors (see Figure 1(a)). In the case of aerial imagery, the most problematic artifacts in practice come from attempting to reconstruct water, which is tricky for stereo systems because
it is often moving and has specular highlights that can
create deceptive photoconsistency maxima. Our method
can identify and fill such errors, leading to results in Figure 1(c).
Second, we apply our system to classify trees. Trees
are an example of a ubiquitous object useful to classify
in a number of applications, ranging from visualization to
modeling to GIS.

An alternative approach to the classification problem
would be to create a set of heuristics that operate on the
photoconsistency volume. Even ignoring the inherent labor required to make separate heuristics for each classification problem, this is a challenging task for any given
particular problem. For example, Figure 2 shows graphs
of a photoconsistency for a land and a water area, where
it happens that the curves are quite similar.
Our method overcomes these challenges. Our contributions include:
• A CNN-based method to classify the optic rays
through a photoconsistency volume. This results in
a classified 3D point cloud. Pixel-level correlations
2

among views help classify materials with complex
reflection functions, such as water and trees. Using
context-based semantic segmentation would require
large, fully-labelled image portions in our training
set. In contrast, our system is only ”weakly” supervised in the sense that we do not mark boundaries
between classes in images, needing only sparse pixel
labels for its samples.

selected regions around the globe, where each region contains just one class.
Semantic segmentation is commonly handled using
a state-of-the-art convolutional neural network (CNN)
framework [7],[10]. Several papers investigated weaklysupervised learning due to the time consuming requirements (per-pixel labelling) in semantic segmentation
[19],[9],[3]. Semantic segmentation can be achieved by
classifying image patches around each input pixel. Convolutional networks are at the core of most state-of-theart computer vision solutions for many tasks. Deep convolutional networks are mainstream, yielding very good
performance in various benchmarks used at large scale
(e.g., [14][16][1][5]). Recently, the Inception-v3 architecture [13] has been shown to achieve very good performance on the ILSVRC 2012 classification challenge with
substantial gains over the state of the art. All of these
works address image segmentation where the input is just
one image. Semantic segmentation needs context, so large
parts of images with dense labels are required for training data. Producing such a huge, fully-labelled training
set for world-wide classification, with a variety of challenging regions and contexts, is very difficult. Volumetric
and multi view CNN classification was previously used
in [12] but the motivation there is object categorization
from geometry. We focus on pixel-level classification. As
far as we know this is the first attempt to classify image
pixels by considering the properties captured by the correspondences among several images. Moreover, the classification in this context leads to a 3D point cloud classification. Using pixel-level classification with tiny context
(small patches) makes the labelling much easier as we can
sample sparse pixels classified as land or water.
In our application we exploit the fact that several images see the same structure and we use plane sweep multiview stereo and its multi-view structure. We use projections to all available images (or a subset) and let our CNN
to learn their mutual correlation. Using the additional information we are able to use a quite standard CNN very
much resembling the LeNet CNN [6] with weak supervision to obtain the world-wide accurate semantic segmentation that is essential for our 3D reconstruction. We are
not aware of any prior work that does this but we do our
best to compare our work against the state of the art in
image-based classification.
A related direction has been investigated by [4], with

• An extension of the above to use results from multiple images to create a consistent 2D mask, aggregating probabilities from all available classified 3D
points.
• For the water classification application: a method to
fill the gaps in reconstructed regions that were classified as water including robust estimation of the water
level from the boundaries and a filling algorithm.
While our system is general, the case study in this paper focuses on water and trees, leaving to future work the
exploration of other applications of the multiview pixellevel correlation property. Our main claim is that for
many classes of objects, classification based solely on
individual images is challenging, especially when practical considerations limit the size of training sets in size
and character (we do not assume densely-labelled training sets). We overcome this challenge by using information from multiview correlations, which encode how appearance changes with viewpoints. We also note that for
world-wide applications, there are always regions where
context cannot help because a pixel does not have any
context very far away (e.g. large bodies of water). This
means that we must be related on pixel level classification.
Previous work Our problem is essentially semantic segmentation where we restrict ourselves to using
”weakly” supervised learning in the sense that we do not
mark boundaries between classes in images. This restriction comes from the desire to perform multi-class segmentation over the entire world; we want to avoid the complex
labelling / curation process needed for a training set that
accounts for the variations present in the input data. Even
for the specific case of water classification, this variation
is large and difficult to capture with a small training set.
So we prefer to train our algorithm on just a few manually3

the goal of performing joint reconstruction and segmentation. Compared to our approach, their work uses trainingderived classification priors: (a) appearance likelihoods
derived from individual source images, and (b) geometric
priors derived from a 3D dataset. Our volume-based approach expands on (a): we find that classification results
are vastly improved by a system that explicitly considers relationships between multiple image projections. We
hope to avoid the chicken-and-egg problem of obtaining a
3D training set as required for their system in (b).

2

General stereo background and alternative methods can
be found in [13]. Within their taxonomy, our algorithm
can be summarized as:
• Scene representation: we use a depth map to represent a scene
• Photo-consistency measure: we use a measure in
scene space
• Visibility model: outlier-based. We dont explicitly
model occlusions, and instead assume that lowest
k photoconsistency scores might be from occluded
views

The raw volumetric classification

2.1

• Shape prior: In absence of information, we favor
horizontal planes (although this can be modified to
an extent)

Plane sweep stereo

• Reconstruction algorithm: use graph cuts [20] to
compute optimal depth maps, and merge them in a
post-process
• Initialization requirement: we assume that we can
initialize a voxel grid that is guaranteed to contain
the 3D surface

2.2
Figure 3: Plane Sweep Stereo

Difficulty of image classification for water

To motivate our approach, in Figure 4 we show 14 image patches; 7 are water and 7 are land. It is clear that
distinguishing based on these patches alone is quite hard.
Our 3D classification makes use of multi-view image projections and analyzes not only source image patches, but
the interactions between projections of multiple images.
This gives our method more power than image classification alone.

Our algorithm takes its structure from the 3D reconstruction technique referred to as plane sweep stereo.
There are two (or more) slightly offset views imaging a
scene (Figure 3). Under favorable conditions, a point on
the 3D surface, when projected to the images, should have
the same value. The voxel containing this surface point is
said to have high photoconsistency. To help disambiguate
among many plausible high-photoconsistency surfaces, a
smoothness prior is added. The rough form of the algorithm is then:

2.3

• Form a voxel grid containing the surface

Multi-view volumetric data representation

Our classification is done in 3D. Using the plane sweep
algorithm we obtain for each view (per each input image)
a surface which is the computed depthmap corresponding to this view. Our goal is to classify each pixel in this
depthmap (a point in 3-space). In a subsequent section,
we will use all depthmaps together to compute a robust

• Project each voxel to source image and compute a
photoconsistency score
• Solve for a depthmap in the voxel grid that has high
photoconsistency and high smoothness
4

ery 3D grid cell. For time performance and efficiency our
plane-sweep algorithm works in a coarse-to-fine manner,
so we actually compute the projections only for points in
the sub-volume around each depthmap pixel. For faster
runtime, we use grayscale intensities rather than full RGB
color.
Creating the sub-volumes for classification Let D be
a depthmap corresponding to one of the views. D is emFigure 4: It is hard to distinguish water and land just from bedded in 3 space so we can take the set of projected colimage patches. Can we tell which is water, which is land ors from V for every point around any depthmap pixel in
(top is land, bottom is water)?
D. For classification and training we extract a sub-volume
around every pixel in D containing all grid volume cells
from V around the surface point. Having k images per
2D mask. To take into account information from all views volume grid cell, we arrange our data in a 3D array with
and to account for spatial arrangement in the 3D space one color per entry, containing the k samples in the z axis
we represent the data as a sub-volume centred at each 3D of the volume. As commonly done for images, we norlocation on the computed depthmap. This enables us to malize the colors in the sub-volume to compensate for
adapt state-of-the-art CNN-based classifiers designed for lighting variations regularly observed among all input imimage classification, extending them to 3D convolution in ages.
a straightforward way.

V

p

S

D

Figure 6: Data representation. 8 slices in the input vector
Figure 5: A sub-volume extracted around a point p in 3- for water (bottom) and land (top).
space. V is the multi-view volume with projected colors
per voxel. D is a depthmap computed by the plane sweep
In our work we use 8 image projections per grid cell
and S is the created sub-volume.
and the sub-volume size we extract around any cell has
size 2 × 13 × 13. This means that the dimensionality of
the data is 2 × 8 × 13 × 13 = 2704. As we show in
Extracting a set of colors for every cell in a volume grid our results, the overall numerical classification over the
We create a 3-dimensional grid volume V similar to the evaluation data as well as the output 2D masks are very
one used by the plane sweep. Figure 5 shows the arrange- good.
ment around a point p in the volume V . For each cell p in
this volume, we use the projection matrices corresponding 2.4 The network
to all input images seeing this region and we store a set of
colors (intensity) from each projected pixels. Our volume The input to our network is structured as an 13 × 13 × 16
therefore contains k colors (one for each image) for ev- voxel volume while the feature representation is a 512
5

dimensional vector. Following several similar ConvNet
architecture for classifying 2D images and local image
patches [6],[21], the network consists of two convolutional layers with ReLU non-linearity and pooling layers.
The volume size is small so we cannot use many pooling
layers. Our convolution uses 3D kernels. The detailed
kernel sizes and number of filters are shown in Figure 7.
We then have 4 fully connected layers with dropout that
prevents the network from over-fitting [8]. Our loss is the
sparse softmax cross entropy between logits and labels
and we apply exponential decay to the learning rate. For
optimization we use the momentum algorithm [11]. The
implementation of the CNN is TensorFlow [2].

gons that completely contain a target class. Each such
polygon labels all the depthmaps that are located inside it,
avoiding the need to perform manual segmentation. One
of our pleasant surprises was that including regions containing class boundaries (which would require extensive
manual work) is not required to achieve good results.

0.94 0.06

dropout

softmax

fc,2

fc,32

dropout

fc,128

dropout

fc,512

relu

3x3x32 conv,64

relu

/2 pool x-y

3x3x8 conv,32

11x11x8 data

Figure 8: Example of training data sampled from San
Francisco area
We train the system using the described network, randomly dividing the training set to 9/10 for training and
1/10 for validation (unseen).

Figure 7: The network

2.5

3

Training data and numerical results

Generating the 2D mask

The volumetric classification labels each 3D depthmap
point with a probability corresponding to each class. One
option to use this data is to create a fully-classified 3D
surface. A simpler option and the one we take here (we
left the 3D classification for future work) is to create a
2D mask (which can be considered as the projection of
our points on earth) containing the labels of the class with
the largest probability. In the example of water, this yields
a boolean mask telling us for each 2D point whether it is
“water” or “land.” To do this we have to consider the
projection of the points along with their class confidence.
We use a method to combine individual probabilities derived directly from Bayes rule [15]. For ease of notation,
we consider two classes below in the context of water vs
land classification, but this generalizes easily into multiple classes.
The formula is valid only if the probabilities are independent, which is not the case here here, but it is a useful
idealization since the statistical correlations between individual probabilities is not known. A similar approach was

The training data is considered to be only ”weakly” supervised in the sense that we don’t mark boundaries in
images. We have a full labelling of the training data but
the labelling is sparse (per sampled pixel) and the fraction
of the training data is very small relative to the size of the
data we process (the entire world) and the variations in the
input. Our training data consists of small regions defined
by 100 polygons of area approximately 1 km2 each, balanced between the classes. The number of regions around
the globe we take is fairly small, 30 different regions overall. The generalization of our classifier must be powerful
so these regions would be sufficient to classify all unseen
regions. We show in our results that we obtain very good
classification in unseen regions with large variations in the
classes. An example of some polygons in San Francisco
is shown in Figure 8. We sample uniformly the data in
the polygons which makes our entire data set consisting
of approximately 1M vectors for each class.
Note that while the training data is fully labelled, we’re
able to create the training set in practice by drawing poly6

taken in filtering spam messages [18]:
p=

close to zero because most of the water sources we consider are indeed horizontal (e.g., seas, oceans). Figure 9
shows plane fitting applied to different nearby shorelines,
each is a separate connected component that has different
water level.

p1 p2 ...pn
p1 p2 ...pn + (1 − p1 )(1 − p2 )...(1 − pn )

Where p is the probability that the projected point is
“water” and pi are all individual probabilities of being
“water” from all depthmaps. This enables us to create a
discrete 2D mask (a boolean image) containing the label
derived by thresholding the projected confidence at 0.5.
To do this, we discretize the 3-space, insert all points to
their corresponding 3D cells and then consider a vertical column in this volume as the projection to earth. We
then accumulate all probabilities in the columns cells using the formula above. The 2D boolean mask is then used
to apply further processing steps. For example, for water
classification we have to determine the level of the water
according to the (3D) boundaries of the mask and then
fill an artificial surface representing the water to allow a
full 3D representation of the entire scene containing water and land.

4

Figure 9: Water level estimation using plane fitting on two
nearby different connected components.
The final water levels are set by smoothly filling in from
the shoreline using Laplace’s equation. Note that while
water is generally flat, in some cases such as rivers it may
change altitude over large distances. We handle this situation by running our fitting operation on large (1.2 km)
tiles that overlap and feathering the margins.

5

Robustly estimating the water
level

Results

We present three sets of results. First, we compare our approach with a traditional, image-based CNN, in the context of water classification. Second, we present the endto-end results of using our system to perform volumetric
classification of water, to form the 2D mask, and to fill
missing values. Finally, we show the generality of our
system by performing tree classification.

Having obtained a 2D mask of regions containing water,
we can now remove the noisy reconstructed surface from
the water region. Once we remove the noisy surface, we
still need to fill in this region with an estimated water surface. Since the altitude of the water itself cannot be obtained from the stereo reconstruction, we instead estimate
the altitude of the shoreline, and assume that a local body
of water is planar.
Planarity assumption for a local body of water is reasonable – in fact the water should be horizontal for region
of no flow. To determine the shoreline altitude, we use
the 2D mask obtained by Section 3 and we collect samples from all depth map points whose horizontal position
is within 40 m of the shoreline. We group the samples
into connected components of shoreline pieces. For each
component we fit a plane using RANSAC that robustly
represents the most probable water level on the bounds
of the water region. Since our system works on a huge
scale, we do this step locally in each tile separately. In
our fitting process we use a (tiny) prior that prefers levels

5.1

Comparison
with
image
based
Inception-V3 and the power of multiview correspondence

A natural question about our system is: how much does
the volumetric and multi-view nature of the setup really help compared to standard image classification? The
patches shown in Figure 4 suggest that distinguishing between land and water patches solely by using image data
is difficult if not impossible. Using larger context, labelling images with the boundaries as in several semantic
segmentation works can definitely help but it is not practical in huge training sets that must be labelled. As mentioned above, context is not always achievable in world7

wide applications. It is intuitive then, that the coherency
and relationship between corresponding views is valuable.
But we wanted to test this intuition.
We applied direct comparisons with the state of
the art image classification network – the Inception-v3
model [17]. Of course, one can claim that we do have
the poses of the images so in principle, pixel-level correspondence is possible – indeed, this is part of our contribution. Moreover, along with the pixel correspondences
we also apply 3D convolution to exploit these correspondences better. A fair comparison would then be to use
exactly the same training data with the same amount of information but with no correspondence, replacing our volume with a set of images (as projected to all images) given
to the Inception-V3. This verifies how much information
is captured in the correspondence and correlation between
the projections. We use the greyscale values of the source
image at the depthmap points. Colors could be used as
well with increasing time.
We conducted three experiments to analyse the effect
of multiview correspondence which is the core of our
method. In all experiments we have used the same amount
of training data but with different degrees of correspondence. As mentioned, the core data for our 3D CNN is a
volume in space where we project every point to all available images getting grayscale for each such projection.
The volume has space size 13x13x2 (for a 25cm grid) and
it is centred at each depth point on the surface computed
by our 3D plane-sweep reconstruction. Each cell is projected to 8 available cameras so the overall size of the volume is 13x13x16. The three experiments are as follows:

(a)

(b)

(c)

Figure 10: The structure of the training data in all 3 experiments: (a) No correspondence - Unrelated 16 images
per each volume. (b) Patch correspondence - One image
containing 16 13x13 patches. (c) Pixel correspondence 3D volume

• Full pixel correspondence: This is our 3D volumetric classification having the original volume as our
training vectors. The volume creates pixel correspondence as corresponding pixels lie in the same
volume column and we apply 3D convolution.
Figure 10 shows the three options of creating training
data. The data set for this comparison is composed of
20,000 samples from our entire set. We also take another
1/10 for test which is used only at the end of the training.
Note that here we have two classes and the error weight
for each is the same so accuracy is sufficient to evaluate the performance and no precision/recall evaluation
is needed. Our supplemental material shows a complete
large scale reconstruction of land and water with difficult
appearances. The ability to create smooth water regions
in the final textured model comes from the large-scale error free water classification. Based on our classification
we replace the noisy water with a smooth surface.
A comparison of the 3 experiments can be viewed in
Figure 11. We show training accuracy and validation accuracy.

• Unrelated patches with Inception-v3: We take every
z-slize in the volume and create a 13x13 patch out of
it. The training set contains all of these patches. The
term unrelated means that they are not connected to
each other in anyway and each one of them is an independent patch. We indeed expect this to be very
hard with no context and no multivire correspondence.

5.2

• Patch correspondence: As above but here we create a new image built from 4x4 such small patches.
The training data thus contains images of size 52x52,
each of which is a concatenation of 4x4 small 13x13
patches.

Large-scale water classification

We used our classification system to implement an endto-end system to find and fill water gaps in a world-scale
3D reconstruction task. See Figures 13–15 for examples
of our water detection and filling algorithms running at
scale on various data sets.
8

Figure 12 demonstrates how our trained model works
on a source image with both tree and non-tree regions.

(a) Train accuracy

6

(c) Validation accuracy

In this paper, we presented a method for multi-view 3D
classification. Motivated by reconstruction of aerial imagery, we extended this method to perform a consistent
2D classification. While the method is general, we explored it in the context of two applications of high practical value: classifying water, and classifying trees. For the
example of water, we added a technique to fill gaps. We
demonstrated the viability of our method with large-scale
experimental results on real-world data. We also demonstrated by comparison that the multi-view nature of our
setup is necessary to achieve these results.

Figure 11: Train and Validation accuracy of the 3 experiments. The test accuracy is 79.2 for no correspondence,
90.2 with patch correspondence and 96.3 with 3D CNN.

5.3

Conclusions

Classifying trees

To demonstrate the generality of our approach, we also
applied our technique to classify trees. As with the water
classification, we avoided onerous labelling of individual
images by drawing polygons in all-tree and no-tree parts
of the map, and using any imagery within those regions
for training. Eight such polygons were used in this experiment.

(a)

(b)

Figure 12: Our approach used to classify trees. An example image is shown in (a) and the resulting classification
in (b).
9

(a)

(b)

(c)

(d)

Figure 13: (a) A set of imagery covering a 14 km wide region near Niagara Falls, with a large variety of different
water bodies. (b) 2D water classification mask for this area aggregated from individual classifications of thousands of
images (yellow = water, black = land). (c) Raw 3D reconstruction of a portion of this area. (d) 3D reconstruction after
applying our water mask and water level estimation.

(a)

(b)

(c)

(d)

Figure 14: (a) A 1 km wide inset of an imagery collection on the north coast of France. (b) 2D water classification
mask for this area. (c) 3D reconstruction of the jetties without water classification. (d) 3D reconstruction of the jetties
with water classification.

(a)

(b)

(c)

(d)

Figure 15: (a) A challenging area that is quite different from our training data. This 4 km wide part of San Francisco
Bay contains salt ponds with narrow roads between them. Micro-algae in the water make the coloration unusual. (b)
2D water classification mask for this area. Note that the mask ends where the water becomes mud. Also, we are
able to recover fine details such as the pillars of electrical towers. (c) Detail showing a 200 m wide region of our 3D
reconstruction, without applying the water mask. (d) Reconstruction after applying the water mask.

10

References
[1] I. S. A. Krizhevsky and G. E. Hinton. Imagenet classification with deep convolutional neural networks. 2012.
Advances in neural information processing systems, pages
10971105. 3
[2] M. Abadi et al. Tensorflow: A system for large-scale machine learning, 2016. CoRR abs/1605.08695. 6
[3] A. Bearman, O. Russakovsky, V. Ferrari, and L. Fei-Fei.
What’s the point: Semantic segmentation with point supervision. ECCV, (7) 549-565, 2016. 3
[4] C. Hane, C. Zach, A. Cohen, R. Angst, and M. Pollefeys.
Joint 3d scene reconstruction and class segmentation. In
Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, pages 97–104, 2013. 3
[5] S. R. K. He, X. Zhang and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet
classification. 2015. arXiv preprint arXiv:1502.01852. 3
[6] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. In Proceedings of the IEEE, pages 2278–2324, 1998. 3, 6
[7] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional
networks for semantic segmentation. CVPR. 3
[8] A. K. I. S. R. S. Nitish Srivastava, Geoffrey E. Hinton.
Dropout: a simple way to prevent neural networks from
overfitting, 2014. Journal of Machine Learning Research
15(1): 1929-1958. 6
[9] D. Pathak, P. Krähenbühl, and T. Darrell. Constrained convolutional neural networks for weakly supervised segmentation. In ICCV, 2015. 3
[10] D. Pathak, E. Shelhamer, J. Long, and T. Darrell. Fully
convolutional multi-class multiple instance learning. In
ICLR Workshop, 2015. 3
[11] N. Qian. On the momentum term in gradient descent learning algorithms, 1999. Neural Networks : The Official
Journal of the International Neural Network Society, 12(1),
145151. 6
[12] C. Ruizhongtai et al. Volumetric and multi-view cnns for
object classification on 3d data. 2016. CVPR 2016: 56485656. 3
[13] S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and
R. Szeliski. A comparison and evaluation of multi-view
stereo reconstruction algorithms. In Proceedings of the
2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1, CVPR ’06, pages
519–528, Washington, DC, USA, 2006. IEEE Computer
Society. 1, 3, 4
[14] K. Simonyan and A. Zisserman. Very deep convolutional
networks for large-scale image recognition. 2014. arXiv
preprint arXiv:1409.1556. 3

11

[15] P. N. Stuart J. Russell. Artificial intelligence - a modern
approach (3. internat. ed.), 2010. Pearson Education 2010,
ISBN 978-0-13-207148-2, pp. I-XVIII, 1-1132. 6
[16] C. Szegedy et al. Going deeper with convolutions. 2015. In
Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, pages 19. 3
[17] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer
vision. 2015. CoRR. 8
[18] G. P. Vangelis Metsis, Ion Androutsopoulos. Spam filtering with naive bayes, 2006. CEAS. 7
[19] T. Wang, B. Han, and J. Collomosse. TouchCut: Fast image and video segmentation using single-touch interaction.
Computer Vision and Image Understanding, 120:14–30,
Mar. 2014. 3
[20] O. V. Y. Boykov and R. Zabih. Fast approximate energy
minimisation via graph cuts, 2001. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 29, 12221239.
4
[21] J. Zbontar and Y. LeCun. Computing the stereo matching
cost with a convolutional neural network. In The IEEE
Conference on Computer Vision and Pattern Recognition
(CVPR), June 2015. 6

