Computer Vision and CIFAR10
Computer vision has been a target for many Deep Learning models that perform tasks such as Image Classification […] Continue reading

photo by: Juan Antonio Piñera García

Computer vision has been a target for many Deep Learning models that perform tasks such as Image Classification and Image Captioning among others. *CIFAR10* is another dataset with a long history in computer vision and machine learning. It consists of a set of 60,000 color images of size 32×32 pixels, each belonging to one of ten categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.

State-of-the-art deep learning methods for this dataset are as good as humans at classifying these images. In this post, however, we will cover a simpler method that will obtain decent performance.

Loading the CIFAR10 Dataset

The first step we need to take is to obtain the training data we’ll use later to train the model. In order to do that we need to download the Python version of the dataset and extract the files into a local directory. You should now have the following files:

data_batch_1, data_batch_2, data_batch_3, data_batch_4, data_batch_5

test_batch

batches_meta

readme.html

The data_batch_X files are serialized data files containing the training data, and test_batch is a similar serialized file containing the test data. The batches_meta file contains the mapping from numeric to semantic labels. The .html file is a copy of the CIFAR-10 dataset’s web page. The code is based upon the guide done by Tom Hope in his book about building Deep Learning Systems.

Since this is a relatively small dataset, we load it all into memory:

class CifarLoader(object):

def __init__(self, source_files):

self._source = source_files

self._i = 0

self.images = None

self.labels = None def load(self):

data = [unpickle(f) for f in self._source]

images = np.vstack([d["data"] for d in data])

n = len(images)

self.images = images.reshape(n, 3, 32, 32).transpose(0, 2, 3, 1)\

.astype(float) / 255

self.labels = one_hot(np.hstack([d["labels"] for d in data]), 10)

return self



def next_batch(self, batch_size):

x, y = self.images[self._i:self._i+batch_size],

self.labels[self._i:self._i+batch_size]

self._i = (self._i + batch_size) % len(self.images)

return x, y

In the previous code, we need to make use of the following utility functions:

def unpickle(file):

with open(os.path.join(DATA_PATH, file), 'rb') as fo:

dict = cPickle.load(fo)

return dict

def one_hot(vec, vals=10):

n = len(vec)

out = np.zeros((n, vals))

out[range(n), vec] = 1

return out

The unpickle() function returns a dict with fields data and labels, containing the image data and the labels, respectively. one_hot() recodes the labels from integers (in the range 0 to 9) to vectors of length 10, containing all 0s except for a 1 at the position of the label.

As a final step for the data loading, let’s create a data manager that includes both the training and the test data:

class CifarDataManager(object):

def __init__(self):

self.train = CifarLoader(["data_batch_{}".format(i)

for i in range(1, 6)])

.load()

self.test = CifarLoader(["test_batch"]).load()

Convolutional Neural Network

We will use a rather simple Convolutional neural network that will take as input the 32x32x3 images contained inside the CIFAR10 images, add two convolutional layers, each followed by a pooling layer. After that, we flatten the output and add a couple of fully connected layers used for classification into the CIFAR10 class labels.

The following code represents the model:

cifar = CifarDataManager() x = tf.placeholder(tf.float32, shape=[None, 32, 32, 3])

y_ = tf.placeholder(tf.float32, shape=[None, 10])

keep_prob = tf.placeholder(tf.float32)

conv1 = conv_layer(x, shape=[5, 5, 3, 32])

conv1_pool = max_pool_2x2(conv1) conv2 = conv_layer(conv1_pool, shape=[5, 5, 32, 64])

conv2_pool = max_pool_2x2(conv2)

conv2_flat = tf.reshape(conv2_pool, [-1, 8 * 8 * 64]) full_1 = tf.nn.relu(full_layer(conv2_flat, 1024))

full1_drop = tf.nn.dropout(full_1, keep_prob=keep_prob) y_conv = full_layer(full1_drop, 10)

cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y_conv,y_)) train_step = tf.train.AdamOptimizer(1e-3).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

Running the model with the CIFAR data

Now that we have defined the model, we must create a TensorFlow session that will instantiate such a model and then feed it the training data sequentially to slowly train it. Also, we could use a way of knowing just how good is our model doing with the test data, which means data the model did not see while training. The following code does precisely that:

def test(sess):

X = cifar.test.images.reshape(10, 1000, 32, 32, 3)

Y = cifar.test.labels.reshape(10, 1000, 10)

acc = np.mean([sess.run(accuracy, feed_dict={x: X[i], y_: Y[i], keep_prob: 1.0})

for i in range(10)])

print "Accuracy: {:.4}%".format(acc * 100) with tf.Session() as sess:

sess.run(tf.global_variables_initializer()) for i in range(STEPS):

batch = cifar.train.next_batch(BATCH_SIZE)

sess.run(train_step, feed_dict={x: batch[0], y_: batch[1], keep_prob:0.5}) test(sess)

Once you have trained this model, it will achieve approximately 70% accuracy within a few minutes of training. As of now, the state-of-the-art deep learning methods achieve over 95% but using much larger models and usually many, many hours of training. If you are interested in seeing another example of TensorFlow being applied to classification problems, check out this other post.

https://technopremium.com/blog/
['Techno Premium']
['Machine Learning', 'Technology', 'Benchmark', 'Computers', 'Deep Learning']