> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

1

A Trainable Neuromorphic Integrated Circuit
that Exploits Device Mismatch
Chetan Singh Thakur (Student Member, IEEE), Runchun Wang, Tara Julia Hamilton (Member, IEEE),
Jonathan Tapson (Member, IEEE) and André van Schaik (Fellow, IEEE)
Abstract— Random device mismatch that arises as a result of
scaling of the CMOS (complementary metal-oxide semiconductor) technology into the deep submicron regime degrades
the accuracy of analogue circuits. Methods to combat this
increase the complexity of design. We have developed a novel
neuromorphic system called a Trainable Analogue Block (TAB),
which exploits device mismatch as a means for random
projections of the input to a higher dimensional space. The TAB
framework is inspired by the principles of neural population
coding operating in the biological nervous system. Three
neuronal layers, namely input, hidden, and output, constitute the
TAB framework, with the number of hidden layer neurons far
exceeding the input layer neurons. Here, we present
measurement results of the first prototype TAB chip built using a
65nm process technology and show its learning capability for
various regression tasks. Our TAB chip exploits inherent
randomness and variability arising due to the fabrication process
to perform various learning tasks. Additionally, we characterise
each neuron and discuss the statistical variability of its tuning
curve that arises due to random device mismatch, a desirable
property for the learning capability of the TAB. We also discuss
the effect of the number of hidden neurons and the resolution of
output weights on the accuracy of the learning capability of the
TAB.
Index Terms— Neuromorphic Engineering; Analogue
Integrated Circuit Design; Stochastic Electronics; Neural
Network Hardware.

I. INTRODUCTION

O

ver time, electronic devices have witnessed a higher
packing density of transistors in accordance with
Moore’s law [1]. Owing to this, computing devices have
become smarter, faster, and more efficient. The increase in the
number of transistors has been made possible due to a
decrease in the minimum feature size, which has already
reduced below 22nm. However, keeping up with Moore’s law
has not been easy. In submicron technologies, factors such as
minor variations of process, external unknown fields, minor
layout changes, and leakage currents have large effects on the
performance of analogue circuits, making them difficult to
design and, thus, creating significant challenges. These effects
may be minimised by increasing device size; however, this
increases the size of an integrated circuit (IC) and as a result
increases its cost [2][3]. Further, the failure of a few
transistors may result in the failure of the entire chip,
rendering it unusable. Thus, there is a trade-off between
performance yield and costs in the submicron technology.
Neuromorphic systems, inspired by neurobiological
processing systems, offer an attractive alternative to
conventional analogue IC design technology. The brain is an

incredible computational device that surpasses today’s modern
computers in various tasks such as vision and audition. Similar
to the problems of transistor failure and device mismatch in an
IC, the brain is faced with the problems of heterogeneity of
neuronal responses to stimuli and neuronal cell death. The
biological nervous system has been able to resolve these
problems over the course of evolution, and provides an
excellent model for IC implementation.
In many regions of the brain, information is encoded by
patterns of activity occurring over populations of neurons, a
phenomenon referred to as population coding [4]. We have
developed a novel neuromorphic system called a Trainable
Analogue Block (TAB) that works in a similar manner by
using a large pool of neurons for encoding the input, and
linearly combining the neuronal responses to achieve
decoding. The TAB chip architecture explicitly uses random
device mismatch to its advantage, and is thus ideally suited for
submicron technologies. Other research groups have
previously suggested the use of random device mismatch in
their architectures [5], [6], but, to the best of our knowledge,
none of these have yet been tested on ICs. The TAB attempts
to incorporate the features of neurobiological systems, such as
low power consumption, fault tolerance and adaptive learning.
For example, the TAB can use learning to compensate for the
failure of a few transistors, thus improving the yield of the
system. Owing to adaptive learning, the designs will be
portable across technologies and applications, eliminating the
need for custom IC design for those functions that can be
implemented with our TAB. We envisage that the TAB will
contribute to a considerable speed-up in IC design by
shortening the design cycle for analogue circuits, and result in
a drastic decrease in design costs. The TAB framework may
be used to design systems that will employ hardware
variability to achieve their engineering goal, thus qualifying as
a design circuit paradigm for stochastic electronics [7]. The
TAB circuits are effectively universal function approximators
[8], thereby allowing for complex processing on a simple and
repeatable substrate.
This paper is organised as follows: Section II explains the
framework of the TAB. VLSI (Very Large Scale Integration)
implementation of the TAB is described in section III, the
algorithm setup for offline learning in section IV and
constrained algorithm in section V. We present the
measurements of the building blocks of our TAB
implementation in section VI and conclusions in section VII.

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

2

II. FRAMEWORK OF TAB
The TAB framework draws inspiration from the
phenomenon of neural population coding. In population
coding, biological neurons in several parts of the brain encode
information in a collective and distributed manner using spike
rates. The accuracy of information processing in the cortex
depends on the quality of population coding, which in turn is
affected by the heterogeneity of neuronal responses and the
shape of neuronal tuning curves [10][11][10]. The tuning
curve of a neuron is a plot of its average firing rate as a
function of the input stimulus. As examples of population
coding, neurons in monkeys, cricket, barn owl, cats, bats and
rats encode the direction of arm movements [12], the direction
of a wind stimulus [13], the direction of a sound stimulus [14],
saccade direction [15], echo delay [16] and the position of the
rat in its environment [17], respectively. The TAB framework
is designed to use neuronal tuning curves instead of individual
spikes. Further, we have used a heterogeneous population of
neurons in the TAB architecture. Heterogeneity of the tuning
curves of the neurons increase the encoding capacity of the
network [18].
In order to decode the response of a neuronal population, it
is required to combine the firing rates of neurons into a
population ensemble estimate. Generally, the tuning curve of
each neuron contributes a basis function and the best estimate
of the physical variables is computed from the sum of these
functions weighted by the spike rate occurring in each neuron
[19]. In our TAB system, we have used a similar approach to
decode the stimulus.
Accurate encoding of an input occurs when a population of
neurons covers the entire range of the input variable. This is
best achieved if the neuronal tuning curves are equally spaced,
and may be imposed in a neural system by encoding the
defined physiological properties of neurons in each
population. However, the resulting costs to the system are
unreasonably high. Instead, randomly chosen parameters from
the distribution are likely to perform an equally good
approximation [20]. Recently, Caron et al. showed the
existence of such randomness in the olfactory system, where
inputs from the glomeruli to individual Kenyon cells lack any
organisation with respect to their odour tuning, anatomical
features, or developmental origins [21]. In our TAB
framework too, we have projected the input from the input
layer neurons to the hidden layer neurons in a random manner.
Random device mismatch cannot be avoided in smaller
process technology, and instead we are exploiting it in the
TAB framework to encode the input variable.
The TAB is a feed-forward network of three neuronal
layers, namely input, hidden, and output, structured on the
LSHDI principle [22] (Fig. 1). The input layer neurons are
connected to a larger number of hidden layer neurons via fixed
random weights. Consequently, the inputs are projected
randomly and transformed to a higher dimensional feature
space by the nonlinear hidden layer of neurons. In effect, the
input data points, which are not linearly separable in their
current space, find a linear hyperplane in the higher
dimensional space that approximates a desired function as a

Fig. 1. Architecture of the TAB framework. The connections from the
input layer neurons/nodes to the non-linear hidden neurons are via random
weights and controllable offsets, O1 to OM. The hidden layer neurons are
connected linearly to the outer layer neurons via trainable weights. The
output neurons compute a linearly weighted sum of the hidden layer
values. Adapted from [18]

regression solution, or represents a classification boundary for
the input-output relationship. The output layer neurons derive
a solution by computing only a linearly weighted sum of the
hidden layer values. These linear weights are evaluated
analytically by computing the product of the desired output
values and the pseudoinverse of the hidden layer neurons
output [23]. The TAB also employs systematic offset (Oi, Fig.
1) and preferred direction (PE), both of which help to diversify
the tuning curves of the hidden layer neurons. Preferred
direction (PE) implies that the activation function for one half
of the hidden layer neurons may be tanh, and -tanh for the
other half, and this may be assigned deterministically or
randomly. Previously proposed networks based on the LSHDI
principle include the Functional–Link Net computing (FLNN)
by Pao et al in 1992 [24], the Extreme Learning Machine
(ELM) by Huang et al in 2006 [25], and the Neural
Engineering Framework (NEF) [26], which performs spikebased computation and is quite popular in the neuromorphic
engineering community.

III. VLSI IMPLEMENTATION OF TAB
In order to demonstrate that the TAB is effective in smaller
process nodes that are normally prohibitive to analogue design
(at and beyond 65 nm) [27], we have designed the TAB
prototype in a 65nm technology. Further, a substantial section
of the TAB was designed using minimum feature sizes so as to
maximise mismatch among transistor parameters. Differences
among the hidden layer neuronal responses can be enlarged by
using an additional distinct systematic offset for each hidden
layer neuron. As a proof of concept we have implemented a
single input-single output (SISO) version, with a single input

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

3

curve depending on the process variations such as offset
mismatch between the transistors in the differential pairs, bias
current mismatch due to variability in M3 and current mirror
mismatch. Each neuron may receive a systematically different
Vref in the TAB, which is a failsafe method to achieve a
distinct tuning curve for each neuron. This may be required in
the case of insufficient random variations, which is likely in
higher feature size process technology.

B. Output Weight

Fig. 2. Hidden Neuron Block. Schematic of the Hidden Neuron block
that implements the tanh nonlinear activation function for the TAB
framework. Adapted from [18]

voltage and a single output current. We elucidate below the
VLSI implementation of the major building blocks of the
TAB, namely the Hidden Neuron and the Output Weight.

A. Hidden Neuron
Evidence has shown that neurons in a population respond to
the same stimuli heterogeneously [28]. Each individual neuron
encodes the input stimuli according to its own distinct tuning
curve, which is found by presenting varied input stimuli to the
neuron and recording its firing rate. We use a differential pair
to implement a simple neuron model in the TAB. The
differential pair performs a hyperbolic tangent (tanh)
nonlinear operation on its input, similar to the sigmoidal
tuning curve of the stereo V1 neurons in the cortex [29]. M1
and M2 constitute the differential pair in the circuit (Fig. 2). Vin
(input voltage) and Vref (constant reference voltage) are the
gate voltages for M1 and M2, and influence the sharing of
currents between them. If all the MOSFETs (metal-oxide
semiconductor field-effect transistors) are operating in weakinversion and at saturation, with the slope factor (n) ranging
from 1.1 to 1.5, then the currents in transistors M1 and M2 can
be approximated as:
(1)
I1 = Ib[exp(Vin/nUT)] / [exp(Vin/nUT) + exp(Vref/nUT)]
I2 = Ib[exp(Vref/nUT)] / [exp(Vin/nUT) + exp(Vref/nUT)] (2)
where, Ib is the maximum bias current, Vin is the ramp input
voltage, Vref is the constant input voltage, and UT is the
thermal voltage.
For ideal transistors, the output currents, I1 and I2, are a
function of the input differential voltage between Vin and Vref
and their difference is identical to the tanh function. The
current I1 saturates to the maximum bias current if Vin is higher
than Vref by more than 4xUT (~100 mV). The assumptions in
(1) and (2) are true only for ideal transistors, and do not hold
true for real circuits, resulting in deviation from the tanh
curve. The current I1 is copied to Itanh via a current mirror. The
voltage at the M3 transistor, Vb, sets the bias current (~few
nanoamperes). In the TAB, each neuron has a distinct tuning

The output weight block connects the hidden layer and the
output layer via linear weights. These are controlled by a 13bit binary number that regulates the amount of current flowing
from the hidden layer neurons to the output layer neurons. We
have implemented binary weighted connections using a
splitter circuit (Fig. 3) [30]. The output from the hidden
neuron block, Itanh, is the input current for the output weight
block. Itanh is divided successively to form a geometricallyspaced series of smaller currents. A digital binary switch
controls each current branch. A fixed fraction of the current is
split off at each branch, and the remainder continues to the
later branches. There are a total of N stages in the splitter
circuit. The current at the kth stage is given by (Itanh/2k). The
master bias voltage Vgbias is the reference voltage for the pFET gates in the splitter [30]. Two transistor switches in the
lower half of the R2R block act as a binary synapse for every
branch, routing the branch current to either useful current,
Igood, or to current that goes to ground, Idump. Igood is mirrored to
generate a presynaptic current, Iout, for the output layer neuron.
Internal shift registers of the hidden neurons are connected
serially as a long chain. The shift registers are loaded with offchip calculated weights and are used to regulate the current in
the output weight block.
A TAB prototype was integrated on a 1 mm2 die and
fabricated in a 65nm process technology. This chip was
designed for a single input and a single output configuration
with 456 neuron blocks, the number of neuron blocks being
constrained by the chip area. Each neuron block integrates a
hidden neuron, an output weight block and a 13-bit shift
register, which is used for loading the learnt weights. At a
particular time, each neuron block receives the same input
voltage, Vin, which is weighted by a random weight and a
random offset arising due to process variations. Additionally,
each neuron may exhibit a distinct reference voltage, Vref, in
the differential pairs of the tanh block. This leads to different
differential voltages for each neuron block, and as a result
different currents, Itanh, are generated for each block. Each Vref
is tapped from a long poly-silicon wire, the end points of
which are connected to the top level voltage pins, Vref1 and
Vref2. The poly-silicon wire behaves as a long distributed
resistor element that acts as a voltage divider and generates
different reference voltages, Vref, for each neuron block. For
each new input, the tanh block calculates Itanh, which passes to
the output weight block. In the output weight block, Igood is
mirrored to make Iout (Fig. 3). Iout is further routed to currents

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

4

Fig. 3. Output Weight Block. Schematic of the Output Weight block, comprising a splitter circuit wherein MR and the two M2R transistors form an R2R
network, which gets repeated 13 times in the block. The octave splitter is terminated with a single MR transistor. Adapted from [18]

IoutP (positive current) or IoutN (negative current), as determined
by signW signal, which indicates the polarity of the output
weight connected between the hidden neuron and the output
neuron. IoutP and IoutN currents of each neuron block are
connected globally to each other, and they are summed up to
provide the final current that is the final output of the TAB.
We use an off-chip current-to-voltage converter and amplifier
circuits to convert the final output current into voltage for ease
of measurement.

IV. LEARNING SET-UP
The algorithm used for the offline learning of the TAB IC
is discussed here. In the TAB framework, learning is achieved
by computing output weights to train the system for desired
regression/classification tasks. The input layer of neurons
connects to a much larger hidden layer using random weights.
The output weights (between the large hidden layer and the
linear output neurons) are estimated analytically by calculating
the product of the pseudoinverse of the hidden layer
activations with the target outputs [31].
Let us consider a three-layer feed-forward TAB network
with L number of hidden neurons. Let G(.,.,.) be a real-valued
function so that G(wi(1), bi(1), oi(1), x, di(1)) is the output of the ith
hidden neuron, corresponding to the input vector x ϵ ℝm and
the random input weight vector wi(1) = (wi1(1),… wim(1)), where
wiS(1) is the weight of the connection between the ith hidden
neuron and sth neuron of the input layer, with random bias bi(1)
ϵ ℝ, both arising due to random mismatch of the transistors.
These random variables vary under a Gaussian distribution.
Preferred direction (PE), denoted as di(1) ϵ [-1,1] is added to
incorporate flexibility of changing the direction of the tuning
curve either towards positive or negative values. PE
assignment to the hidden neurons could be chosen either
randomly or deterministically. Systematic offset oi(1) ϵ ℝ is
added to ensure that each neuron exhibits a distinct tuning

curve, which is an essential requirement for learning in the
LSHDI framework [18]. The output function f(.) is given by:
L

f(x) = Ʃ

i=1

wi(2) G(wi(1), bi(1), oi(1), x, di(1))

(3)

where, wi(2) = (w1i(2),… wki(2)) ϵ ℝk is the weight vector
where wji(2) ϵ ℝ is the weight connecting the ith hidden neuron
with the jth neuron of the output layer. Here, G(.,.,.) takes the
following form:
G(wi, bi, x) = (g(wi(1).x + bi(1) + oi(1))).di(1)

(4)

where, g: ℝ  ℝ is the activation function.

Suppose, for a training data set {(xn, yn)n=1,2..C, xn = (xn1,…,
xnm) ϵ ℝm denotes the input vector, yn = (yn1,…, ynk) ϵ ℝk is its
corresponding output vector, and C is the total number of
input data patterns. Let the values of the input weight vectors,
wi(1) ϵ ℝm, the bias, bi(1) ϵ ℝ, be randomly assigned and oi(1) ϵ
ℝ, be assigned systematically. Then, the standard TAB
framework with L number of hidden neurons approximates the
input samples with zero error if and only if there exists wi(2) ϵ
ℝk such that:
L

yn = Ʃ wi(2) G(wi(1), bi(1), oi(1), di(1), xn)
i=1

(5)

where, n = 1,2,..C
The above set of equations can be rewritten in the
following matrix form as:
HW(2) = Y

(6)

where,
𝐻𝐶𝚡𝐿 =
(1)
(1)
𝐺�𝑤
1 , 𝑏1 , 𝑜1 , 𝑑1 , 𝑥1 � … … 𝐺�𝑤𝐿 , 𝑏𝐿 , 𝑜𝐿 , 𝑑𝐿 , 𝑥1 � ⎫
⎧
:
∶
⎪
⎪
:
∶
⎨
⎬
:
∶
⎪
⎪
(1)
(1)
⎩𝐺�𝑤1 , 𝑏1 , 𝑜1 , 𝑑1 , 𝑥𝐶 � … … 𝐺�𝑤𝐿 , 𝑏𝐿 , 𝑜𝐿 , 𝑑𝐿 , 𝑥𝐶 �⎭

(7)

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

𝑊 (2) LxK =

⎧
⎪
⎨
⎪
⎩

(2)

w1
:
:
:
(2)
wL

⎫
⎪
⎬
⎪
⎭

,

𝑌𝑀𝚡𝐾 =

⎧
⎪
⎨
⎪
⎩

𝑦1
∶
∶
∶
𝑦𝐶

⎫
⎪
⎬
⎪
⎭

VI. RESULTS
(8)

Here, the ith column of H will be the output of the ith
hidden neuron for all the input training data samples (x1,…,
xC). Further, the matrix H need not be a square matrix. Under
the assumption that the activation function g(.) is infinitely
differentiable, it has been shown that for fixed input weight
vectors, wi(1), and biases, bi(1), oi(1), the least squares solution
W(2) for the matrix (6) is:
W(2) = H+Y

(9)

where, H+ is the Moore-Penrose generalised pseudoinverse
of the matrix H.
Estimation of Output: A network with the determined
output weight vectors wi(2) ϵ ℝk for the randomly chosen
weight vectors wi(1) ϵ ℝm, random bias bi(1) ϵ ℝ and systematic
offset oi(1) ϵ ℝ for i = 1,2,…L will compute the estimated
output value ŷ for any input test sample x ϵ ℝm using the
following formula:
L

ŷ=Ʃ

i=1

wi(2) (g(wi(1).x + bi(1) + oi(1))) .di(1)

(10)

A. Analysis of hidden nodes and output weights
The size of a circuit grows linearly with the number of
hidden nodes and the number of bits in the output weights. We
first examined the number of hidden nodes and bits needed in
our system using software simulations. The optimum number
of hidden nodes and bits was found to vary with the desired
target function, with some functions, such as sinc(x), proving
much more difficult to learn than others, such as x2. Fig. 4A
shows the RMS error between the target function and the
learned function as a function of the number of hidden
neurons, for both y = sinc(6πx), and y = x2. The mean RMS
error was calculated for 10 simulations with different random
weights from the input to the hidden nodes. In these
simulations, we used 13-bits for the output weights. It can be
seen that the standard deviation is quite high for low number
of hidden nodes, but for a larger number of hidden nodes the
result becomes largely independent of the random weights. It
is also clear that the y = sinc(6πx) function needs significantly
more hidden nodes than y = x2 to be learned accurately.
The RMS error as a function of the number of bits used for
the output weights, for y = sinc(6πx) for a network with 100
hidden nodes is shown in Fig. 4B. Again, these are the results
of 10 simulations with different random weights from the

i=1

fgrad(W(2)) =
C
L
(1/C) Ʃ Ʃ (wi(2) g(wi(1).x + bi(1) + oi(1)). di(1) – y).x
i=1

i=1

30

sinc(x)

20

x*x

10
0
0

20

5

7

40
60
80
# Hidden nodes

100

50
RMS error

The pseudoinverse algorithm is a quick and easy method to
estimate the parameters of a linear regression problem with a
quadratic cost function. It is an unconstrained algorithm that
can compute output weights in any range, and thus is not
suited for hardware implementation of the TAB system. As
shown in Fig. 3, the R2R block acts as a current divider,
implying that the ratio of the output to the input current is
always less than 1. Thus, we used the method of least squares
to calculate the output weights, with additional constraints to
calculate weights in the range of (-1, 1). As mentioned in the
set up above, we collect matrix H (in (7)) for all the training
inputs. We created a function, fcost_grad, which calculates the
squared error cost function fcost and gradient fgrad for a given
training set. Our purpose is to estimate W(2) while minimising
the cost function fcost. The function fcost is passed to the
fmincon function of MATLAB as an argument along with the
weight constraints. We define the cost function and gradient of
the linear regression problem as:
fcost(W(2)) =
C
L
(1/2C) Ʃ Ʃ (wi(2) g(wi(1).x + bi(1) + oi(1)). di(1) – y)2 (11)

RMS error

V. CONSTRAINED ALGORITHM

i=1

5

40
30
20
10
0

(12)

The function fmincon is an optimisation solver that finds the
minimum of a constrained function. For linear regression, we
want to minimise the cost function fcost with parameters W(2)
using a given fixed training set.

9

11

13

15

#bits/weight
Fig. 4. A. Plot of the RMS error between the target function and the
learned function versus the number of hidden nodes. The error bars show
the standard deviation. B. The RMS error versus the number of
bits/weight used for the output weights for the function y = sinc(6πx). The
error bars show the variance.

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
input to the hidden nodes. Clearly, the higher the resolution in
the output weights, the closer the digital weights can approach
the weights found by the offline learning (which are realvalued numbers) and the better the function can be
implemented. From about 8-bits onwards, the standard
deviation is negligibly small, and the RMS error becomes
almost totally independent of the random weights. Increasing
the number of bits per weight is a matter of diminishing
returns, and 11-bits seem sufficient, even to learn this difficult
function.
B. TAB learning in software simulation
Here, we show the software simulation results of the TAB
network using a single input and a single output configuration.
We built the TAB network with 50 hidden neurons with 13bits
output weight, and tested its ability to learn different functions
such as sin and square, using constrained offline learning. In
the TAB, we used the tanh function as the nonlinear activation
function (tuning curve) for each hidden neuron. We used the
offline learning setup as mentioned in section IV, and the
fmincon solver from MATLAB to calculate the output weights
externally. We presented the training data to the network, each
training pair containing an input, x and an output, y. Each
input training value is randomly and systematically offset and
multiplied by random weights of a Gaussian distribution for
each hidden neuron, and is projected randomly to 50 hidden
neurons in this manner. For every input data point, we
collected the response of the hidden neurons and created a
matrix H as shown in (7). We used the constrained algorithm
to calculate the output weights. In the testing phase, we
presented the test input to the network and obtained a
predicted output. We show that the TAB system is able to
learn the various functions successfully (Fig. 5). We
normalised our input range between -1 to +1 V to simplify the
function argument. From the simulation results, we can see
that the learned function closely matches with the target
function. Fig. 5A and 5B show the results from training the
network for the sin and square functions, respectively.

6

serially as a long chain. Due to the large number of hidden
neurons, it is not feasible to have dedicated output ports to
probe the output current for each hidden neuron. Therefore,
the output current of each hidden neuron is probed indirectly
through the ‘OUT’ port of the IC one-by-one. The output
weight block behaves as a current splitter, i.e., if all the bits of
a weight are one, the output current of the output weight block
would be almost the same as its input current, and if all the
bits of the weight are zeros, the output weight block will
produce nearly zero output. We characterised each neuron
sequentially. We loaded the output weight between the hidden
neuron of interest and the ‘OUT’ port to be 13’h1FFF, and all
other weights between the remaining hidden neurons and OUT
port to be 13’h0000. The MSB (most significant bit, 13th bit
here) represents the sign of the output weight, where ‘1’
represents negative weight. Then, we provided the ramp input
to the TAB and measured the current at the output port. The
input to the TAB and the outputs of the tuning curves of a few
neurons were plotted in Fig. 6A and Fig. 6B respectively. We
provided 0.25V to the bias transistor M3 in the hidden neuron
circuit (Fig. 2), and we expected ~8-10 nA output current
according to circuit simulations. We collected tuning curves of
all the neurons and plotted the statistical variation in amplitude
and offset of the tuning curves (Fig. 6C, D).

C. Neuron characterisation in the TAB IC
A TAB prototype was fabricated in the 65nm process
technology with 456 neuron blocks. Here, we characterise the
tuning curve of each neuron to analyse the mismatch and
differences between the tuning curves of the hidden neurons
without any systematic offset by connecting Vref node (Fig.2)
of each hidden neuron to the same voltage,. Learning is better
if there is a high diversity between the tuning curves of
neurons. As shown in Fig. 6, we obtained heterogeneity in the
neuronal tuning curves due to random device mismatches and
process variations in the fabrication. Each neuron block
contains a hidden neuron and output weight block (section
III). Also, each neuron block contains a 13-bit shift register.
The shift registers of all the hidden neurons are connected
Fig. 5. Learning curves for the regression functions A. sin, B. square.
The red curve represents the target function, and the green curve
represents the learnt function.

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

7

Fig. 6. Hidden neurons characterisation results for a total of 456 neurons. A. Input to the system (red line). B. tanh tuning curves of a few neurons to show
the variation in offset and the minimum current value. C. Variation in the current amplitude of tuning curves of all the 456 neurons. D. Random offset
present in the chip across all the 456 neurons.

D. Learning capability of the TAB IC
In this section, we show the learning capability of the TAB
IC. This chip was designed for a single input and a single
output. Each neuron block contains a hidden neuron block and
an output weight block, as explained in section III. Also, each
neuron block contains a 13-bit shift register that is used to load
the offline trained weights. To calculate the output weights
externally, the output of hidden neurons for a given input
sample needs to be probed. For L hidden neurons and C
distinct input training data (x1..xC), the matrix HCxL as shown in
(7) is obtained, and the value of weights is calculated using the
offline learning setup as described in section IV. We collected
the tuning curves of all the hidden neurons sequentially by
measuring the current at the output port, as described above in
section VI.D. After collecting all the data, the output weights
were calculated using the constrained algorithm. We trained
the TAB IC for various regression tasks such as sin, cube,
square functions as shown in Fig. 7. The output for each of the
functions matched well with the desired output, indicating that
the TAB IC learnt the specific function.

E. Encoding capacity of the TAB IC due to random mismatch
We have seen the learning capability of the TAB IC
consisting of 456 hidden neurons. Variations in the neuronal
tuning curves have been shown in Fig. 6, which are the result
of random device mismatch. These variations make the tuning
curves different from each other, which is key in learning any
regression or classification task. In this section, we estimate
the actual number of hidden neurons needed to learn a
regression function, for example the sinc function.
Importantly, we will measure variations in the RMS error
from using different combinations of tuning curves encoding
the input. We use a few tuning curves out of the 456 hidden
neurons from the TAB chip in response to the ramp input for
learning the sinc function. Fig. 8 shows the distribution of the
errors for 100 different combinations of 40 hidden neurons
chosen randomly out of a total 456 neurons. The results
suggest that some combinations of hidden neurons encode
better than others. Moreover, combinations of 40 hidden
neurons out of 456 are sufficient to learn this function with a
average RMS error of 4.9%, and a standard variation of 0.95%
with respect to the RMS of the target signal. It can be

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

8

Fig. 7. Learning in the TAB chip for the functions: A. sine, B. cube, C. square. Top and bottom graph in each figure represent input to the TAB and
trained output from the TAB.

concluded that the TAB chip has a large encoding capacity as
a result of the diversity of neuronal tuning curves, which in
turn arises from random device mismatches. The TAB is thus
a unique framework that overcomes the limitations of random
device mismatches and employs them to its advantage.

VII. CONCLUSIONS
In this paper, we have presented a novel neuromorphic TAB
architecture inspired from the phenomenon of population
coding present in the nervous system, and exploits random
device mismatch (fixed-pattern mismatch) and variability in
the fabrication process to perform reliable computation. We
have presented measurement results of our first prototype IC
designed in 65nm for a single input and a single output
configuration of the TAB system. To our knowledge, we have
developed the first IC that exploits random device mismatch to
its advantage. The TAB also incorporates systematic offset as
a failsafe method to spread the tuning curves of the neurons.
Systematic offset may be required when there is insufficient

Fig. 8. Distribution of error while training the TAB for learning the sinc
function using 40 hidden neurons randomly sampled 100 times in total from
a pool of 456 neurons.

random variation among transistors to produce a distinct
tuning curve for each neuron, which is highly likely in higher
feature size process technology. We have also shown the
learning capability of the TAB system for various regression
tasks.
The TAB is inspired by neural population coding which is
very robust in the face of damage of a few neurons, and does
not have a disastrous effect on the encoded representation as
the information is encoded across many neurons. The TAB
system is designed using neuromorphic principles based on
stochastic computation, which has the advantages of low
power consumption since the circuit is operating in the
subthreshold regime, fault tolerance, adaptability to local
change and the ability to learn. We envisage the TAB to
overcome the limitations of analogue IC design at low process
nodes and drive the integration process with digital blocks in
the same circuit and process node. This may find applications
in analogue/digital converters (ADCs) and digital-to-analogue
converters (DACs) for submicron mixed signal chips such as
those used in mobile processor chips and data acquisition
chips.
The main significance of our TAB is that it solves the
problem that increased device mismatch in modern IC
manufacturing technologies causes for analogue design. It
works better with a considerable amount of device mismatch,
and accurate mappings from input values to output values can
be obtained without needing to engineer the effect of device
mismatch out of the circuit, as is done currently. A further
significant advantage of this approach is that the same TAB
may be reused for many different purposes once
manufactured, and the same architecture may be used in
different manufacturing technologies. This will lead to a
significantly reduced design cycle for analogue circuits, with
an associated reduction in design cost, and a speed-up of
technological progress. The TAB may also be (re-)trained ‘on
the job’. This could be a major advantage in systems where
the input-output mapping of a TAB needs to be changed
because of changes in the system. An example would be in a
communication system where a TAB is used as a filter to
process the analogue signal before digitisation, in which the
communication channel changes over time. The TAB can be

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
re-trained with the communication channel in the loop to
compensate for the changes. Furthermore, as the TAB
framework desires large random mismatch among devices,
and as mismatch is inversely proportional to device area, it
could lead to significant reductions in chip area and
manufacturing costs. Also, the failure of a few neurons would
not affect the performance of the TAB as information is
encoded into a large ensemble of neurons. This property
would help significantly in improving the overall yield of the
chip production. Future work will aim to design TAB chips for
multiple inputs.

[1]

[2]

[3]

[4]
[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

REFERENCES
G. E. Moore, “Cramming more components onto
integrated circuits, Reprinted from Electronics,
volume 38, number 8, April 19, 1965, pp.114 ff.,”
IEEE Solid-State Circuits Newsl., vol. 20, no. 3, pp.
33–35, Sep. 2006.
P. R. Kinget, “Device mismatch and tradeoffs in the
design of analogue circuits,” IEEE J. Solid-State
Circuits, vol. 40, no. 6, pp. 1212–1224, Jun. 2005.
A. Marshall, “Mismatch and Noise in Modern IC
Processes,” Synth. Lect. Digit. Circuits Syst., vol. 4,
no. 1, pp. 1–140, Jan. 2009.
T. J. Sejnowski, “Neural populations revealed,” vol.
332, no. March, p. 21218, 1988.
E. Yao, S. Hussain, A. Basu, and G.-B. Huang,
“Computation using mismatch: Neuromorphic
extreme learning machines,” 2013 IEEE Biomed.
Circuits Syst. Conf., no. 1, pp. 294–297, Oct. 2013.
C. Merkel and D. Kudithipudi, “A current-mode
CMOS/memristor hybrid implementation of an
extreme learning machine,” Proc. 24th Ed. Gt. lakes
Symp. VLSI - GLSVLSI ’14, no. 3, pp. 241–242, 2014.
T. J. Hamilton, S. Afshar, A. van Schaik, and J.
Tapson, “Stochastic Electronics: A Neuro-Inspired
Design Paradigm for Integrated Circuits,” Proc. IEEE,
vol. 102, no. 5, pp. 843–859, May 2014.
K. Hornik, “Approximation capabilities of multilayer
feedforward networks,” Neural Networks, vol. 4. pp.
251–257, 1991.
A. S. Ecker, P. Berens, A. S. Tolias, and M. Bethge,
“The Effect of Noise Correlations in Populations of
Diversely Tuned Neurons,” Journal of Neuroscience,
vol. 31, no. 40. pp. 14272–14283, 2011.
M. I. Chelaru and V. Dragoi, “Efficient coding in
heterogeneous neuronal populations.,” Proc. Natl.
Acad. Sci. U. S. A., vol. 105, no. 42, pp. 16344–9, Oct.
2008.
M. Rigotti, O. Barak, M. R. Warden, X.-J. Wang, N.
D. Daw, E. K. Miller, and S. Fusi, “The importance of
mixed selectivity in complex cognitive tasks.,”
Nature, vol. 497, no. 7451, pp. 585–90, May 2013.
A. P. Georgopoulos, A. B. Schwartz, and R. E.
Kettner, “Neuronal population coding of movement
direction.,” Science, vol. 233, pp. 1416–1419, 1986.
J. P. Bacon and R. K. Murphey, “Receptive fields of
cricket giant interneurones are related to their
dendritic structure.,” J. Physiol., vol. 352, pp. 601–
623, 1984.

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]
[29]

[30]

9

E. I. Knudsen and M. Konishi, “A neural map of
auditory space in the owl.,” Science, vol. 200, pp.
795–797, 1978.
J. A. Van Gisbergen, A. J. Van Opstal, and A. A. Tax,
“Collicular ensemble coding of saccades based on
vector summation.,” Neuroscience, vol. 21, pp. 541–
555, 1987.
W. E. O’Neill and N. Suga, “Target range-sensitive
neurons in the auditory cortex of the moustache bat.,”
Science (80-. )., vol. 203, pp. 69–72, 1979.
J. O’Keefe and J. Dostrovsky, “The hippocampus as a
spatial map. Preliminary evidence from unit activity in
the freely-moving rat.,” Brain Res., vol. 34, pp. 171–
175, 1971.
C. S. Thakur, T. J. Hamilton, R. Wang, J. Tapson, and
A. van Schaik, “A neuromorphic hardware framework
based on population coding,” arXiv Prepr.
arXiv1503.00505, pp. 1–8, Mar. 2015.
E. Salinas and L. F. Abbott, “Vector reconstruction
from firing rates,” J. Comput. Neurosci., vol. 1, no. 1–
2, pp. 89–107, Jun. 1994.
C. Eliasmith and C. H. Anderson, Neural Engineering
(Computational Neuroscience Series): Computational,
Representation, and Dynamics in Neurobiological
Systems. MIT Press Cambridge, MA, USA, 2002.
S. J. C. Caron, V. Ruta, L. F. Abbott, and R. Axel,
“Random convergence of olfactory inputs in the
Drosophila mushroom body.,” Nature, vol. 497, no.
7447, pp. 113–7, May 2013.
J. C. Tapson, G. K. Cohen, S. Afshar, K. M. Stiefel,
Y. Buskila, R. M. Wang, T. J. Hamilton, and A. van
Schaik, “Synthesis of neural networks for spatiotemporal spike pattern recognition and processing,”
Front. Neurosci., vol. 7, p. 153, Jan. 2013.
J. Tapson and A. van Schaik, “Learning the
pseudoinverse solution to network weights,” Neural
Netw., vol. 45, pp. 94–100, Sep. 2013.
Y.-H. Pao and Y. Takefuji, “Functional-link net
computing: theory, system architecture, and
functionalities,” Computer (Long. Beach. Calif)., vol.
25, no. 5, pp. 76–79, May 1992.
G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme
learning machine: Theory and applications,”
Neurocomputing, vol. 70, no. 1–3, pp. 489–501, Dec.
2006.
T. C. Stewart and C. Eliasmith, “Large-Scale
Synthesis of Functional Spiking Neural Circuits,”
Proc. IEEE, vol. 102, no. 5, pp. 881–898, May 2014.
A. B. Kahng, “Design Challenges at 65nm and
Beyond,” in 2007 Design, Automation & Test in
Europe Conference & Exhibition, 2007, pp. 1–2.
L. F. Abbott, “Theoretical neuroscience rising,”
Neuron, vol. 60, no. 3, pp. 489–95, Nov. 2008.
D. G. Albrecht and D. B. Hamilton, “Striate cortex of
monkey and cat: contrast response function.,” J.
Neurophysiol., vol. 48, no. 1, pp. 217–237, 1982.
T. Delbrück and A. Van Schaik, “Bias Current
Generators with Wide Dynamic Range,” Analog
Integr. Circuits Signal Process., vol. 43, no. 3, pp.
247–268, Jun. 2005.

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
[31]

J. Tapson and A. van Schaik, “Learning the
pseudoinverse solution to network weights,” Neural
Networks, vol. 5, 2013.

10

