Neuroevolution on the Edge of Chaos
Filip Matzner

Charles University in Prague
Faculty of Mathematics and Physics
Malostranské náměstı́ 25
Prague, Czech Republic 118 00
floop@floop.cz

CCS CONCEPTS
•Computing methodologies → Neural networks; •Mathematics of computing → Time series analysis; Information theory;

KEYWORDS
recurrent neural networks, echo state networks, neuroevolution,
edge of chaos, phase transition
ACM Reference format:
Filip Matzner. 2017. Neuroevolution on the Edge of Chaos. In Proceedings of
the Genetic and Evolutionary Computation Conference 2017, Berlin, Germany,
July 15–19, 2017 (GECCO ’17), 8 pages.
DOI: http://dx.doi.org/10.1145/3071178.3071292

1

INTRODUCTION

Even though recurrent neural networks seem to provide an efficient
way to process data in biological organisms, artificial models of
recurrent networks are currently considered to be more difficult to
train than their feed-forward counterparts [17]. In feed-forward
networks, input data propagate through the network in the feedforward manner only, hence the data pass through each neuron
only once. In the case of recurrent neural networks, the data might
flow through the network back and forth for an extended time
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
GECCO ’17, Berlin, Germany
© 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM.
978-1-4503-4920-8/17/07. . . $15.00
DOI: http://dx.doi.org/10.1145/3071178.3071292

time

time

Echo state networks represent a special type of recurrent neural networks. Recent papers stated that the echo state networks maximize
their computational performance on the transition between order
and chaos, the so-called edge of chaos. This work confirms this
statement in a comprehensive set of experiments. Furthermore,
the echo state networks are compared to networks evolved via
neuroevolution. The evolved networks outperform the echo state
networks, however, the evolution consumes significant computational resources. It is demonstrated that echo state networks with
local connections combine the best of both worlds, the simplicity
of random echo state networks and the performance of evolved
networks. Finally, it is shown that evolution tends to stay close to
the ordered side of the edge of chaos.

time

arXiv:1706.01330v1 [] 5 Jun 2017

ABSTRACT

neuron

(a) Ordered net
activity tends to
die out.

neuron

(b) Edge of chaos
net produces repeating patterns.

neuron

(c) Chaotic net
output
resembles
random
white noise.

Figure 1: Three echo state networks tuned for ordered, edge
of chaos and chaotic dynamics (left to right). Each row represents the activities of 50 neurons in the recurrent part of
the network. The network was driven by an input sequence
of 5 random numbers and 45 zeros.
period, each neuron might process the data multiple times, and the
data from different time points can be combined to build various
time dependent relations. In other words, recurrent networks implicitly benefit from memory capability, that allows them to solve
various non-Markovian tasks1 without being explicitly provided
by additional inputs.
Recurrent networks are harder to train, however, they provide a
set of very desirable properties. To overcome the cumbersome training, Jaeger developed a new approach known as echo state networks
[7]. This model significantly speeds up training and avoids most of
the training pitfalls at the cost of decreased adaptation ability. The
key element in echo state networks is a large, randomly generated
recurrent network. Echo state networks rely on the assumption that
this large random network nonlinearly transforms the input into
so many variations that extraction of useful information becomes
simple.
There are, however, some restrictions to the random weights
in order for the recurrent network to behave reasonably. When
the weights are too large, the network’s output resembles a white
noise and it is called to have a chaotic dynamics (Figure 1c). In
the opposite case, where the weights are too small, the activity
of the network tends to die out and it is called to have an ordered
dynamics (Figure 1a). In neither of the two cases is it possible to
1 In Markovian tasks the input in a single time point provides a complete state informa-

tion necessary to solve the task, opposite to the non-Markovian tasks, where a longer
history of input values might be required. For instance, deducing whether a car has
reached its known destination point, knowing the position of the car in each time step,
is a simple Markovian task. However, if only the velocity of the car in each time step
is known, the task is non-Markovian, since the velocity has to be integrated over time
to calculate the position.

2

METHODS

In this section, all the methods used in the experiments will be
explained to a greater depth.

3.1

Echo State Networks

An echo state network, defined by Jaeger [7], consists of a recurrent
−
network with weight matrix W , a vector of input coefficients →
w in ,
−
and a vector of readout coefficients →
w out (Figure 2). The activations
of the neurons in the recurrent network, the input value, and the
−a t , x t and ot , respectively.
output value in time t are denoted by →
The activations and the output value are calculated as follows:
→
−a t = tanh(W →
−a t −1 + →
−
w xt ) ,
in

t

−
−a t .
o =→
w out · →
The recurrent network and the input coefficients are generated
randomly and never change, the only part that is trained for the

desired output

x1

o1

y1

x2

o2

y2

x3

o3

y3

input sequence

PROBLEM SPECIFICATION

Our primary goal is to evolve the recurrent part of echo state networks using neuroevolution and analyze the computational power
of such evolved networks and their relation to the edge of chaos.
Both subjects, the computational power and the edge of chaos relation, will be compared with the corresponding properties of the
original, pure random, echo state networks.
Echo state networks in combination with the edge of chaos
suggest an idea of how biological networks might achieve such an
unbeaten performance. Instead of training each neural synapse
separately, the brain tissue might grow more or less randomly
and still obtain great results by remaining on the edge of chaos.
Unfortunately, the papers by Bertschinger and Natschläger [3]
and Boedecker et al. [4], which evaluate the performance of echo
state networks on the edge of chaos, only consider pure random
networks where all pairs of neurons have the same probability of
being connected. Such networks consist of a structure with no
regularities, no repeating patterns and no locality dependencies.
Such a model does not comply with the knowledge of biological
neural tissue, that might have a higher degree of regularity [21].
To allow evolution of biologically plausible networks, we will
use the HyperNEAT algorithm [5], which provides the means to
build complex regular structures. We are interested whether the
biologically plausible networks will perform comparably to their
pure random counterparts and whether their performance will
relate to the edge of chaos.
Before the main experiment, we will replicate the original results of Bertschinger and Natschläger [3] and Boedecker et al. [4]
who propose that computational power of echo state networks is
maximized on the edge of chaos.

3

time

extract anything useful out of the network. According to the papers
by Bertschinger and Natschläger [3] and Boedecker et al. [4], the
best properties are provided when the recurrent network dynamics
is on the very transition between order and chaos (Figure 1b). This
particular transition was given the name edge of chaos [11].
Similar phenomena has been observed also in the fields of cellular
automata [11], boolean networks [10], spiking networks [12], and
even biological cortical circuits [2].

Filip Matzner
predicted output

GECCO ’17, July 15–19, 2017, Berlin, Germany

input coeﬃcients

recurrent network

readout coeﬃcients

x4

y4

x5

y5

Figure 2: Illustration of an echo state network.
given problem are the readout coefficients. They are chosen so that
the predicted sequence and the desired output sequence minimize
their squared distance.
For the echo state network to work properly, the recurrent network cannot be completely random. Instead, it shall have a so-called
echo state property (sometimes called fading memory). Informally, it
means that the state of the netwok only depends on a finite history
of its inputs. A more formal definition will not be provided, as it is
equivalent to the definition of a network with ordered dynamics
described in the next section [3]. More information about echo
state networks and their training can be found in the original paper
by Jaeger [7].

3.2

Chaotic and Ordered Dynamics

Let us briefly introduce chaotic and ordered dynamics. A system in
which a sufficiently small perturbation of initial parameters disappears in a finite time is called to be ordered. A system in which a
perturbation amplifies is called to be chaotic.
In this section, we will describe a measure of chaoticity called
Lyapunov exponent (denoted by λ) in the context of neural networks. Its rationale is to let a neural network run from two slightly
perturbed initial states and measure the distance between the two
network states from that moment on. If the two network states
tend to converge, the system is in the ordered phase and λ < 0. If
they diverge, the system is in the chaotic phase and λ > 0. The edge
of chaos lies right in the middle, where the two states tend to keep
the same distance from each other and λ ≈ 0.
The formal definition of Lyapunov exponent λ is following:
λ = limt →∞ (1/t) ln (γt /γ 0 ) ,
where γt is the distance of the two initially perturbed states in
time t and γ 0 is the distance of the initial states, i.e., the size of the
perturbation.
We will adopt the algorithm by Sprott [20] for numerical estimation of Lyapunov exponent. Let us explain the process.
(1) A random sequence of 2000 values is generated to drive the
neural network.
(2) The network is run for 1000 time steps and its outputs are
discarded. This action is performed for the network to stabilize.
(3) After the 1000 steps, the network is duplicated. In the second
network, the activation value of one of its neurons is perturbed
by the value of γ 0 (in this paper γ 0 = 10−12 ).
(4) Both the networks proceed one time step forward and the distance between their states is calculated. Formally, the distance

Neuroevolution on the Edge of Chaos

y0

γ0

y3

y2

y1
γ1

GECCO ’17, July 15–19, 2017, Berlin, Germany

γ2
x

γ3
γ0

γ0

γ0

x2

1

y4

x

x4

3

x0

Figure 3: Illustration of the Lyapunov exponent algorithm.
After each time step, the distance between the original and
the perturbed network is recorded and normalized to the initial value of γ 0 .
−x t − →
−
−x t and →
−
will be denoted by γt = →
y t , where →
y t are the
activations of the neurons in the first and the second network
respectively and || · || denotes the Euclidean distance. This
distance is recorded for later use.
(5) The state of the perturbed network isnormalized
 to the initial
→
−
→
−
→
−
→
−
t
t
t
t
distance of γ , i.e., y = x +(γ /γ ) y − x . This action is
0

0

t

performed because the activation value of a neuron usually has
a limited range (e.g., [0; 1] for sigmoidal transfer function) and
therefore, the distance between the states is also limited. This
action ensures that the two states do not diverge close to this
limit and also avoids numerical overflows. Figure 3 visualizes
this operation.
(6) Repeat steps 4 and 5 until the end of the input sequence.
(7) Return to (3) and choose a different neuron to be perturbed.
This process is repeated for each neuron in the network. The final
Lyapunov exponent is the average logarithm of the distance of the
two trajectories, averaged over all neurons:

λ = ln γtn /γ 0 t,n ,
where γtn denotes the distance of the two states in time t and
perturbed neuron n and h·i t,n denotes the arithmetic average over
time and all neurons.
The theory behind order and chaos is, of course, much more
extensive and far beyond the scope of this paper. For more information, please refer to [19].

4

INFORMATION THEORY MEASURES

To gain an additional insight on what is happening inside a recurrent network, we will present two measures from the informationtheoretical framework defined by Lizier et al. [15]. The first measure
is called active information storage (AIS) and it denotes the average
mutual information between the past states of a random process
and its next state. Its definition is following:
(k )
Í
p(X n ,X n+1 )
(k )
AX = limk→∞ X (k ),X p(X n , X n+1 ) log2
,
(k )
n

(k )

n+1

p(X n )p(X n+1 )

where X n = {X n , X n−1 , ..., X n−k +1 } denotes the semi-infinite past
of the process X n and p(·) denotes the probability function. In the
context of neural networks, the active information storage of a
neuron measures how much does the neuron’s history influence its
future state. Self-links or transfers of information to other neurons
and back are also considered.
The second measure is called transfer entropy (TE). It always
regards two random processes, a source and a destination, between

whom is the transfer entropy measured. It denotes the amount
of information from the source which determines the value of
the destination and was not already provided in the destination’s
history. In other words, it is the mutual information between the
current state of the source process Y and the next state of the
destination process X conditioned on the history of the destination
process:
TY →X = limk →∞

Í

ωn

(k)

p(x n+1 , x n , yn ) log2

(k )

p(x n+1 |x n ,yn )
(k )
p(x n+1 |x n )

,

In the context of neural networks, the transfer entropy measures
how much does the current state of the source neuron influence
the next state of the destination neuron. This measure was first
introduced by Schreiber [18] without the limit of k → ∞, which
was suggested later by Lizier et al. [14].
The aforementioned measures represent a universal tool for analyzing random processes. Please refer to [15] for a unified overview.

5

NEUROEVOLUTION

In this section, we will present two genetic algorithms specialized
solely on neural networks. The first one is called NeuroEvolution of
Augmenting Topologies (NEAT) and the second one is its extension
called Hypercube-based NeuroEvolution of Augmenting Topologies
(HyperNEAT).
The NEAT algorithm was introduced by Stanley and Miikkulainen in 2002 and it is still widely used. It has proven to be an
efficient method to simultaneously evolve both, the weights of a
neural network and the network’s topology. Please refer to the
original paper by Stanley and Miikkulainen [22] for a thorough, yet
succinct explanation.
The original NEAT algorithm evolves each connection independently, which is sufficient for small-scale neural networks. However,
in the case of larger networks (i.e., hundreds of neurons), the number of connections is just too large for the evolution to succeed
in a reasonable time. To overcome this issue, Gauci and Stanley
exploited the idea of indirect genetic encoding and proposed an algorithm called HyperNEAT. This algorithm uses the original NEAT
to evolve a population of neural networks called compositional pattern producing networks (CPPNs). The CPPNs are later presented
with a user-defined set of neurons on a Cartesian plane, called substrate, and they are queried for the weight and the presence of each
potential connection between all pairs of neurons. The substrate
may be much larger than the CPPN itself and the CPPNs may thus
compactly represent complex structures with repeating patterns
and geometric regularities. For a more thorough description, please
refer to the original paper by Gauci and Stanley [5].

6

PERFORMANCE MEASURES

To measure the computational power of a neural network, we have
selected the following three benchmarks: Memory Capacity (MC)
[8], Nonlinear AutoRegressive Moving Average (NARMA) [4] and a
novel measure called Negative Ratio (NR). They should assess the
network’s ability to store the data into a short-term memory (MC),
operate with the memory (NARMA) and analyze it (NR).
The memory capacity (MC) task evaluates the maximum duration
for which is the network capable of remembering its inputs. The
evaluated network is driven by a single input sequence and predicts

GECCO ’17, July 15–19, 2017, Berlin, Germany
an infinite number of output sequences. The desired value of the kth output sequence yk is an exact copy of the input sequence delayed
by k time steps. For each of the output sequences, the k-delay
memory capacity is calculated as the squared Pearson correlation
coefficient between the predicted output and the desired output:
cov2 (y ,o )

k k
MCk = var(y ) var(o
,
k
k)

cov2

where
denotes the squared sample covariance, var denotes
the sample variance, yk is the k-th desired output sequence, and
ok is the k-th output sequence predicted by the network. The total
memory capacity value is the sum of these k-delayed memory
capacities:
Í
MC = k∞=1 MCk .

During our experiments, we found this measure to be numerically unstable. When one of the output sequences has a very
low variance (e.g., if the network always predicts a value close to
1.0), the MC value is unpredictable and can go up to infinity. This
may represent a problem especially in the case of evolutionary
algorithms. Whenever an evolutionary algorithm detects such an
instability of the fitness function, the instability is quickly exploited
and the evolution converges to an undesired result. For this particular reason, we propose a numerically stable alternative of the
memory capacity task called memory mean squared error (MMSE).
The evaluation of the MMSE is very similar to the evaluation
of the MC. However, this time the network predicts only a finite
number N of output sequences. The desired value of the k-th output
sequence is, again, the input sequence delayed by k time steps. The
final MMSE value is the normalized root mean squared error of
the predicted sequences with respect to the corresponding desired
output sequences, as defined in the following formula:
rD
E
MMSE =
/var(y0 ) ,
(ykt − okt )2
t,k

where ykt

and okt

are the values of the k-th desired sequence and
the k-th predicted sequence in time t, respectively. y0 is the input
sequence, and h·i t,k denotes the arithmetic average over time and
all output sequences.
In Nonlinear AutoRegressive Moving Average (NARMA) task,
the network is driven by a single input sequence and the task is to
predict the following nonlinear combination of the past 30 inputs:
Í29 t −i
y t +1 = 0.2y t + 0.004y t i=0
y
+ 1.5x t −29x t + 0.001 ,

where y t and x t are the values of the desired output sequence and
the input sequence in time t, respectively. The performance of this
task is measured using the normalized root mean squared error:
q
NARMA =
(ot − y t )2 t /var(y) ,
where ot is the value of the predicted sequence in time t and h·i t
denotes the arithmetic average over time.
In the negative ratio (NR) task, the network shall estimate the
ratio of negative numbers in the last K inputs. NR is the only of the
proposed measures, where the exact input values are not important
and instead, the values shall be conditioned on a specific property.
Formally, the desired output sequence is defined as following:
ÍK
y t = K1 i=0
neg(x t −i ) ,

Filip Matzner
where x t and y t are the values of the input sequence and the desired
output sequence in time t, respectively. neg(x) is equal to 1 iff x ≤ 0
and 0 otherwise. The performance of this task is measured using
the normalized root mean squared error:
q
NR =
(ot − y t )2 t /var(y) ,
where ot is the value of the predicted sequence in time t.

7

RANDOM ECHO STATE NETWORKS

In this section, the results of the related works by Bertschinger and
Natschläger [3] and Boedecker et al. [4] are replicated. Both papers
suggest that the computational power of randomly generated echo
state networks is maximized on the transition between order and
chaos. Furthermore, Boedecker et al. suggest that there is a peak of
active information storage and transfer entropy right on the edge
of chaos.

7.1

Experimental Settings

The experiment is conducted by generating a large set of random
echo state networks of different parameters. All the evaluated echo
state networks have 151 neurons (including the input neuron) and
use hyperbolic tangent transfer function with no bias. The weights
from the input neuron to all the other neurons are drawn uniformly
from the range of [−0.1; 0.1]. All the other weights are drawn from
a normal distribution with zero mean and variance σ . The σ values
are chosen so that log(σ ) is in the range of [−3.7; −0.8] increasing
its value by 0.02. For each σ , we generate and evaluate 10 random
networks. The input sequences for the MC task are drawn uniformly
from the range of [−1; 1]. For the NARMA task, the range is [0; 0.5].
The length of the input sequences is 3000 time steps. The first 1000
time steps are used to stabilize the network, i.e., the network is
driven by the sequence, but its outputs are discarded. The next 1000
time steps are used to train the linear coefficients and the last 1000
time steps are used to evaluate the network’s performance. The
MC is evaluated only up to the delay limit of 300 time steps.
To evaluate AIS and TE, an input sequence of length 3000 time
steps is generated uniformly from the range of [0; 0.5]. The first
1000 steps are used to stabilize the network and the remaining 2000
time steps are used to calculate the measures. A history of size 2 is
used for both the measures, instead of infinity. The AIS is averaged
over all the neurons in the network and the TE is averaged over all
the nonzero connections.
The experiment consumed approximately 120 CPU days.

7.2

Results

The memory capacity (MC) against λ is plotted in Figure 4a. The
MC increases until the edge of chaos, where it reaches its maximum
value. After the performance peak, there is a sharp drop, which
suggests that in the chaotic regime, even if it is very close to the
transition, no data survive the surrounding noise for long.
Our results of the MC task actually differ from the original paper
by Boedecker et al. [4], where the MC only rarely reached a value
higher than 10. On the other hand, our results are in accordance
with the paper by Barančok and Farkaš [1] who investigated the
effect of structured input sequences (i.e., sequences which are not
purely random) on the MC task.

Neuroevolution on the Edge of Chaos

(a) MC versus λ. Higher is better.

GECCO ’17, July 15–19, 2017, Berlin, Germany

(b) NARMA versus λ. Lower is better.

(c) MC versus λ. Comparison of fully connected networks and networks randomly
pruned to 75 and 30 connections per neuron.

Figure 4: Plots of various measures versus λ. Each cross represents a single random echo state network.
The NARMA error against λ is plotted in Figure 4b. The error
is decreasing until the edge of chaos, where it reaches its minima.
After the transition to the chaotic regime, the error sharply raises
to the same value as if the network’s output would be absolutely
random. This observation supports the idea that in the chaotic
regime, the network output resembles a random white noise.
We have evaluated also the MMSE and NR tasks and the results
are very similar. It seems that for all the evaluated tasks, the performance is indeed maximized on the edge of chaos. A rigorous
reason for this behaviour remains an open question.
Let us analyze the effects of randomly removing the majority
of the connections between the neurons. According to our simulations, such a restriction of the connectivity does not significantly
influence the network’s performance. Figure 4c demonstrates this
phenomena on the MC task and a similar pattern appears on the
other evaluated performance tasks as well. It should be noted
that restricting the number of connections while keeping the same
weights makes the network dynamics more ordered (as stated by,
e.g., Bertschinger and Natschläger [3]).
Boedecker et al. [4] provide an additional insight on what is
happening within a network on the edge of chaos. In the paper, the
AIS and TE are measured, relative to λ. The results are replicated
in Figure 5a and Figure 5b. Both of the measured entropies slowly
decrease through the ordered regime until the edge of chaos, where
they form a high sharp peak. In the chaotic regime, their value
drops and remains at its lowest level. There does not seem to be a
direct correlation between the network’s performance and the AIS
or the TE. Nevertheless, the high peak leads to the belief that there
is an unexpected irregularity on the transition between the order
and chaos.
In the original paper by Boedecker et al. [4], the AIS and TE were
plotted separately from the performance measures. Let us, instead,
draw the entropies and the performance measures to a single plot
focused tightly on the edge of chaos (Figure 5c). After a careful
analysis of the figure, it can be seen that the best performance on
the NARMA task ends immediately before the peaks of AIS and
TE. When the entropies increase, the NARMA error increases with
them. The reason may be that when the entropies reach a critical
threshold, the network performance is impaired.

8

EVOLVED ECHO STATE NETWORKS

In this section, we are going to evolve the recurrent part of echo
state networks via HyperNEAT algorithm. To avoid overfitting to
one of the given tasks, the evolution in our experiment is instructed

to maximize the performance on the MMSE and the NARMA tasks
simultaneously. We believe that these two tasks require contradictory properties of the evolved network, which may reduce overfitting by balancing between memory capacity and computational
performance. Furthermore, the NR task is hidden to the evolution
and it is instead used to validate the performance of the evolved
network on tasks never seen before. The MC task is not evaluated
at all because of its numerical instability discussed in Section 6.
The MMSE task is used to asses the network’s short-term memory
instead.

8.1

Experimental Settings

Five runs of evolution are executed, each of which evolves a population of 150 CPPNs for 2000 generations. The substrate used in
our experiment is a “golden angle spiral”, in which
p the coordinate
[X k ; Yk ] of the k-th neuron is defined as X k = k/N cos(kφ + ω)
p
and Yk = k/N sin(kφ + ω), where N is the number of neurons,
√
φ is the “golden angle” equal to π (3 − 5), and ω is the rotation
angle (i.e., the phase) of the whole spiral. The rotation angle ω is
generated randomly for each of the five evolutionary runs.
The size of the substrate is 151 neurons. A single input neuron,
whose activation always corresponds to the current input value, is
placed in the centre of the substrate, on coordinate [0; 0]. When using a CPPN to build a connection between two neurons, the CPPN
is fed by the distance between the neurons in addition to their coordinates. The networks generated on the substrate use hyperbolic
tangent transfer function with no bias. Neurons disconnected from
the input are not considered for λ, AIS and TE measures.
The fitness function of a CPPN is defined as 2 − MMSE(x) −
NARMA(x), where x is the network generated by the CPPN on the
aforementioned substrate. The fitness value is evaluated three times
and the results are averaged. For the first 15 generations of a life
of a genome, its fitness is boosted by 10%. The number of species
is kept between 5 and 10. The difference between two genomes
is defined as F /N + W /2, where N is the number of genes in the
larger genome, F is the number of non-matching genes, and W is
the average weight difference of matching genes. The difference
threshold δ t for creating a new species begins at 2.0 and may be
dynamically increased or decreased in each generation by 0.3.
Only the fittest 25% genomes of a species are allowed to reproduce. The elitism is set to 5%. If a species does not improve its
fitness for 20 generations, it is forbidden to reproduce. To select the
best genomes, tournament selection of size four is used on the top

GECCO ’17, July 15–19, 2017, Berlin, Germany

(a) AIS versus λ.

Filip Matzner

(b) TE versus λ.

(c) AIS, TE and NARMA versus λ.

Figure 5: Information-theory measures versus λ. Each point represents a single random echo state network.
25% of the genomes in the species. Overall crossover probability is
70%. There is only a 0.01% chance of interspecies mating.
Overall mutation probability is 15%. If the mutation occurs,
the chance of adding a new neuron is 1%, the chance of adding
a new connection is 8%, the chance of removing a connection is
2%, and each weight has a 90% chance of being perturbed by a
uniformly distributed random value from the range of [−0.2; 0.2].
The probability of mutating the bias of a neuron is 1% and the
mutation process is the same as for the weight mutation.
Additionally, if the mutation occurs, the transfer function of
each neuron might mutate as well. The probability of mutating the
transfer function is 3% and in such a case, the function is replaced
by one of the following functions: hyperbolic tangent, sine, signed
step, signed gaussian, and linear transfer function.
We have evaluated a few more different configurations and the
results seemed to be insensitive to small parameter changes. The
experiment consumed approximately 80 CPU days.

8.2

Results

The evolution of the fitness value (2 − MMSE − NARMA) is plotted
in Figure 6a. All the evolutionary runs converged stably to a similar
value. The improving evolution of MMSE is depicted in Figure 6b
and NARMA manifested similar results (not depicted). Since MMSE
and NARMA form the fitness function, it is no surprise they consistently improve throughout the evolution. However, also the NR
task (Figure 6c), that was not optimized directly, slightly improved
and stabilized, suggesting that the evolved network is not overfitted
to the fitness function.
To compare the evolved networks with the orginal random echo
state networks, the best representatives of both categories are selected. The selection criteria is the average of ten evaluations of
the fitness function. The two best representatives are statistically
compared in Table 1. The performance on the NARMA task is similar for both the random and the evolved network. However, on the
MMSE task, the evolved network outperforms the random network
by more than one order of magnitude. On the NR task, which has
not been explicitly optimized by any of the two approaches, the
evolved network also performs significantly better.
Now, we will address the question whether the evolution has
any relation to the edge of chaos. Figure 7 plots NARMA versus λ
and the histogram of λ’s encountered during any of the five runs
of the evolution. Only a very few networks have strongly ordered
dynamics and their performance is rather poor. The vast majority
of evolved networks is concentrated on the ordered side of the edge
of chaos, in the range of [−0.3; 0] of λ. It is clear that the evolution
avoided chaotic regime at all costs. The results might suggest that

the ordered side of the edge of chaos is a part of the search space
favoured by evolutionary algorithms. One explanation may be, that
if the evolution is completely unable to solve the given task, it may
generate a network more or less randomly. If this random network
is on the edge of chaos, its performance is still better compared to
the performance of an overly ordered or a chaotic network.

8.3

Topology

To better understand the main difference between the best pure
random echo state network and the best evolved network, their
visualization is provided in Figure 8. The random network has
more than 23 thousand neural connections, in contrast with the
evolved network that has only 383. The evolved network only
connects neurons which are spatially close and furthermore, the
network has no intersecting connections. It should be noted that
the evolved network has only a single connection heading out of
the input neuron and this connection has a low weight compared
to the other connections in the network. The evolution of the best
network is depicted in Figure 9. We have also visualized all the
other evolutionary runs and found out that the most successful
networks share very similar visual features.

8.4

Locally Connected Echo State Networks

A natural question emerges whether the topological features of
the most successful evolved networks could be used to improve
the pure random fully connected echo state networks as well. We
will attempt to answer this question by restricting the pure random
networks to only build local connections between the neurons. We
will use the same neural substrate as in the case of evolved networks
and limit the length of the connections to 0.25. Additionally, only a
single connection heading out of the input neuron is allowed.
According to the plot of the NARMA task in Figure 10, the
performance of the locally connected networks is again maximized
on the ordered side of the edge of chaos. The other tasks manifested
similar results.
To compare locally connected and fully connected echo state
networks, we again select the best candidates of both categories
according to the fitness function (2 − MMSE − N ARMA). The best
locally connected network is visualized in Figure 11 and its comparison with fully connected networks is provided in Table 2. On the
MMSE task, the locally connected network outperforms the fully
connected network by one order of magnitude. On the NR task,
the difference is less noticeable, yet still statistically significant.
On the NARMA task, the difference is not statistically significant.
The corresponding results and p-values are provided in Table 2.

Neuroevolution on the Edge of Chaos

GECCO ’17, July 15–19, 2017, Berlin, Germany

(a) The evolution of the fitness.

(b) The evolution of MMSE error.

(c) The evolution of NR error.

Figure 6: The evolution of the best individual in a generation.

evolved
random
p-value

MMSE

NARMA

NR

Lyap. exp.

0.0094 (± 0.0009 std)
0.2625 (± 0.0098 std)
< 0.01

0.4119 (± 0.0297 std)
0.4150 (± 0.0171 std)
0.2066

0.5403 (± 0.0688 std)
0.5728 (± 0.0730 std)
< 0.01

-0.2652 (± 0.0000 std)
-0.0280 (± 0.0000 std)

Table 1: The comparison of the best evolved network and the best random echo state network on 50 full evaluation cycles
(stabilize, train, evaluate). The p-values for the hypothesis that the evolved network outperforms the random network are
calculated using one-tailed one-sample t-test.

local
full
p-value

MMSE

NARMA

NR

Lyap. exp.

0.0104 (± 0.0003 std)
0.2625 (± 0.0098 std)
< 0.01

0.4173 (± 0.0171 std)
0.4150 (± 0.0171 std)
0.1318

0.5325 (± 0.0653 std)
0.5728 (± 0.0730 std)
< 0.01

-0.1451 (± 0.0000 std)
-0.0280 (± 0.0000 std)

Table 2: The comparison of the best locally connected echo state network and the best fully connected echo state network
on 50 full evaluation cycles (stabilize, train, evaluate). The p-values for the hypothesis that the performance of the locally
connected network and the fully connected network differ are calculated using two-tailed one-sample t-test.

Figure 7: Statistics over all the five evolutionary runs. Red NARMA versus λ; each cross denotes a single evaluation of
an evolved network. Blue - the histogram of λ’s.

Figure 8: Visualization of the best random (left) and the best
evolved (right) echo state networks. Green color represents
positive weights and red color represents negative weights.

Figure 9: Visualization of the fittest networks of each 250th
generation of the most successful evolutionary run.

Figure 10: Plot of the NARMA error versus λ. Lower is better.

GECCO ’17, July 15–19, 2017, Berlin, Germany

Filip Matzner
implementation using massively parallel operations. Such an implementation may allow for significantly larger networks while
keeping the same computational costs.

ACKNOWLEDGMENTS

Figure 11: Illustration of the best random echo state network with local connectivity. There is a single connection
heading out of the input neuron to its closest neighbor, however, it is so weak that it is barely visible.
To summarize the results, the performance of the best locally connected network is close to the performance of the best evolved
network. Locally connected networks provide a convenient alternative to neuroevolution when the available time and resources are
limited.

9

CONCLUSIONS

Echo state networks represent a fast and powerful approach to time
series analysis and prediction. However, it is difficult to choose the
right set of parameters for this approach to maximize its computational performance. To simplify the parameter selection, it was
stated that the performance of echo state networks is maximized
when the network’s dynamics is on the transition between order
and chaos. We have confirmed this statement in a comprehensive
set of experiments. A rigorous reason for this behaviour remains
an open question.
Even though the echo state networks were designed as a model
of biological brain, their fully connected topology does not appear
to be biologically plausible. We have addressed this issue via evolutionary algorithms and created a network with a more “organic”
layout. The evolved network turned out to significantly outperform
the fully connected echo state networks. Furthermore, we have
demonstrated that the evolution favoured the ordered side of the
edge of chaos and avoided chaotic and overly ordered networks.
We have transferred the properties of the most successful evolved
networks back to the original echo state networks and introduced
an approach called locally connected echo state networks. This model
has also proven to significantly outperform the fully connected
networks and provides a convenient alternative to neuroevolution
when the computational resources are limited.

10

FUTURE WORK

For the comparison with other methods from the literature, both
the proposed models need to be evaluated using a well known
benchmark. An example of such a benchmark are the LSTM tasks
defined by Hochreiter and Schmidhuber [6] on which the fully
connected echo state networks have already been evaluated by
Jaeger [9]. Moreover, the proposed models may be evaluated on a
set of real-world problems, such as speech prediction and music
prediction (similarly to Martens and Sutskever [16]).
Locally connected networks have a low number of connections
with a regular structure. This opens new perspectives for an efficient

This research was supported by Charles University GA UK project number 1578717
and SVV project number 260 453.
Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum, provided under the programme
“Projects of Large Research, Development, and Innovations Infrastructures” (CESNET
LM2015042), is greatly appreciated.
We thank the authors of MultiNEAT (C++/Python), JIDT [13] (Java), and SciPy
(Python) software packages for sharing their hard work under permissive open source
licences.

REFERENCES
[1] Peter Barančok and Igor Farkaš. 2014. Memory Capacity of Input-Driven Echo
State Networks at the Edge of Chaos. In International Conference on Artificial
Neural Networks. Springer International Publishing, 41–48.
[2] John M Beggs. 2008. The Criticality Hypothesis: How Local Cortical Networks
Might Optimize Information Processing. Philosophical Transactions of the Royal
Society of London A: Mathematical, Physical and Engineering Sciences 366, 1864
(2008), 329–343.
[3] Nils Bertschinger and Thomas Natschläger. 2004. Real-Time Computation at the
Edge of Chaos in Recurrent Neural Networks. Neural computation 16, 7 (2004),
1413–1436.
[4] Joschka Boedecker, Oliver Obst, Joseph T. Lizier, N. Michael Mayer, and Minoru
Asada. 2012. Information Processing in Echo State Networks at the Edge of
Chaos. Theory in Biosciences 131, 3 (2012), 205–213.
[5] Jason Gauci and Kenneth Stanley. 2007. Generating Large-Scale Neural Networks
Through Discovering Geometric Regularities. In Proceedings of the 9th annual
conference on Genetic and evolutionary computation. ACM, 997–1004.
[6] Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory.
Neural computation 9, 8 (1997), 1735–1780.
[7] Herbert Jaeger. 2001. The “Echo State” Approach to Analysing and Training
Recurrent Neural Networks. Bonn, Germany: German National Research Center
for Information Technology GMD Technical Report 148 (2001).
[8] Herbert Jaeger. 2002. Short Term Memory in Echo State Networks. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 152 (2002).
[9] Herbert Jaeger. 2012. Long Short-Term Memory in Echo State Networks: Details of
a Simulation Study. Technical Report No. 27. Jacobs University.
[10] Stuart A Kauffman. 1993. The Origins of Order: Self-Organization and Selection in
Evolution. Oxford University Press.
[11] Chris G Langton. 1990. Computation at the Edge of Chaos: Phase Transitions
and Emergent Computation. Physica D: Nonlinear Phenomena 42, 1-3 (1990),
12–37.
[12] Robert Legenstein and Wolfgang Maass. 2007. Edge of Chaos and Prediction of
Computational Performance for Neural Circuit Models. Neural Networks 20, 3
(2007), 323–334.
[13] Joseph T. Lizier. 2014. JIDT: An Information-Theoretic Toolkit for Studying the
Dynamics of Complex Systems. Frontiers in Robotics and AI 1, 11 (2014).
[14] Joseph T. Lizier, Mikhail Prokopenko, and Albert Y. Zomaya. 2008. Local Information Transfer as a Spatiotemporal Filter for Complex Systems. Physical
Review E 77, 2 (2008), 026110.
[15] Joseph T. Lizier, Mikhail Prokopenko, and Albert Y. Zomaya. 2014. A Framework
for the Local Information Dynamics of Distributed Computation in Complex
Systems. In Guided Self-Organization: Inception. Springer Berlin Heidelberg,
115–158.
[16] James Martens and Ilya Sutskever. 2011. Learning Recurrent Neural Networks
with Hessian-Free Optimization. In 28th International Conference on Machine
Learning (ICML-11). 1033–1040.
[17] Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the Difficulty of
Training Recurrent Neural Networks. (2013), 1310–1318.
[18] Thomas Schreiber. 2000. Measuring Information Transfer. Physical Review Letters
85, 2 (2000), 461–464.
[19] Julien C. Sprott. 2003. Chaos and Time-Series Analysis. Oxford University Press.
[20] Julien C. Sprott. 2015. Numerical Calculation of Largest Lyapunov Exponent.
(2015). http://sprott.physics.wisc.edu/chaos/lyapexp.htm Accessed online: 201605-25.
[21] Larry Squire, Darwin Berg, Floyd E. Bloom, Sascha Du Lac, Anirvan Ghosh, and
Nicholas C. Spitzer. 2013. Fundamental Neuroscience (4th ed.). Academic Press.
[22] Kenneth O. Stanley and Risto Miikkulainen. 2002. Evolving Neural Networks
Through Augmenting Topologies. Evolutionary Computation 10, 2 (2002), 99–127.

