arXiv:1303.6145v2 [] 8 Aug 2013

Particles Prefer Walking Along the Axes:
Experimental Insights into the Behavior of a Particle
Swarm
Manuel Schmitt

Rolf Wanka

Department of Computer Science
University of Erlangen-Nuremberg, Germany
{manuel.schmitt, rolf.wanka}@cs.fau.de
Abstract
Particle swarm optimization (PSO) is a widely used nature-inspired meta-heuristic for solving
continuous optimization problems. However, when running the PSO algorithm, one encounters
the phenomenon of so-called stagnation, that means in our context, the whole swarm starts to
converge to a solution that is not (even a local) optimum. The goal of this work is to point
out possible reasons why the swarm stagnates at these non-optimal points. To achieve our
results, we use the newly defined potential of a swarm. The total potential has a portion for
every dimension of the search space, and it drops when the swarm approaches the point of
convergence. As it turns out experimentally, the swarm is very likely to come sometimes into
“unbalanced” states, i. e., almost all potential belongs to one axis. Therefore, the swarm becomes
blind for improvements still possible in any other direction. Finally, we show how in the light
of the potential and these observations, a slightly adapted PSO rebalances the potential and
therefore increases the quality of the solution. Note that this is an extended version of [SW13b].

1

Introduction

Background. Particle swarm optimization (PSO), introduced by Kennedy and Eberhart [KE95,
EK95], is a very popular meta-heuristic for solving continuous optimization problems. It is inspired
by the social interaction of individuals living together in groups and supporting and cooperating
with each other. Fields of very successful application are, among many others, in Biomedical Image
Processing [WSZ+ 04], Geosciences [OD10], Mechanical Engineering [GWHK09], and Materials Science [RPPN09], to name just a few, where the continuous objective function on a multi-dimensional
domain is not given in a closed form, but by a “black box.” The popularity of the PSO framework
in these scientific communities is due to the fact that it on the one hand can be realized and,
if necessary, adapted to further needs easily, but on the other hand shows in experiments good
performance results with respect to the quality of the obtained solution and the speed needed to
obtain it. By adapting its parameters, users may in real-world applications easily and successfully
control the swarm’s behavior with respect to “exploration” (“searching where no one has searched
before”) and “exploitation” (“searching around a good position”). A thorough discussion of PSO
can be found in [PSL11].
To be precise, let an objective function f : RD → R on a D-dimensional domain be given
that (w. l. o. g.) has to be minimized. A population of particles, each consisting of a position (the
1

candidate for a solution), a velocity and a local attractor, moves through the search space RD . The
local attractor of a particle is the best position with respect to f this particle has encountered
so far. The population in motion is the swarm. In contrast to other evolutionary algorithms,
the individuals of the swarm cooperate by sharing information about the search space via the
global attractor, which is the best position any particle has found so far. The particles move in
time-discrete iterations. The movement of a particle is governed by both its velocity and the two
attractors and by some additional fixed parameters (for details, see Sec. 2).
PSO is widely used in real-world applications. It is usually desired that the swarm converges to a
single point in the search space, and that this point is at least a local optimum. It is well investigated
how the fixed parameters mentioned above should be chosen to let the swarm converge at all [Tre03,
JLY07]1 . However, experiments sometimes show the phenomenon of stagnation, meaning here that
the particle swarm sometimes converges to a single search point and gets stuck, although this point
is not even a local optimum. In [LW11], Lehre/Witt have for a certain situation formally proven that
the probability for a swarm to converge to a non-optimal point is positive. Several approaches to
deal with this stagnation phenomenon have been developed. Clerc [Cle06] examines the distribution
of the particles during the stagnation phase, derives properties of these distributions, and provides
several possibilities to adapt the algorithm for the case that the number of iterations without
an improvement reaches a certain threshold. Van den Bergh/Engelbrecht [vdBE02] substantially
modify the movement equations, enabling the particles to count the number of times they improve
the global attractor and use this information. Empirical evidence for the capability of their method
to find local optima on common benchmarks is given. Closest to our work, in [LW11] the movement
equations are modified by adding in every iteration a small random perturbation to the velocity.
New results. In this paper, we focus on the swarm’s behavior right before the convergence starts, in
order to find out about possible causes that let the swarm converge, i. e., why the global attractor
starts to stagnate. Although one would like the output of the PSO algorithm to be at least a
local optimum, we point out two possible reasons for a swarm to converge far away from any
local optimum. In order to state the causes for this premature convergence, we define a potential
that reflects the capability of the swarm to move. That means the swarm converges iff the total
potential approaches ~0. The swarm’s total potential has a portion for every dimension of the
D-dimensional search space. The experiments carried out suggest that unwanted stagnation can
indeed be explained in terms of the potential.
The first possible cause of a non-optimal limit of convergence is that, even though the global
attractor is updated frequently, the potential drops, so the swarm has not sufficient momentum to
find significantly improved points. In Sec. 3, we present experiments that demonstrate that this
phenomenon can in fact be observed for some parameter selections. Fortunately, it also turns out
that common parameter choices and a reasonable swarm size already avoid this problem.
The second and more important reason is that the potential tends to becoming imbalanced
among the dimensions, so dimensions in which only small improvement is possible may neverheless
have the by far highest potential. That means that every step in such a dimension results in a
worsening strong enough to void possible improvements in other dimensions. So, the global attractor of the swarm stagnates and the swarm starts to converge. We demonstrate that indeed the
swarm tends to reach a state where the potentials are unbalanced, i. e., one dimension gets the by
far highest portion of the total potential while all other portions are about equal. Then, we present
1

In the companion paper [SW13a] to the paper at hand, the quality of the best point found by the swarm (the
global attractor) is formally analyzed.

2

experimental evidence showing that this phenomenon makes the particles converge at non-optimal
search points. So the experiments suggest that first the swarm starts to prefer a promising direction
that is parallel to one of the axes and increases the potential in this dimension far above the potentials in the other dimensions. As soon as the chosen direction does no longer yield improvements,
its potential stays much larger than in the remaining dimensions where improvements would still
be possible. From that point on, improvements become rare and the swarm starts to stagnate,
although no local optimum is found yet.
Since the cause of this premature convergence is an imbalance of the potentials, we show how
a small, simple and easy to implement modification of the algorithm enables it to handle such
situation. Namely, if the potential is sufficiently small, we let the particles make pure random steps,
which do not favor any direction. We conclude with showing that the modification does not totally
overwrite the PSO algorithm by replacing it by some greedy random search procedure. Instead, our
experiments show that the modification is only applied in case of (premature) convergence. As long
as there is still room for improvements left in the search space, the unmodified behavior prevails.

2

Definitions

First we present the underlying model of the PSO algorithm. The model describes the positions of
the particles, the velocities and the global and local attractors as real-valued stochastic processes.
Furthermore, we define in Def. 2 the potential of a swarm that will be a measure for its movement.
A swarm with high potential is more likely to reach search points far away from the current global
attractor, while a swarm with potential approaching 0 is converging.
Definition 1 (Classical PSO Algorithm). A swarm S of N particles moves through the D-dimensional
search space RD . Each particle n consists of the following components:
• position X n ∈ RD , describing the current location of the particle in the search space,
• velocity V n ∈ RD , describing the vector of the particle’s current velocity,
• local attractor Ln ∈ RD , describing the best point particle n has visited so far.
Additionally, the swarm shares information via the global attractor G ∈ RD , describing the best
point any particle has visited so far. After some arbitrary initialization of the X n and V n (usually,
one assumes them to be initialized u. a. r. over some domain), the actual movement of the swarm
is governed by the procedure described in Algorithm 1 where f : RD → R denotes the objective
function.
Here, χ, c1 and c2 are some positive constants called the fixed parameters of the swarm, and
rand() returns values that are uniformly distributed over [0, 1] and all independent.
Note that in case of a tie between the previous attractor and the new point X n , we use the new
value, i. e., whenever a particle finds a search point with value equal to the one of its local attractor,
this point becomes the new local attractor. If additionally the function value is equal to the one
of the global attractor, this one is also updated. Also note that the global attractor is updated as
soon as a new better solution has been found.
Now we want to define a potential for measuring how close the swarm is to convergence. A
meaningful potential should of course involve the velocities of the particles. These considerations
lead to the following definition:
3

Algorithm 1: classical PSO
output : G ∈ RD
1 repeat
2
for n = 1 → N do
3
for d = 1 → D do
4
Vdn := χ · Vdn + c1 · rand() · (Lnd − Xdn )
+ c2 · rand() · (Gd − Xdn );
n
n
5
Xd := Xd + Vdn ;
6
end
7
if f (X n ) ≤ f (Ln ) then
8
Ln := X n ;
9
end
10
if f (X n ) ≤ f (G) then
11
G := X n ;
12
end
13
end
14 until termination criterion met;

Definition 2 (Potential). For d ∈ {1, . . . , D}, the potential of swarm S in dimension d is Φd with
Φd :=

N 

X
|Vdn | + |Gd − Xdn | ,

n=1

the total potential of S is Φ = (Φ1 , . . . , ΦD ).
Note that we slightly deviate from the notion of the potential found in [SW13a] since the version
in the definition above is simpler and sufficient for the present work. However, the underlying idea
is the same for both versions of the potential.
So the current total potential of a swarm has a portion in every dimension. Between two
different dimensions, the potenial may differ much, and “moving” potential from one dimension to
another is not possible. On the other hand, along the same dimension the particles influence each
other and can transfer potential from one to the other. This is the reason why we do not define a
potential of an individual particle.

3

1-Dimensional PSO

In this section, we examine the behavior of a 1-dimensional PSO with respect to the potential.
If the swarm is close to a local optimum and there is no second local optimum within range,
the attractors converge and it is well-known that with appropriate choices for the parameters of
the PSO, convergence of the attractors implies convergence of the whole swarm. Such parameter
selection guidelines can be found, e. g., in [JLY07].
On the other hand, if the swarm is far away from the next local optimum and the function is
monotone on an area that is large compared to the current potential of the swarm, the preferred
behavior of the swarm is to increase the potential and move in the direction that yields the improvement until a local optimum is surpassed and the monotonicity of the function changes. In [LW11],
4

the authors show that there are non-trivial choices of parameters for which the swarm converges
even on a monotone function. In particular, if N = 1, every parameter choice either allows convergence to an arbitrary point in the search space, or it generally prevents the one-particle-swarm
from converging, even if the global attractor is already at the global optimum.
We ran the particle swarm algorithm on a monotone function to measure the development of
the potential over time. For our experiment, we chose the 1-dimensional function f (x) = −x as
objective function wanting the swarm always “running down the hill.” Note that this choice is not a
restriction, since the particles compare points only qualitatively and the behavior is exactly the same
on any monotone decreasing function: due to the rules for updating the attractors in lines 7 and
11, resp., of Algorithm 1, the new attractors are the points with greater x-coordinate. Therefore,
we used only one function in our experiment. The parameters for the movement equations are
common choices obtained from the literature. We let the particles make 1000 iterations and stored
the potential at every iteration. We made a total of 1000 experiments for each set of parameters and
calculated both average and standard deviation. The averages are stated in Fig. 1, the standard
deviations are of the same order and therefore omitted. In cases (a), (c), (d) and (e), the particles
have shown the expected behavior, namely an exponential increase of the potential. So the swarm
keeps running down the hill which is what we want it to do.

10

Potential

10

150

(e)

100

10

(a)

50

(c)
(d)

10

0

(b)

10

-50

0

Figure 1:

200

400

600

800

1000

1200

Iteration

(a) χ = 0.729, c1 = c2 = 1.49, N = 2 [CK02]
(b) χ = 0.729, c1 = 2.8 · χ, c2 = 1.3 · χ, N = 2 [CD01]
(c) χ = 0.729, c1 = 2.8 · χ, c2 = 1.3 · χ, N = 3 [CD01]
(d) χ = 0.6, c1 = c2 = 1.7, N = 2 [Tre03]
(e) χ = 0.6, c1 = c2 = 1.7, N = 3 [Tre03]

However, in case (b) where only two particles are involved, we see the potential decreasing
exponentially because the number of particles is presumably too small. In this case, the swarm will
eventually stop, i. e., stagnate. But we also see in case (c) that using one additional particle and
not changing the remaining parameters, the swarms keeps its motion.
5

In all cases, for the small swarm size of ≥ 3, the common parameter choices avoid the problem
mentioned in [LW11].

4

D-Dimensional PSO

In the D-dimensional case, the situation is more complicated as now the relations between distinct
dimensions become important. A new problem arising is the following: Assume that the whole
swarm is close to a point x ∈ RD such that every change of the first coordinate leads to a significantly worse value of the objective function, while in the other dimensions there is still room for
improvements. Furthermore let the swarm have high potential in the first and low potential in
any other dimension. Then an improvement of the global attractor is still possible, but it is very
unlikely and between two updates are many steps without an update. The reason is that any improvement in some of the dimensions 2, ..., D is voided by the much larger worsening in dimension
1. It follows that the attractors stay constant for long times between two updates and so the swarm
tends to converge and therefore looses potential. As long as the global attractor stays constant, the
situation is symmetric in every dimension, so while converging, the imbalance is still maintained.
First, we want to examine if and how such imbalances arise. Assume that the fitness function is
(on some area) monotone in every dimension. One of our main observations is that indeed in such a
situation the swarm tends to pick one dimension and favor it over all the others. As a consequence,
the movement of the swarm becomes more andPmore parallel to one ofP
the axes.
D
We used the fitness-functions f (~x) = − D
x
and
g(~
x
)
=
−
i=1 i
i=1 i · xi which are both
monotonically decreasing in every dimension and set D to 10. Initially, we distribute the particles
randomly over [−100; 100]D and the velocities over [−50; 50]D and let the swarm make 500 iterations.
The swarm size N was 10 and the parameters were set to χ = 0.729, c1 = c2 = 1.49 as found
in [CK02]. After each iteration, we calculated the potential for each dimension. We made 1000
runs and for each run, the dimensions were sorted according to the final value of Φ, i. e., we switched
the numbers of the dimensions such that after the last iteration dimension 1 always had the highest
potential, dimension 2 the second highest and so on. We calculated the mean of the potentials over
the 1000 runs for each of the sorted dimensions. The results are stated in Fig. 2. One can see that
the dimension with the greatest potential has for both functions a value far higher than the others,
while the other dimensions do not show such a significant difference between each other. In other
words: Particles like to move parallel to an axis.
An explanation for this behavior is the following: Assume that at some time, one dimension d0
has more potential than the others. Further assume that the advance is great enough such that for
some number of steps the particle with the largest value in dimension d0 is the one that determines
the global attractor. In a companion paper, we call a swarm in this situation “running”. Since
randomness is involved and this situation has a positive probability to occur, it will actually occur
after sufficiently many iterations. Then, each update of the global attractor increases the potential
in d0 considerably because it increases the distance of every single particle to the global attractor
except for the one particle that updated it. In any other dimension d 6= d0 , the situation is different.
Here, the decision which particle updates the global attractor is stochastically independent of the
value xd in dimension d. In other words: If one looks only on the dimension d, the global attractor
is chosen uniformly at random from the set of all particles. As a consequence, after some iterations,
the d0 ’th coordinate of the velocity becomes positive for every particle, so the attraction towards
the global attractor always goes into the same direction as the velocity, while in the remaining
6

10

10

200

300

200

Potential

10

Potential

10

300

10

100

10

10

0

0

100

200

300
400
Iteration
(a) Fitness function f

500

100

10

600

0

0

100

Figure 2: Growth of potential when processing (a) f (~x) = −

200

300
400
Iteration
(b) Fitness function g

PD

i=1 xi ,

500

(b) g(~x) = −

600

PD

i=1 i ·

xi .

dimensions, the velocities may as well point away from the global attractor, meaning that the
particles will be slowed down by the force of attraction. An overview over the situation is given in
Fig. 3.
So, roughly speaking, most of the time the global attractor is somewhere in the middle of the
different xd values, giving less potential increase then in dimension d0 where it has a border position.
That means that the balanced situation is not stable in a sense that after the imbalance has reached
a certain critical value, it will grow unbounded.
If at some point no more improvements can be made in dimension d0 , the swarm is in the
situation described above where it starts to converge while the convergence is, other than the
acceleration phase, balanced. That means after the same time the potential of every dimension is
decreased by approximately the same factor, so dimension d0 has still far more potential than any
other dimension and the swarm stays blind for possible improvements in dimensions other than d0 .
To supplement the results about the behavior of the PSO in that “artificial” setting, we ran it
on two well-known benchmark functions to show that the problems described above really occurs
on actual instances. Since the described scenario may happen with positive but, depending on
the situation, small probability, we choose the number of particles N compared to the number of
dimensions D small in order to be able to view the phenomenon in a preferably pure condition.
Table 1 lists our results on the sphere function with optimal solution z ∗ = (0, ..., 0), where we
distributed the particles randomly over [−100; 100]D and the velocities over [−50, 50]D , and another common benchmark, the Rosenbrock function with optimal solution z ∗ = (1, ..., 1) (found
in [Ros60]), where the initial population was randomly distributed over [−5; 10]D and the initial
velocity over [−2.5, 5]D . The results obtained on this function are stated in Table 2. We repeated
each experiment 1000 times and calculated the means. Additionally we calculated for each rep-

7

particle
global attractor
velocity
attraction

Figure 3: Particles running in direction d0 . In dimension d0 , the differences between the coordinate
of the particle and the global attractor is on average higher than in dimension d1 . The velocities
of dimension d0 point in the direction of the global attractor.
etition the dimension with the minimal and the one with the maximal value for the potential Φ
after the last iteration (see columns Φ) and the difference between the global attractor and the
optimal solution in the dimension with the lowest resp. highest remaining potential. One can see
that the dimension with the highest value for Φ usually is much closer to its optimal value than
the dimension with the lower value. In particular, in the 2-dimensional case the potential became
0 in one dimension preventing the swarm from any movement in this direction and consequently
from finding the minimum.
Table 1: Sphere-function
D
N
tmax
Value
min. Φ
max. Φ
*

Φ
dist. opt.
Φ
dist. opt.

4
2
10000
51.04
0*
1.58
3.75 · 10−8
1.16 · 10−8

60
10
100000
12.18
5.84 · 10−62
1.32
7.53 · 10−8
1.91 · 10−9

150
20
100000
11.97
4.15 · 10−37
1.30
1.59 · 10−7
2.28 · 10−9

Due to double precision.

We calculated the standard deviation for each unit and obtained values that were of the same
order but usually higher than the mean values. The reason for this high deviations is that the
examined phenomenon occurs randomly and therefore one cannot predict the potential level of the
swarm when it occurs, i. e., the level of imbalance at the point when the swarm starts to converge
is unpredictable.
Now that we know that the swarm tends to optimize functions dimensionwise, it is interesting
to see what happens if we try it on a function that is in some area increasing in every dimension
but still decreasing in some direction not parallel to one of the axes.

8

Table 2: Rosenbrock-function
D
N
tmax
Value
min. Φ
max. Φ
*

Φ
dist. opt.
Φ
dist. opt.

4
2
10000
126.54
0*
1.1075
4.72 · 10−5
2.59

60
10
100000
34.57
6.27 · 10−5
0.37
0.93
0.11

150
20
100000
28.88
2.32 · 10−4
0.12
8.07
0.06

Due to double precision.

Fix some b > 1 and define the D-dimensional function f as follows:
 Pn
 Pi=1 x2i , ∃i, j : xi ≥ b · xj ∨ xj ≥ b · xi

 

n
f (~x) =
x2
xi
 i=1 i · 2 maxi6=j
− b − 1 , otherwise
b−1
xj

Figure 4: Continuous, not differentiable function f
In Fig. 4 one can see a plot of f for b = 1.1 and D = 2. For y not between x/b and x · b, this
function behaves like the well-known sphere-function, leading the particles close to the origin. For
x = y, f (x, y) = −2 · x2 and from y = x/b (y = x · b) to y = x, the function falls into a valley. One
can easily verify that this function is, though not differentiable, at least continuous. One would
want the particles to be able to move through the valley.
9

As in our previous experiment, we set χ = 0.729, c1 = c2 = 1.49, N = 10 and initialized the
particles uniformly at random over [−100; 100]D (except for the first particle that was initialized
at (1, ..., 1) such that the swarm could see the direction where the improvements are possible) and
the velocities over [−50; 50]D , with the value D = 3. We let the swarm do 1000 runs with 5000
iterations each. The potential of the dimension with the highest potential after the last iteration
was determined and the mean and standard deviation of the respective dimensions were calculated
over the 1000 repetitions. This was done for two different swarm sizes, namely N = 10 and N = 50.
We repeated the experiment with 10 particles and only 100 iterations, using the function frot , which
is obtained by first rotating the input vector and then applying f such that the valley
now leads
√
the particles along the x1 -axis. Formally speaking, the rotation maps the vector ( N , 0, . . . , 0) to
(1, 1, . . . , 1) and keeps every vector that is orthogonal to this two invariant. The results of the three
experiments can be seen in Fig. 5. In all three cases, for about the first 20 iterations, the swarm
behaves like on the sphere function and reduces its potential. Then, it discoveres the valley and
tries to move through it. However, in the unrotated case with 10 particles (Fig. 5a), the swarm
fails to accelerate and instead, it converges towards a non-optimal point. With much more effort,
the swarm consisting of 50 particles (Fig. 5b) is able to accelerate, but the acceleration rate and
therefore the speed are comparatively poor. Finally, Fig. 5c shows how the swarm handles the
rotated version much better than the original function f before. Here, after only 100 iterations,
the potential increased to a value of about 1045 . The reason for this large difference between the
behavior on f and on frot is the capability of the swarm to favor one direction only if this direction
is parallel to one of the axes.
In particular, this experiment shows that PSO is not invariant under rotations of the search
space.

(a) f with b = 1.1, 10 particles

(b) f with b = 1.1, 50 particles

(c) frot with b = 1.1. 10 particles

Figure 5: Behavior of the particles on functions f and frot

5

Modified PSO

In the previous section, we have seen that the particle swarm might get stuck if its potential is too
high in dimensions that are already optimized and too low in dimensions that could still be improved.
Then the global attractor stagnates and the swarm starts to converge. Since the convergence

10

happens in a symmetric manner along the different dimensions, the imbalance is maintained. A
small and simple modification of the PSO algorithm avoids that problem by enabling the swarm to
rebalance the potentials in the different dimensions:
Definition 3 (Modified PSO). For some arbitrary small but fixed δ > 0, we define the modified
PSO via the same equations as the classic PSO in Def. 1, only modifying the velocity update in line
4 of Algorithm 1 to

′
n
n

(2 · rand() − 1) · δ, if ∀ d ∈ {1, ..., D} : Vd′ + Gd′ − Xd′ < δ,
Vdn :=


χ · Vdn + c1 · rand() · (Lnd − Xdn ) + c2 · rand() · (Gd − Xdn ),
otherwise.

In words: As soon as the sum of the velocity and the distance between the position and the
global attractor of a particle are below the bound of δ in every single dimension, the updated velocity
of this particular particle is drawn u. a. r. from the interval [−δ, δ]. Note the similarity between
this condition and the definition of the potential. Indeed, we could have used the condition Φd < δ
(with some fixed a) instead, but we decided to keep the modification as simple and independent
from the terms occurring in the analysis as possible. Now the potential can no longer converge to
0 while staying unbalanced because if it decreases below a certain bound, we randomly assign a
value to the velocity which on expectation has an absolute value of δ/2. If a velocity is assigned
that way, we call the step forced.
This modified PSO is similar to the Noisy PSO proposed by Lehre and Witt in [LW11] where
they generally add a random perturbation drawn u. a. r. from [−δ/2, δ/2] for some small δ and
prove that their swarm is able to find a local optimum. However, their analysis is restricted to one
specific 1-dimensional fitness function.
The modification does not prevent the PSO from emerging imbalance between the potentials
of different dimensions. But the imbalanced convergence phenomenon described above is no longer
possible. When the global attractor of the modified PSO gets stuck and the potential decreases,
there will be a point when both the velocities and the distances to the global attractor in every dimension get lower than δ. From that moment on, the relationship between the different dimensions
gets balanced by the forced steps which on expectation give every dimension the same amount of
potential. So, the potentials of the different dimensions are made equal.
We repeated the experiment from the previous section in the same setting as before, but using
the modified PSO. The results can be seen in Table 3. It turns out that the modified PSO algorithm
actually leads to a better solution than the unmodified one.
We also calculated the standard deviation for each unit and obtained values that were of the
same order but usually higher than the mean values. The reason for this high deviations is that
the examined phenomenon occurs randomly and therefore one cannot predict the potential level of
the swarm when it occurs.
To make sure that the modification does not fully take over, we plotted the forced points with
δ = 10−7 and the 2-dimensional sphere function as objective function in Fig. 6. As can be seen
in the figure, the particles get forced near (−2 · 10−5 , 0) but their movement does not stay forced.
Instead, the swarm becomes running again until the particles approached the optimum at (0, 0).
This implies that for sufficiently smooth functions, the modification does not take over, replacing
the PSO by some random search routine. Instead, the modification just helps to overcome “corners”.
As soon as there is a direction parallel to an axis with decreasing function value, the swarm becomes
“running” again and the unmodified movement equations apply.
11

Table 3: Comparison between the classic and the modified PSO algorithm
Function
Sphere
Sphere
Sphere
Sphere
Sphere
Sphere
Rosenbrock
Rosenbrock
Rosenbrock
Rosenbrock
Rosenbrock
Rosenbrock
1

6

D
4
4
60
60
150
150
4
4
60
60
150
150

N
2
2
10
10
20
20
2
2
10
10
20
20

tmax
10000
10000
100000
100000
100000
100000
10000
10000
100000
100000
100000
100000

δ
10−12
10−12
10−12
10−7
10−7
10−3
-

Value
43.34
51.04
4.07
12.18
6.41
11.97
8.80
126.54
2.02
34.57
2.25
28.88

Due to double precision.

Conclusion

This paper focuses on the behavior of a particle swarm to find good regions in the search space. We
found out that the potentials of the different dimensions are most likely to become unbalanced and
that this imbalance possibly causes the particle swarm to get stuck at non-optimal search points.
A suggestion to modify the algorithm by randomly assigning a small velocity if the potential of a
particle falls below a certain bound is suggested. Additionally, we show that the modification does
not take over the swarm, it just corrects the direction before the classic movement equations are
applied again.

References
[CD01]

A. Carlisle and G. Dozier. An off-the-shelf PSO. In Proc. Particle Swarm Optimization
Workshop, pages 1–6, 2001.

[CK02]

Maurice Clerc and James Kennedy. The particle swarm – explo- sion, stability, and
convergence in a multidimensional complex space. IEEE Transactions on Evolutionary
Computation, 6:58–73, 2002. doi:10.1109/4235.985692.

[Cle06]

Maurice Clerc. Stagnation analysis in particle swarm optimization or what happens
when nothing happens. http://hal.archives-ouvertes.fr/hal-00122031, 2006. Technical
report.

[EK95]

Russell C. Eberhart and James Kennedy. A new optimizer using particle swarm theory.
In Proc. 6th International Symposium on Micro Machine and Human Science, pages
39–43, 1995. doi:10.1109/MHS.1995.494215.

[GWHK09] A. Gnezdilov, S. Wittmann, S. Helwig, and G. Kókai. Acceleration of a relative positioning framework. International Journal of Computational Intelligence Research,
5:130–140, 2009.
[JLY07]

M. Jiang, Y. P. Luo, and S. Y. Yang. Particle swarm optimization – stochastic trajectory analysis and parameter selection. In Felix T. S. Chan and Manoj Kumar Tiwari,
12

× 10

-5

3
2

y

1
0
-1
-2
-3
-3

-2

-1

0
x

1

2

3
× 10

-5

Figure 6: Behavior of the modified PSO on the sphere function
editors, Swarm Intelligence – Focus on Ant and Particle Swarm Optimization, chapter 17, pages 179–198. I-TECH Education and Publishing, Vienna, 2007.
[KE95]

James Kennedy and Russell C. Eberhart. Particle swarm optimization. In Proc.
IEEE International Conference on Neural Networks, volume 4, pages 1942–1948, 1995.
doi:10.1109/ICNN.1995.488968.

[LW11]

Per Kristian Lehre and Carsten Witt. Finite first hitting time versus stochastic convergence in particle swarm optimisation. arXiv:1105.5540, 2011.

[OD10]

J. E. Onwunalu and L. J. Durlofsky. Application of a particle swarm optimization
algorithm for determining optimum well location and type. Computational Geosciences,
14:183–198, 2010. doi:10.1007/s10596-009-9142-1.

[PSL11]

Bijaya Ketan Panigrahi, Yuhui Shi, and Meng-Hiot Lim, editors. Handbook of
Swarm Intelligence — Concepts, Principles and Applications.
Springer, 2011.
doi:0.1007/978-3-642-17390-5.

[Ros60]

H. H. Rosenbrock. An automatic method for finding the greatest or least value of a
function. The Computer Journal, 3:175–184, 1960. doi:0.1093/comjnl/3.3.175.

[RPPN09]

K. Ramanathan, V. M. Periasamy, M. Pushpavanam, and U. Natarajan. Particle
swarm optimisation of hardness in nickel diamond electro composites. Archives of
Computational Materials Science and Surface Engineering, 1:232–236, 2009.

[SW13a]

Manuel Schmitt and Rolf Wanka. Particle swarm optimization almost surely finds local
optima. In Proc. Genetic and Evolutionary Computation Conference (GECCO), 2013.
doi:10.1145/2463372.2463563.
13

[SW13b]

Manuel Schmitt and Rolf Wanka. Particles prefer walking along the axes: Experimental
insights into the behavior of a particle swarm. In Proc. Genetic and Evolutionary
Computation Conference (GECCO), 2013. doi:10.1145/2464576.2464583.

[Tre03]

Ioan Cristian Trelea. The particle swarm optimization algorithm: Convergence
analysis and parameter selection. Information Processing Letters, 85:317–325, 2003.
doi:10.1016/S0020-0190(02)00447-7.

[vdBE02]

F. van den Bergh and A. P. Engelbrecht. A new locally convergent particle swarm optimiser. In Proc. IEEE Int. Conf. on Systems, Man and Cybernetics (SMC), volume 3,
pages 94–99, 2002. doi:10.1109/ICSMC.2002.1176018.

[WSZ+ 04]

Mark P. Wachowiak, Renata Smolı́ková, Yufeng Zheng, Jacek M. Zurada, and Adel S.
Elmaghraby. An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Transactions on Evolutionary Computation, 8:289–301,
2004. doi:10.1109/TEVC.2004.826068.

14

