Optimizing PID parameters with machine learning
Adam Nyberg

arXiv:1709.09227v1 [] 26 Sep 2017

Abstract
This paper examines the Evolutionary programming (EP) method for optimizing
PID parameters. PID is the most common type of regulator within control theory,
partly because it’s relatively simple and yields stable results for most applications. The
p, i and d parameters vary for each application; therefore, choosing the right parameters
is crucial for obtaining good results but also somewhat difficult. EP is a derivative-free
optimization algorithm which makes it suitable for PID optimization. The experiments
in this paper demonstrate the power of EP to solve the problem of optimizing PID
parameters without getting stuck in local minimums.

1

Introduction

This paper will examine one approach to the optimization of Proportional-Integral-Derivative
(PID) parameters. PID is the most common type of regulator within control theory. The
mathematical expression of a PID regulator is relatively simple and yields stable results for
most applications. The p, i and d parameters vary for each application; therefore, choosing
the right parameters is crucial for obtaining good results but also somewhat difficult. According to K.J.Åström and T.Hägglund [1], there are several methods used in the industry
for tuning the p, i and d parameters. Some methods involve developing models while others
involve manual tuning by trial and error.
This paper examines the Evolutionary programming (EP) method for optimizing PID
parameters. EP is an iterative algorithm that runs until some performance criteria are met.
The basic steps are as follows: EP generates a population, evaluates every member in the
population, selects the best member, creates a new population based on the best member,
and then repeats the aforementioned process with an evaluation of the new population.
The new population is generated by adding a Gaussian mutation to the parent. The results
of this paper are compared with the results generated in R.Johns [4], which uses Genetic
algorithm (GA), and S.Hadenius [3], which uses Particle Swarm optimization (PSO). The
GA and PSO methods are further explained in [5].
All training and testing of the algorithm was done in a simulated environment called
MORSE on a Robot Operating System (ROS). The algorithm and movements of the robot
were developed as three ROS nodes. An ROS node is one subsystem; typically, a robotics
environment is built with multiple ROS nodes. The actual simulation of the robot was done
by MORSE which is a generic 3D simulator for robots. Nodes are only able to communicate
with other nodes using streaming topics, Remote Procedure Calls (RPC) services, and the
Parameter Server. One route was used for training and a separate route was used for
evaluating the step response. The experiments in this paper tuned two PIDs simultaneously.

1

The PIDs tuned were the ones controlling linear velocity and angular velocity on the robot.
Only the Husky robot was used.
EP can perform effectively in this type of domain due to its ability to optimize without
getting stuck in local minimums and its efficiency over brute force. The resulting parameters
were evaluated by looking at the step response. Important measurements are: rise time,
overshoot, and the steady state error.

2
2.1

Method
Set up environment

This paper describes three different experiments that all ran on the same environment.
Everything required to run the experiments is listed in table 1.
Name

Description

Roscore

Roscore is a collection of nodes and programs that serve as prerequisites of a ROS-based system.
The state machine is the ROS node that controls general movement and includes the PID for both linear velocity and angular
velocity. This node is built in Modelled Architecture (March).
This node is only responsible for making the robot drive a predefined route. It is triggered by the Determinator, and it sends a
callback back to the Determinator when the route is completed.
The route used for training can be seen in algorithm 1 with parameters start = −0.3 and end = 0.3. The route used for testing
uses the same algorithm but with parameters start = 0.1 and
end = 0.7. The parameters start and end determine the velocity
of the robot. The test route was run with the best PID parameters
from the training.
This node runs the algorithm and logs data to file. This is also
the node that starts the route node. A detailed description of the
EP algorithm is in section 2.3.

State machine

Route

Determinator

Table 1: Table of systems required for running the experiments.
Algorithm 1 The route used by the robot for train and test.
1: procedure Route(start, end)
2:
linearV elocity ← start
3:
angularV elocity ← start
4:
wait 3 seconds
5:
linearV elocity ← end
6:
angularV elocity ← end
7:
wait 3 seconds
8:
Notify finnished to Determinator

2

2.2

Definitions

The Determinator receives both the actual velocity and desired velocity of the robot 50 times
per second. With that data, an error can be estimated by calculating abs(desired − actual).
That calculation is executed for every sample, summed, and then divided by the total
number of samples. Thus, the total error for one run of a route is the average error of that
route. This average error (AE) is used by the algorithm to evolve the parameters. Fitness
refers to lowest AE in this paper.
A population, or generation, is a collection of individuals in which each individual contains values for kpv , kiv , kdv , kpa , kia , kda . The values denoted kpv , kiv , kdv controls the linear
velocity PID and values denoted kpa , kia , kda controls the angular velocity PID. The velocity
and angular parameters were evaluated separately from one another. Figure 1 provides a
visual illustration of the generations within the EP 2.3 algorithm. Each box in figure 1
illustrates an individual and each row illustrates a population. The green boxes indicate
the fittest individual of each population (i.e., lowest AE).

2.3

Evolutionary programming algorithm

The version of evolutionary programming used in this paper follows the algorithm described
in [5].
1. Generate an initial population of individuals. The number of individuals differs between the experiments.
2. Evaluate fitness as described in 2.2.
3. Select the fittest individual of the population by selecting the individual with lowest
AE.
4. If the fittest individual has an average error better 0.01, return that individual and
exit. 0.01 was chosen as a way to try to get the AE lower than 1 percent.
5. Else generate a new population by applying a mutation to the fittest individual. The
mutation differs between the experiments but the general principle is to add a random
number from a Gaussian distribution to each of the new offspring. The mutation used
for the experiments is described in detail in section 2.4.
6. Go to step 2.
Algorithm 2 Function used in experiment 1.
procedure Mutate(value)
add ∼ N (0, 0.05)
while value + add < 0 do
add ← add ∗ 0.5
return value + add

3

Figure 1: Figure illustrating 3 generations with 6 individuals in each generation.
Algorithm 3 Function used in experiment 2 and 3.
1: procedure Mutate(value)
2:
add ∼ value ∗ N (0, 0.5)
3:
while value + add < 0 do
4:
add ← add ∗ 0.5
return value + add

2.4

Experiments

Table 2 describes all of the experimental configurations. All experiments ran for 100 generations and no experiment reached the set criteria of AE < 0.01. All experiments began with
the same initial population, which looked like the following: kpv ∼ U(0, 1) , kiv ∼ U(0, 0.1) ,
kdv ∼ U(0, 0.01) , kpa ∼ U(0, 1) , kia ∼ U(0, 0.1) , kda ∼ U(0, 0.01) .
The mutate algorithm provided in [2] and used in experiment 1 did not perform well for
this application because the parameters differed in size by more than an order of magnitude.
Therefore, experiments 2 and 3 used a mutate algorithm that scales the mutation with the
value being mutated.
Experiment
1
2
3

Mutation algorithm
See algorithm 2
See algorithm 3
See algorithm 3

Population size
10
10
20

Table 2: Configurations for the experiments.

4

3

Result

For each experiment, this paper will present the AE value as well as a graph illustrating
the step response of the PID for both train and test. Table 3 lists every experiment’s best
parameters and lowest AE for train and test.
Experiment
1
1
2
2
3
3

Type
Linear
Angular
Linear
Angular
Linear
Angular

kp
7.78 ∗ 10−2
7.83 ∗ 10−2
8.16 ∗ 10−2
1.17 ∗ 10−1
1.23 ∗ 10−1
1.21 ∗ 10−1

ki
1.63 ∗ 10−1
4.24 ∗ 10−2
2.12 ∗ 10−6
6.34 ∗ 10−6
2.12 ∗ 10−5
3.25 ∗ 10−4

kd
0
0
0
2.60 ∗ 10−9
0
3.35 ∗ 10−8

AE train
0.0563
0.0615
0.0312
0.0253
0.0313
0.0234

AE Test
0.0707
0.0529
0.0406
0.0414
0.0514
0.0421

Table 3: Results for the experiments.
In addition to the results presented in Table 3, Figure 2 illustrates how the PID parameters evolve over every generation in experiment 2. For experiment 2, Figure 3 shows the
AE across all generations and Figure 4 shows the step response for train and test for both
linear and angular velocity.

Figure 2: Figure showing PID parameters evolving over generations.

5

Figure 3: The AE over generations during experiment 2.

Figure 4: Step response for train and test for both linear and angular velocity.

4

Discussion

Overall, we can conclude that using EP is successful in finding sufficient parameters given
enough training time as described in [2]. The results presented in this paper are somewhat
surprising. As we can see, the d part of PID becomes either 0 or negligible after running
the algorithm for a few generations. The same thing happens to the i part, but to an
lesser extent. An explanation for this behavior could be the way that the simulator is built.
This begs the question of whether the results are useful in an environment outside of our
6

simulation. Finally, we noticed that increasing the population size from 10 to 20 did not
significantly improve the results.
When comparing the results with the results from R.Johns [4] and S.Hadenius [3], we can
see that they got similar results, including very small ID-parameters. This adds confidence
that the algorithm works but that the simulation is not optimal for tuning the PIDs.
There are several variations to the experiment that could be done to improve the results.
One way is to divide the route into different parts and then calculate the AE for each part.
That would minimize the risk of a good integral value being punished by the algorithm for a
slow rise time. Second, other fitness functions could be used instead of AE, including mean
squared error. Third, training the parameters on a dynamic route could take the algorithm
out of a local minima. Lastly, running the robot with the best known parameters in between
every run would allow each individual to start with as low of an error as possible. In this
paper, bad parameters from a previous individual could affect the AE of the next tested
individual.

References
[1]

K.J. Åström and T. Hägglund. “The future of PID control”. In: Control Engineering
Practice Volume 9, Issue 11, November 2001, Pages 1163–1175 (2001), pp. 1163–1175.

[2]

David B. Fogel and Lawrence J. Fogel. “An introduction to evolutionary programming”. In: Alliot JM., Lutton E., Ronald E., Schoenauer M., Snyers D. (eds) Artificial
Evolution. AE 1995. Lecture Notes in Computer Science, vol 1063. Springer, Berlin,
Heidelberg (1995), p. 30.

[3]

S. Hadenius. “Optimizing PID Parameters with PSO”. In: (2017).

[4]

R. Johns. “Tuning a PID-controller with machine learning”. In: (2017).

[5]

B. Nagaraj, S. Subha, and B.Rampriya. “Tuning Algorithms for PID Controller Using
Soft Computing Techniques”. In: IJCSNS International Journal of Computer Science
and Network Security, VOL.8 No.4, April 2008 (2008).

7

