\begin{table}[H]
\centering
\caption{Details of the D4PG algorithm.}
\begin{tabular}{l|r}
\hline
D4PG Parameters        & Values                                                                               \\ \hline
$V_{min}$              & 0                                                            \\
$V_{max}$              & 100                                                             \\
$V_{bins}$             & 51                                                                                   \\
N step (return)        & 5                                                                                    \\
Actor learning rate    & 0.0001                                                                               \\
Critic learning rate   & 0.0001                                                                               \\
Optimizer              & Adam~\citep{kingma2014adam}                                                                                 \\
Batch size             & 256                                                                                  \\
Discount factor        & 0.99                                                                                 \\
Number of actors       & 16 (\emph{walker}), 128 (\emph{lifting, stacking}) \\ \hline
Network Specifications & Values                                                                               \\ \hline
Actor network          & MLP={[}300, 200{]}                                                                   \\
Critic network         & MLP={[}400, 300{]}                                                                   \\ 
Activation function    & ReLU                                                                 \\ \hline
\end{tabular}
\label{table:policy_spec}
\end{table}
D4PG Parameters Values
V
min
V
max
V
bins
N step (return)
Actor learning rate
Critic learning rate
Optimizer
Batch size
Discount factor
Number of actors 0
100
51
5
0.0001
0.0001
Adam (Kingma & Ba, 2014)
256
0.99
16 (walker), 128 (lifting, stacking)
Network Specifications Values
Actor network
Critic network
Activation function MLP=[300, 200]
MLP=[400, 300]
ReLU