\begin{table}[]
\centering
\begin{tabular}{r|p{0.35\linewidth}|p{0.35\linewidth}}
& LunarLander-v2&Pendulum-v0\\
\hline
For all cases below&\makecell[l]{$t_m$=600\\fw-steps=50\\batch-size=512\\lr=5e-4\\$\gamma=0.99$\\n-evals=20\\eps-start=1.0\\ eps-end=0.01\\eps-decay=0.996\\ eval-interval=20\\n-runs=10}&\makecell[l]{memory-size=20000 (small)\\ 
or memory-size=200000 (large)\\$t_m$=600\\fw-steps=1\\batch-size=64\\ lr=5e-4\\$\gamma=0.99$\\n-evals=20\\ eval-interval=5\\n-runs=10}\\
\hline
MLP & hidden-size=573 (\# of trainable parameters is 8599)&hidden-size=860 (\# of trainable parameters is 16344)\\
\hline
MLP+Bioclock &  hidden-size=492 (\# of trainable parameters is 8594)&hidden-size=710 (\# of trainable parameters is 16416)\\
\hline
Bundle &  hidden-size=97 (\# of trainable parameters is 8587)&hidden-size=222 (\# of trainable parameters is 16318)\\
\hline
Bundle + Bioclock &  hidden-size=64 (\# of trainable parameters is 8578)&hidden-size=128 (\# of trainable parameters is 16290)\\
\hline
\end{tabular}
\caption{Settings of the RL experiments. $t_{m}$ is the maximum time step in an episode, fw-steps is the number of forward and backpropagation steps when training a batch, batch-size is the size of a batch, lr is the learning rate, $\gamma$ is the discount rate, n-evals is the number of repeated evaluations in one evaluation step, eval-interval is the number of episodes between two evaluation steps, n-runs is the number of repeated runs, the exploration probability $\epsilon$ (see Algorithm \ref{algo1}) for episode $i$ is given by $\epsilon(i)$=$\max($eps-end, eps-start * eps-decay$^{i})$, and memory-size is the size of the memory buffer for replay. In addition, we used an Ornstein-Uhlenbeck process with $\theta=0.15$ and $\sigma=0.2$\cite{lillicrap2015continuous} to generate the noise $\mathcal{N}_{t}$ in Algorithm \ref{algo2}; and $T_{min}$ and $T_{max}$ in Eq. \ref{eqn:bioclock} are 1 and $10^4$, respectively.}\label{tb1}
\end{table}
LunarLander-v2 Pendulum-v0
For all cases below t =600
m
fw-steps=50
batch-size=512
lr=5e-4
γ = 0.99
n-evals=20
eps-start=1.0
eps-end=0.01
eps-decay=0.996
eval-interval=20
n-runs=10 memory-size=20000 (small)
or memory-size=200000 (large)
t =600
m
fw-steps=1
batch-size=64
lr=5e-4
γ = 0.99
n-evals=20
eval-interval=5
n-runs=10
MLP hidden-size=573 (# of trainable
parameters is 8599) hidden-size=860 (# of trainable
parameters is 16344)
MLP+Bioclock hidden-size=492 (# of trainable
parameters is 8594) hidden-size=710 (# of trainable
parameters is 16416)
Bundle hidden-size=97 (# of trainable
parameters is 8587) hidden-size=222 (# of trainable
parameters is 16318)
Bundle + Bioclock hidden-size=64 (# of trainable
parameters is 8578) hidden-size=128 (# of trainable
parameters is 16290)