\begin{table}[t]
\centering
\begin{tabular}{l|p{8cm}}
\hline
Active learners & The $k$-th learner \\
\hline
Active learnable parameters & Weights of the data encoder and weights of the task-specific head in  the $k$-th learner\\
\hline
Supporting learnable parameters & Encoder architecture shared by all learners\\
\hline
Active training datasets & Training dataset of the $k$-th learner \\
\hline
Active auxiliary datasets & --  \\
\hline
Training loss &  The $k$-th learner trains the weights of its data encoder and the weights of its task-specific head on its training dataset: $L(A, W^{(m)}_k, H^{(m)}_k,D_k^{(\textrm{tr})})$ \\
\hline
Interaction function & The learner encourages its  encoder weights to be close to the optimal encoder weights  $\widetilde{W}_{l-1}$ learned in the $l-1$ stage: $\|W^{(m)}_k-\widetilde{W}_{l-1}\|_2^2$ \\
\hline
Optimization problem & $\widetilde{W}_k^{(m)}=\textrm{min}_{W_k^{(m)},H_k^{(m)}} \; L(A, W_k^{(m)}, H_k^{(m)},D_k^{(\textrm{tr})})+\lambda\|W^{(m)}_k-\widetilde{W}_{l-1}(A)\|^2_{2}$ \\
\hline
\end{tabular}
\caption{Learning stage $l$ with the $k$-th learner at the $m$-th round, in interleaving learning}
\label{tb:il-k}
\end{table}
Active learners The k-th learner
Active learnable parameters Weights of the data encoder and weights of the
task-specific head in the k-th learner
Supporting learnable parameters Encoder architecture shared by all learners
Active training datasets Training dataset of the k-th learner
Active auxiliary datasets –
Training loss The k-th learner trains the weights of its data
encoder and the weights of its task-specific head
(m) (m) (tr)
on its training dataset: L(A, W , H , D )
k k k
Interaction function The learner encourages its encoder weights to
be close to the optimal encoder weights W
fl−1
learned in the l −1 stage: ∥W k(m) −W fl−1∥2
2
Optimization problem (m) (m) (m) (tr)
W = min L(A, W , H , D )+
f k W k(m),H k(m) k k k
λ ∥W k(m) −W fl−1(A) ∥2
2