\begin{table}[t]
\centering
\begin{tabular}{l|p{10.5cm}}
\hline
Skillearn & Interleaving Learning\\
\hline 
Learners & $K$ learners \\
\hline
Learnable parameters &  1) Encoder architecture shared by all learners; 2) In each round, each learner has weight parameters for the data encoder and weight parameters for the task-specific head.  \\
\hline
Interaction function &  The encoder weights $W_l$ at learning stage $l$ are encouraged to be close to the optimal  encoder weights $\widetilde{W}_{l-1}$ at stage $l-1$: $\|W_l-\widetilde{W}_{l-1}\|_2^2$.\\
\hline
Learning stages &  1) In the first learning stage (the first learner in the first round), the learner trains the weights of its data  encoder  and the weights of its task-specific head on its training dataset: $\widetilde{W}_1^{(1)}(A) =\textrm{min}_{W^{(1)}_1,H^{(1)}_1} \quad L(A, W^{(1)}_1, H^{(1)}_1,D_1^{(\textrm{tr})})$; 2) In other learning stages, the learner trains the weights of its data encoder and the weights of its task-specific head on its training dataset where the  encoder weights are encouraged to be close to the optimal  encoder weights trained in the previous stage: $\widetilde{W}_k^{(m)}(A) =\textrm{min}_{W_k^{(m)},H^{(m)}_k} \quad L(A, W^{(m)}_k, H^{(m)}_k,D_k^{(\textrm{tr})})+\lambda\|W^{(m)}_k-\widetilde{W}_{k-1}^{(m)}(A)\|^2_{2}$. 
\\
\hline 
Validation stage & Each learner validates its optimal data encoder and task-specific head learned in the last round on its validation dataset.\\
\hline 
Datasets & Each learner has a training dataset and a validation dataset.\\
\hline
\end{tabular}
\caption{Mapping from Skillearn to Interleaving Learning}
\label{tb:sk-il}
\end{table}
Skillearn Interleaving Learning
Learners K learners
Learnable parameters 1) Encoder architecture shared by all learners; 2) In each round,
each learner has weight parameters for the data encoder and
weight parameters for the task-specific head.
Interaction function The encoder weights W at learning stage l are encouraged to
l
be close to the optimal encoder weights W at stage l 1:
fl−1 −
W W 2.
∥ l −fl−1∥2
Learning stages 1) In the first learning stage (the first learner in the first
round), the learner trains the weights of its data encoder and
the weights of its task-specific head on its training dataset:
(1) (1) (1) (tr)
W (A) = min L(A, W , H , D ); 2) In other
f 1 W 1(1),H 1(1) 1 1 1
learning stages, the learner trains the weights of its data en-
coder and the weights of its task-specific head on its train-
ing dataset where the encoder weights are encouraged to be
close to the optimal encoder weights trained in the previous
(m) (m) (m) (tr)
stage: W (A) = min L(A, W , H , D ) +
f k W k(m),H k(m) k k k
λ ∥W k(m) −W k( −m 1) (A) ∥2 2.
f
Validation stage Each learner validates its optimal data encoder and task-
specific head learned in the last round on its validation dataset.
Datasets Each learner has a training dataset and a validation dataset.