The outputs from the Graph Attention Network $\varphi_{mhat}^{\prime}$ are passed through a dense layer activated by a softmax function denoted as $\mathcal{F}_{1}(.)$. In addition we add an auxiliary classifier $\mathcal{F}_{2}(.)$ to improve convergence of the network and training efficiency by directly guiding the learning of the multi-modal feature embedding $\varphi_{mhat}$. The auxiliary classifier is also composed of a dense layer activated by a softmax function. In our proposed architecture, all components are trained end-to-end using the following joint loss function:
Rewrite in a formal way: