\section{Introduction}

The dynamics of non-abelian gauge fields at finite temperature has
attracted some attention recently~\cite{grigoriev}--\cite{son}.
A lot of work on this subject has been
done in perturbation theory. However, some physical quantities
of interest are non-perturbative and the only known method of
calculating them on the lattice is the classical
approximation~\cite{grigoriev}.

One problem of the classical approximation is related to the
fact that the high momentum modes do not decouple from the dynamics of
the low momentum modes. In the naive perturbative expansion, momenta
of order $T$ lead to corrections in the Green's functions which are
proportional to $T^2$. For Green's functions with soft external
momenta $|\vec{p}|\sim g^2 T$, where $g$ is the gauge coupling, these
``corrections'' can be as large as or even larger than the tree level
Green's functions.  These large corrections have been named hard
thermal loops (HTL) and a consistent perturbative expansion requires
that they be resummed~\cite{pisarski}.

In the resummed perturbation theory the gauge field propagators
reflect two distinct collective phenomena. One is the plasma
oscillations which involve simultaneous oscillations of the low and
the high momentum degrees of freedom in the plasma.  The
characteristic frequency of these oscillations is proportional to
$gT$.  Secondly, there is an effect called
Landau damping which is related to an energy transfer from the low
momentum degrees of freedom to the high momentum ones.

In the classical approximation the physics of the high momentum
degrees of freedom is not correctly described. The hard thermal loops
correspond to classically UV-divergent contributions \cite{bodeker}.
How the hard
thermal loops affect non-perturbative correlation functions of the low
momentum fields is an open problem.

The physical picture sketched above is based on 1-loop resummed
perturbation theory. It is therefore interesting to see whether it also
shows up in non-perturbative lattice studies. In a recent paper~\cite{tang}
the correlation functions of two gauge invariant operators
in the SU(2)+Higgs model (in continuum notation),
\begin{eqnarray}
  H (t) &=& \frac{1}{\sqrt{V}}\int \!\! d^3 x \,\varphi^\dagger(t,\vec{x})
  \varphi(t,\vec{x}), \la{operH} \\
  W_i^a(t) &=& \frac{1}{\sqrt{V}}\int\!\! d^3 x \,
  i{\rm Tr\,}\[\Phi^\dagger(t,\vec{x})
  D_i\Phi(t,\vec{x})\tau^a\], \la{operW}
\end{eqnarray}
were computed with classical lattice simulations.  Here $\varphi$
denotes the Higgs doublet, $\tau^a$ is a Pauli matrix, $\Phi$ is the
matrix $\Phi = (\widetilde\varphi\, \varphi)$ with
$\widetilde{\varphi} = i \tau^2 \varphi^*$, and $V$ denotes the space
volume.  In the broken phase the correlator of $W_i^a(t)$
is given by the gauge field propagator. It was claimed that the
plasmon frequency corresponding to this correlator is
independent of the lattice
spacing $a$.
This contradicts the picture above since in the
classical theory the plasmon frequency is proportional to $T/a$
instead of $T^2$. We will demonstrate here that the qualitative
features of the correlators can be understood using perturbation
theory and that the claim of ref.~\cite{tang} concerning the
broken phase $W_i^a$
plasmon frequency is not justified.

Ambj\o rn and Krasnitz \cite{ambjorn} have reported a measurement of
the gauge field correlator in the Coulomb gauge in the symmetric
phase.  We will see that many of the features observed can be
qualitatively understood in perturbation theory, but quantitative
discrepancies remain.

\subsubsection*{Real time correlators and the HTL effective action}

The plasmon properties of the
classical Hamiltonian SU(2)+Higgs gauge theory could
be computed by solving the classical equations of motion
perturbatively with given initial conditions,
and by then averaging over the initial conditions with
the Boltzmann weight~\cite{grigoriev}. However,
it appears that in perturbation theory it is simpler
to perform the computation in the full quantum theory and
to take the classical limit $\hbar\to 0$ only in the end.
We use this approach.

In the classical limit the operator ordering in a correlation
function is irrelevant. Thus the quantum expression of which we are
going to take the
classical limit  can be anything but a commutator.
This still leaves several possibilities, and we choose to consider
\begin{eqnarray}
  C_{\cal O}(t,\vec{p}) = \fr12 \int\! d^3 x e^{-i\vec{p}\vec{x}}
  \left\langle {\cal O}(t,\vec{x}) {\cal O}(0,\vec{0})
  + {\cal O}(0,\vec{0}) {\cal O}(t,\vec{x}) \right\rangle. \la{sym}
\end{eqnarray}
In a perturbative
computation, one first computes the corresponding two-point Green's
function ${\cal D}_{\cal O}(i\omega_n,{\bf p})$ in Euclidean space for
the Matsubara frequencies $\omega_n$.  Then one performs an analytic
continuation which depends on the real time correlator in question.
The symmetric combination in eq.~\nr{sym} is given by
\begin{eqnarray}
        C_{\cal O}(t,{\bf p}) =
        \hbar\int_{-\infty}^{\infty}\frac{d\omega}{2\pi}
        e^{-i\omega t}\left[1+2 n(\omega)\right]
        \mathop{\rm Im} {\cal D}_{\cal O}\left(
        i\omega_n\to \omega+i\epsilon,{\bf p}
        \right), \la{relation}
\end{eqnarray}
where
\begin{eqnarray}
  n(\omega)=\frac{1}{e^{\beta\hbar\omega}-1} \quad
\end{eqnarray}
is the Bose distribution function.

To evaluate the $\omega$-integral
in eq.\ (\ref{relation}) numerically, it is convenient to
rewrite it (for $t>0$) as an integral in the upper half of the complex
$\omega$-plane. In this way one avoids integrating along
the poles and discontinuities on the real $\omega$-axis. Denoting $\omega =
\omega_1 + i \omega_2$, one gets for $t>0$
in the classical limit
$n(\omega)\to T/(\hbar\omega)$,
\begin{eqnarray}
        C_{\cal O}^{\rm classical}(t,{\bf p}) =
        T \lim_{\hbar\to 0} {\cal D}_{\cal O}(0,{\bf p})+
        e^{\omega_2 t}\int_{-\infty}^{\infty}
        \frac{d \omega_1}{2\pi i}\frac{T}{\omega_1+i\omega_2}
        e^{-i\omega_1 t} \lim_{\hbar\to 0}
        {\cal D}_{\cal O}(\omega_1+i\omega_2,{\bf p}). \la{Ctp}
\end{eqnarray}
Note that this expression is independent of the imaginary part
$\omega_2>0$. The second term on the r.h.s.\ of eq.~\nr{Ctp} vanishes
for the equal time case $t=0$. A further virtue of eq.~\nr{Ctp} is
that one can take the limit ${\bf p}\to 0$ inside the integrand since
$\omega_2>0$ guarantees that there are no singularities on the
integration contour.  In the remainder of this letter we will discuss
only classical correlation functions\footnote{Except in eq.~\nr{AA}
and in the discussion thereafter.} and we will therefore omit the
superscript 'classical' from the l.h.s.\ of eq.~\nr{Ctp}.

We thus have to evaluate ${\cal D}_{\cal O}(\omega,{\bf p})$. As
discussed above, a consistent perturbative expansion requires the
resummation of the hard thermal loops~\cite{pisarski}.
In other words, the high momentum modes $p\mathop{\gsi} T$
(or $p\mathop{\gsi} a^{-1}$ in the classical theory) are integrated out,
giving an effective theory for the low momentum modes $p\mathop{\lsi} gT$.
This amounts to
including the hard thermal loops in the tree level action.
Denoting $P=(\omega_n,{\bf p})$,
the quadratic part of the Euclidean
HTL effective theory in momentum space is
\begin{eqnarray}
  {\cal L}_{\rm HTL}^{(2)} = \fr12 A_\mu^a A_\nu^a \left[ P^2
  \delta^{\mu\nu}-P^\mu P^\nu+\Pi_{\rm HTL}^{\mu\nu}(P) \right]
  +\varphi^\dagger \varphi \left[P^2 - \fr12 m_H^2 + \Sigma_{\rm
    HTL} \right], \la{htl}
\end{eqnarray}
where $m_H$ is the Higgs mass. Eq.~\nr{htl} gives
directly the gauge and scalar field free propagators
${\cal D}_{\mu\nu}^{ab}(\omega, \vec{p})$,
${\cal D}(\omega, \vec{p})$, to be used in
the computation of ${\cal D}_{\cal O}(\omega,{\bf p})$.
Note that $\Sigma_{\rm HTL}$ is momentum independent.

Of the HTL self-energies,
we will need especially the components $\Pi_{\rm HTL}^{ij}(P)$,
$\Sigma_{\rm
HTL}$. The expressions are~\cite{pisarski,bodeker,arnold97},
after the replacement $\omega_n\to -i\omega$,
\begin{eqnarray}
  \Pi_{\rm HTL}^{ij}(P) & = &
   -g^2 (2 N + N_{\rm s})\hbar   \int\!\! \frac{d^3q}{(2\pi)^3} \,
  \frac{dn(\omega_q)}{d\omega_q} \frac{v^iv^j\omega}{\omega- p^iv^i},
  \la{PiW} \\
   \Sigma_{\rm HTL} & = & \(6 \lambda + \frac94 g^2\) \hbar
  \int\!\! \frac{d^3q}{(2\pi)^3}\,\frac{n(\omega_q)}{\omega_q}, \la{PiH}
\end{eqnarray}
where, for the SU(2)+Higgs model, $N=2$, $N_{\rm s}=1$. Furthermore,
\begin{eqnarray}
 v^i = \frac{\partial \omega_q }{\partial q_i}
\end{eqnarray}
where $\omega_q$ is the tree level dispersion relation; for a space-time
with discretized spatial dimensions (a cubic lattice),
\begin{equation}
    \omega_q = \frac{2}{a} \sqrt{{\textstyle
    \sum_i}\sin^2\!  \left(a q_i/2 \right)}, \quad v^i =
    \frac{\sin (a q^i)}{a\omega_q}.
\end{equation}
Eqs.~\nr{PiW}, \nr{PiH} hold actually for a generic
dispersion relation~\cite{arnold97}.  In ref.~\cite{bodeker}, scalar
electrodynamics was considered\footnote{The result for $\Pi_{\rm
    HTL}^{ij}$ given in~\cite{bodeker} has the wrong sign.} for which
case $g^2(2N+N_{\rm s})$ has to be replaced with $ 2e^2$ in
eq.~\nr{PiW}.

Note that in the theory of eq.~\nr{htl}, the gauge field $A_0$ is
included. In the classical lattice
simulations~\cite{tang,ambjorn}, in contrast, one puts $A_0=0$ and
imposes the Gauss constraint explicitly.

The hard thermal loop self-energy simplifies
in the limit ${\bf p}=0$, which is relevant
for the operators in eqs.~\nr{operH}, \nr{operW}.
In that limit, the gauge field self-energy is
\begin{eqnarray}
  \Pi_{\rm HTL}^{00}=0,\quad \Pi_{\rm HTL}^{0i}=0,\quad \Pi_{\rm
    HTL}^{ij}=\delta^{ij} \omega_W^2,
\end{eqnarray}
where $\omega_W$ is the plasmon frequency,
\begin{eqnarray}
   \omega_W^2=-\fr13 g^2 (2N+N_{\rm s})\hbar
   \int\! \frac{d^3q}{(2\pi)^3}
   \frac{dn(\omega_q)}{d\omega_q}|{\bf v}|^2. \la{oW2}
\end{eqnarray}
Let us also introduce the ``scalar plasmon frequency''
\begin{eqnarray}
\omega_H^2 = -\fr12 m_H^2 + \Sigma_{\rm HTL}. \la{oH2}
\end{eqnarray}
In the continuum limit of the full quantum
theory, for which $|{\bf v}|^2=1$, one has
\begin{eqnarray}
   \omega_W^2 & = &  g^2(2 N+N_{\rm s})\frac{T^2}{18}
   \frac{1}{\hbar},\la{omegaWT} \\
   \omega_H^2 & = &  -\fr12 m_H^2+
   \left(6\lambda + \fr94g^2\right)\frac{T^2}{12}
   \frac{1}{\hbar}, \la{omegaHT}
\end{eqnarray}
whereas in the classical limit $n(\omega)\to T/(\hbar\omega)$
on a cubic lattice, one gets
\begin{eqnarray}
  \omega_W^2 & = & g^2 (2 N+N_{\rm s}) \left(\fr32\frac{\Sigma}{\pi}-1
  \right) \frac{T}{12a},\la{omegaWa} \\
  \omega_H^2 & = & -\fr12 m_H^2 +
  \left(6\lambda + \fr94g^2\right) \frac{\Sigma}{4\pi}\frac{T}{a}. \la{omegaHa}
\end{eqnarray}
Here
\begin{eqnarray}
\Sigma =  \frac{1}{\pi^2}\int_{-\pi/2}^{\pi/2}\!\! d^3x
\frac{1}{\sum_i{\sin}^2x_i}= 3.175911535625 \la{Sigma}
\end{eqnarray}
is a constant
which can be expressed in
terms of the complete elliptic integral
of the first kind~\cite{fkrs}.

The basic problem of the classical real time simulations can now be
expressed as follows. For the scalar field, one has a mass parameter
in the classical SU(2)+Higgs theory which does not break gauge
invariance. It can be tuned such that the high momentum modes produce
the correct quantum expression in eq.~\nr{omegaHT}, and that the
lattice spacing dependence in eq.~\nr{omegaHa} is canceled. For the
gauge fields, in contrast, the local classical SU(2)+Higgs theory does
not allow a mass term and hence the divergent classical expression in
eq.~\nr{omegaWa} cannot be arranged to coincide with the quantum
expression in eq.~\nr{omegaWT}.  This divergence should also show up
in the classical simulations. This problem is specific for time
dependent correlation functions: in the static case only the
$\omega_n=0$ sector requires a resummation and
$\Pi_{\rm HTL}^{ij}(0,\vec{p})=0$.

Finally, let us fix some notation.
The continuum field $\varphi$
in eqs.~\nr{operH}, \nr{operW} is related
to the dimensionless
lattice field $\bar{\varphi}$ in~\cite{tang} by
\begin{eqnarray}
\bar\varphi^\dagger\bar\varphi=
\frac{a}{T\beta_G}\varphi^\dagger\varphi, \quad \beta_G =
\frac{4}{ag^2T}.
\end{eqnarray}
The operators measured on the lattice are then
\begin{eqnarray}
\bar H & = & \frac{1}{\sqrt{N^3}}\sum_{\bf x}
\bar\varphi^\dagger({\bf x})\bar\varphi({\bf x}), \\
\bar{W}^a_i & = &
\frac{1}{\sqrt{N^3}}\sum_{\bf x}
i{\rm Tr\,} \left[\bar\Phi^\dagger(\vec{x}) U_i(\vec{x})
\bar\Phi(\vec{x}+\vec{e}_i a)\tau^a\right],
\end{eqnarray}
where $U_i(\vec{x})$
is the link operator.
The correlators measured with these operators
are denoted by $C_{\bar H}(t)$,
$\fr13\delta^{ab}\delta_{ij}C_{\bar W}(t)$, and
they are related in the continuum limit $a\to 0$
to the correlators
$C_H(t), C^{ab}_{W,ij}(t)$
of the continuum operators in eqs.~\nr{operH}, \nr{operW}
through
$C_H(t)=a \beta_G^2 T^2 C_{\bar H}(t)$,
$C^{ab}_{W,ij}(t)=\fr13
\delta^{ab}\delta_{ij} a^{-1} \beta_G^2 T^2 C_{\bar W}(t)$.
The factor $3$ in the definition of $C_{\bar W}(t)$ corresponds to
a sum over the different isospin components.

\subsubsection*{The broken phase}

Let us first consider the broken phase. The real time correlators of
the operators $\bar H(t)$, $\bar W^a_i(t)$ have been determined in the
classical approximation in the broken phase of the SU(2)+Higgs model
by Tang and Smit~\cite{tang}.

We parameterize the scalar field in the broken phase as
\begin{eqnarray}
        \Phi=\frac1{\sqrt{2}}\[v(T) + \phi_0+ i\tau^a\phi_a\].
\end{eqnarray}
Then, in continuum notation, the lowest order terms in the gauge invariant
operators of eqs.~\nr{operH}, \nr{operW} become
\begin{eqnarray}
   H(t) & = & {\rm const.} + \frac{1}{\sqrt{V}}
   \int \!\!d^3x\, v(T)\phi_0(t,{\bf x}) , \\
   W^a_i (t) & = & \frac{1}{\sqrt{V}}
   \int \!\!d^3x\, v(T) \Bigl[ m_W(T) A^a_i(t,{\bf x})
   - \partial_i \phi_a(t,{\bf x}) \Bigr], \la{lWa}
\end{eqnarray}
where $m_W(T)=gv(T)/2$.  Due to the integration $\int\! d^3x$, these
operators correspond to zero spatial momentum, ${\bf p}=0$, so that
the second term in eq.~\nr{lWa} does not contribute.  It follows that
the leading terms in the correlation functions $C_H(t)$
and $C^{ab}_{W,ij}(t)$
are given by the propagators of the elementary fields
$\phi_0$ and $A^a_i$.

To compute the required propagators, one can set ${\bf p}=0$ in the time
dependent part of eq.~\nr{Ctp}, so that eqs.~\nr{oW2}, \nr{oH2} can be
used.  At zero spatial momentum the full Euclidean (Matsubara)
propagators for the fields $A^a_i$, $\phi_0$ are of the form
\begin{eqnarray}

   {\cal D}_{ij}^{ab}(i\omega_n, \vec{0}) =
   \frac{\delta^{ab}\delta_{ij}}
   {\omega_n^2+(\omega_W^{\rm b})^{2}+  \mbox{$^\ast\Pi(i\omega_n)$}},\quad

   {\cal D}(i\omega_n, \vec{0}) =
   \frac{1}{\omega_n^2+(\omega_H^{\rm b})^{2}+
   \mbox{$^\ast\Sigma(i\omega_n)$}},
\end{eqnarray}
where
$^\ast\Pi$, $^\ast\Sigma$ denote the parts of the self-energies
which are generated radiatively within the HTL effective theory.
The tree-level terms are
\begin{eqnarray}
   (\omega_W^{\rm b})^2 & = &
   m_W^2(T)+0.05379 \beta_G (g^2T)^2, \la{wbW} \\
   (\omega_H^{\rm b})^2 & = & m^2_H(T), \la{wbH}
\end{eqnarray}
where $m_W(T)$, $m_H(T)$ are the mass parameters generated by the
Higgs mechanism in the static theory: $m_W(T)=gv(T)/2$,
$m_H^2(T)=\omega_H^2+3 \lambda v^2(T)$.  Here the scalar mass
parameter was tuned to its correct value by using the counterterm of
the static classical theory and the fact that $\Sigma_{\rm HTL}$ is
momentum independent. For $(\omega_W^{\rm b})^2$, in contrast, the
$\beta_G$-dependent part comes from eq.~\nr{omegaWa} and cannot be
removed.  Parametrically, $(\omega^{\rm b})^{2}\sim g^2T^2$ and
$^\ast\Pi(\omega_W^{\rm b})$,
$^\ast\Sigma(\omega_H^{\rm b})\sim g^3T^2$, so
that the dominant contributions to the $H$ and $W^a_i$ plasmon
frequencies (appearing as $C(t,{\bf p}) \sim C_0 \exp({-\Gamma t})
\cos(\omega_{\rm pl} t + \delta)$ in the correlator) are just
$\omega_H^{\rm b}$ and $\omega^{\rm b}_W$ according to eq.~\nr{Ctp}.
The damping rates $\Gamma$ are related to
$^\ast\Pi(\omega_W^{\rm b})$
and $^\ast\Sigma(\omega_H^{\rm b})$.

\begin{figure}[tb]

\vspace*{-1.0cm}

\hspace{1cm}
\epsfysize=18cm
\centerline{\epsffile{omega.eps}}

\vspace*{-6cm}

\caption[a]{
A comparison of the leading order perturbative
plasmon frequencies with those determined on the
lattice in~\cite{tang}. The continuous curves are the
perturbative values from eqs.~\nr{wbW}, \nr{wbH}, and the squares
are the data points from Table~5 in~\cite{tang}.
Open symbols correspond to $\omega_W^{\rm b}$ and
filled to $\omega_H^{\rm b}$. The corresponding
$\beta_G$ values are given next to the symbols.}
\la{omega}
\end{figure}

In Fig.~\ref{omega} we compare the leading order plasmon frequencies
in eqs.~\nr{wbW}, \nr{wbH} with the lattice results of
ref.~\cite{tang}. The value of $v(T)$ has been determined from the
1-loop effective potential.  The zero-temperature parameters used
in~\cite{tang} correspond to $m_H$ = 80 GeV.  It is seen that the
lattice results are remarkably close to the leading order
perturbative results. In particular, we conclude that the gauge field
plasmon frequency $\omega_W^{\rm b}$ diverges in the continuum limit
according to eq.~\nr{wbW}, while the scalar plasmon frequency
$\omega_H^{\rm b}$ remains finite and equals the static screening mass
at leading order. One sees that the lattice is so coarse that it is
difficult to notice the divergence of $\omega_W^{\rm b}$ since this is
shadowed by the finite $m_W(T)$.  Thus one is in a sense not close
enough to the continuum limit.  It should also be noted that the
amplitude of $C^{ab}_{W,ij}(t)$ dies out as $1/(\omega_W^{\rm b})^{2}$
in the continuum limit.

The damping rates of the gauge and scalar fields are parametrically of order
$g^2T$ and therefore,
in contrast to the plasmon frequencies,
they are classical. This has been demonstrated
explicitly in a scalar field theory~\cite{aarts,jakovac}.
A full computation of the damping rates
in the broken phase of the SU(2)+Higgs theory
is missing at the moment. However, the order of magnitude can
apparently be understood~\cite{tang} using the known symmetric phase
gauge and Higgs elementary field damping rates~\cite{braaten,biro}.

\subsubsection*{The symmetric phase}

Let us now turn to the symmetric phase.
The correlators of the composite operators in eqs.~\nr{operH},
\nr{operW} have been determined in the symmetric phase
of the SU(2)+Higgs model by Tang and Smit~\cite{tang}.
In addition, the gauge field correlator of the pure SU(2)
model has been measured in the Coulomb gauge by
Ambj{\o}rn and Krasnitz~\cite{ambjorn}.

Consider first the composite operator correlators measured
in~\cite{tang}. In the symmetric phase,
the composite operator character of $H(t)$ and $W^a_i(t)$
manifests itself more clearly than in the broken phase.
For instance, the leading term in $\bar W^a_i$ is
\begin{eqnarray}
  \bar W^a_i = \frac{1}{\sqrt{N^3}}\sum_\vec{x} \left[
  \bar\phi_0(\vec{x} +
  \vec{e}_i a)\bar\phi_a(\vec{x})-\bar\phi_0(\vec{x})\bar\phi_a(\vec{x} +
  \vec{e}_i a)-\epsilon^{abc}
  \bar\phi_b(\vec{x})\bar\phi_c(\vec{x} + \vec{e}_i a)\right],
\end{eqnarray}
which does not contain the gauge field $A^a_i$ at all.

\begin{figure}[tb]

\vspace*{-4.0cm}

\hspace{1cm}
\epsfysize=30cm
\centerline{\epsffile{diagram.ps}}

\vspace*{-22cm}

\caption[a]{The lowest order diagram
contributing  to $C_H(t)$ and $C^{ab}_{W,ij}(t)$ in the symmetric phase.
The solid lines denote scalar propagators.}
\label{diagram}
\end{figure}

Due to the fact that no gauge fields are involved, the leading terms in
$C_H(t)$ and $C^{ab}_{W,ij}(t)$ can be easily computed: both are given
by diagrams of the type depicted in Fig.~\ref{diagram}. To evaluate
them one has to start with a Matsubara external momentum $p_0 =
i\omega_n$. Then the sum over the loop frequencies is written in terms
of an integral in the complex $k_0$ plane (see, e.g.,~\cite{kapusta}).
Only then can one continue to arbitrary complex values of $p_0$.  For
the operator ${\cal O}(t,\vec{x}) =
\varphi^\dagger\varphi(t,\vec{x})$, the diagram in Fig.~\ref{diagram}
finally gives
\begin{eqnarray}
  & & {\rm Im} {\cal D}_{\varphi^\dagger\varphi}(p_0 + i\epsilon,\vec{p})
  = \nonumber \\
  & & \quad\quad
  4 \int\! \frac{d^4k}{(2\pi)^4}
  \[n(k_0) - n(k_0+p_0) \]
  {\rm Im} {\cal D}(k_0 +i\epsilon, \vec{k})\,
  {\rm Im} {\cal D}(k_0 + p_0 + i\epsilon, \vec{k} + \vec{p}).
\hspace*{1.0cm}
\label{imDphi2}
\end{eqnarray}
This expression\footnote{Eq.~\nr{imDphi2} can be written
in other forms by a change of integration variables, but
then one may get a wrong result for the
free case, if $\epsilon$ is put to zero inside the integral
and eq.~(\ref{help1}) is used.} is equivalent to
a corresponding one derived in ref.~\cite{jeon}.
The correlator $C_H(t)=\fr12\langle H(t)H(0)+H(0)H(t)\rangle$
for the operator $H(t)$ of eq.~\nr{operH}
is given by $C_H(t)=C_{\varphi^\dagger\varphi}(t,\vec{p}=0)$,
where $C_{\varphi^\dagger\varphi}(t,\vec{p})$ is obtained from
eq.~\nr{relation}.

To get the leading contribution in $C_H(t)$, one uses the
free propagators for which
\begin{eqnarray}
         {\rm Im} {\cal D}_{\rm free}(k_0 +i\epsilon, \vec{k}) =
         \frac{\pi}{2\sqrt{\vec{k}^2 + \omega_H^2}}
        \[\delta\(k_0 - \sqrt{\vec{k}^2 + \omega_H^2}\) -
        \delta\(k_0 + \sqrt{\vec{k}^2 + \omega_H^2}\)\]. \label{help1}
\end{eqnarray}
Then the integrals over $k_0$ and $p_0$ can be performed
in eqs.~\nr{imDphi2}, \nr{relation}.
In the limit $ \vec{p}\to 0$, one obtains
\begin{eqnarray}
  \label{chfree}
  C_H(t) = T^2 \int \frac{d^3k}{(2\pi)^3} \frac{1}{(\vec{k}^2 +\omega_H^2)^2}
  \left[1 + \cos\left(2\sqrt{\vec{k}^2 +\omega_H^2}\, t\right)\right].
  \la{loCHt}
\end{eqnarray}
On the lattice (in the continuum limit) this corresponds to
\begin{eqnarray}
  C_{\bar H}(t)=
  \frac{1}{4 \pi^2\beta_G^2}
  \int_0^{\infty} dx \frac{x^2}{(x^2+z^2)^2}\left[1+ \cos(4 \bar
  t\sqrt{x^2+z^2})\right], \la{symch}
\end{eqnarray}
where $z=a\omega_H/2$ and $\bar t=t/a$.

The vector correlator
$ C^{ab}_{W,ij}(t)$ has an additional factor of $k_ik_j$
in the integrand compared with eq.~\nr{loCHt}.
Therefore the continuum limit of  $ C^{ab}_{W,ij}(t=0)$ does
not exist. On the lattice one obtains
\begin{eqnarray}
  C_{\bar W}(t) = \frac{2}{\pi^3\beta_G^2}
  \int_0^{\pi/2}\!d^3 x
  \frac{\sum_i\sin^2 2x_i}{[\sum_i\sin^2 x_i+z^2]^2}\left[1+
  \cos\left(4 \bar
  t \sqrt{{\textstyle \sum_i}\sin^2 x_i + z^2} \right)
  \right]. \la{symcw}
\end{eqnarray}

\begin{figure}[tb]

\vspace*{-1.0cm}

\hspace{1cm}
\epsfysize=18cm
\centerline{\epsffile{ch.eps}}

\vspace*{-6cm}

\caption[a]{
The leading order
perturbative correlator $C_{\bar H}(t)$ in lattice units
in the symmetric phase at $T/T_c=1.52$. To be compared
with Fig.~8 in~\cite{tang}. The oscillation frequency
is $2\omega_H$ (this agrees at leading order with the
static screening mass).}
\la{ch}
\end{figure}

\begin{figure}[tb]

\vspace*{-1.0cm}

\hspace{1cm}
\epsfysize=18cm
\centerline{\epsffile{cw.eps}}

\vspace*{-6cm}

\caption[a]{
The leading order
perturbative correlator $C_{\bar W}(t)$
in lattice units
in the symmetric phase at $T/T_c=1.52$. To be compared
with Fig.~10 in~\cite{tang}. The oscillation period here
is proportional to the lattice spacing $a$.}
\la{cw}
\end{figure}

The correlation function $C_{\bar H}(t)$ is shown in Fig.~\ref{ch}
and $C_{\bar W}(t)$ in Fig.~\ref{cw}. These should be compared with
Figs.~8, 10 in~\cite{tang}, respectively. It is seen that the
qualitative features can be
understood quite well with the leading order results.
Note, in particular, that
the scalar correlation function $C_{\bar H}(t)$
in Fig.~\nr{ch} is oscillating with an
amplitude which decreases with time. It should be emphasized that
this decrease is not related to damping (remember that Fig.~\ref{ch}
shows the tree level results without any interactions and damping
occurs only through interactions). The decrease
is rather due to the fact that
there is a continuous spectrum of frequencies $\omega>2\omega_H$
causing a destructive
interference in the phase space integral, eq.~\nr{symch}.  This shows that
it is difficult to determine a damping rate from the gauge invariant
operator $C_{\bar H}(t)$.

Let us then try to estimate how higher order corrections could modify
the qualitative behavior of
$C_H(t)$.  Consider the effect of self-energy insertions in the scalar
propagators ${\cal D}(k_0,\vec{k})$.
The self-energy has an
imaginary part.  Therefore the scalar propagator does not have poles
at $k_0=\pm\sqrt{\vec{k}^2+\omega_H^2 }$ (the lowest order result for
$C_H(t)$ is due to these poles). Since the imaginary part of the
self-energy is small compared with $\sqrt{\vec{k}^2 + \omega_H^2 }$, $C_H(t)$
will nevertheless still be dominated by the region $k_0 \approx
\pm\sqrt{\vec{k}^2+\omega_H^2 }$.  One can therefore approximate the
scalar propagator as
\begin{eqnarray}
        {\cal D} (k_0 + i\epsilon, \vec{k})
        \approx \frac{1}{-(k_0 + i\Gamma_k)^2
        + \vec{k}^2 + \omega_H^2 },
\label{breit}
\end{eqnarray}
where the width $\Gamma_k$ is given by (see, e.g., \cite{jeon})
$\Gamma_k \equiv -{\rm Im}\Sigma(\sqrt{\vec{k}^2 + \omega_H^2
  },\vec{k})/(2\sqrt{\vec{k}^2 + \omega_H^2 })$.  Inserting this into
eq.~(\ref{imDphi2}) and using eq.~(\ref{relation}), we find
\begin{eqnarray}
        C_H(t) =  T^2 \int\frac{d^3k}{(2\pi)^3}
        \frac{e^{-2\Gamma_k t}}
        { [\vec{k}^2 + \omega_H^2 ][\vec{k}^2 + \omega_H^2 + \Gamma_k^2]}
        \left[1 + \cos\left(2\sqrt{\vec{k}^2 + \omega_H^2} \,t
        - 2 \alpha_k\right)\right], \la{gCHt}
\end{eqnarray}
where
$\alpha_k =
\arctan\left({\Gamma_k}/{\sqrt{\vec{k}^2 + \omega_H^2}}
\right)$.
Thus the effect of damping should show up in Fig.~\ref{ch}
such that the constant part (corresponding to the first term
in the square brackets in eq.~\nr{gCHt})
decays away at large times.
This is indeed the qualitative behavior observed in~\cite{tang}.

Finally, we consider the transverse gauge field correlator
$C_{A_{\scriptstyle \rm t}}(t,{\bf p})$ measured in~\cite{ambjorn}. Gauge fields
can only be defined in a particular gauge, which in~\cite{ambjorn}
was chosen to be the Coulomb gauge.  We let the external momentum
point into the $x_3$-direction, ${\bf p}={\bf e}_3 (2\pi k/L)$, where
$L$ is the spatial extent of the lattice and $k$ is an integer.  Then
$C_{A_{\scriptstyle \rm t}}(t,{\bf p})$ is given, e.g., by the correlator of
$A_1^a$,
\begin{eqnarray}
   C_{A_{\scriptstyle \rm t}}(t,{\bf p}) = \frac12 \int d^3 x e^{-i\vec{p}\vec{x}}
   \langle A_1^a(t,{\bf x}) A_1^a(0,{\bf 0}) + A_1^a(0,{\bf 0})
    A_1^a(t,{\bf x})
    \rangle. \la{AA}
\end{eqnarray}

Let us first recall some features of $C_{A_{\scriptstyle \rm t}}(t,\vec{p})$ and of the
corresponding analytic Green's function ${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$ for
$|\vec{p}|\sim g^2 T$ in the quantum theory. After the HTL
resummation, ${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$ has poles at
$\omega\approx\pm\omega_W$, where $\omega_W$ is given by
eq.~\nr{omegaWT}.  These poles lead to an oscillation of
$C_{A_{\scriptstyle \rm t}}(t,\vec{p})$ on the time scale $\sqrt{\hbar}/(gT)$. In addition,
${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$ has a discontinuity on the real $\omega$-axis
for $\omega < |{\bf p}|$.  This discontinuity is related to Landau
damping and it gives a contribution $f_{\rm Landau}(t,\vec{p})$ to
$C_{A_{\scriptstyle \rm t}}(t,\vec{p})$. The function $f_{\rm Landau}(t,\vec{p})$ does not
involve any oscillations and just constitutes a decaying background
for the superimposed plasmon oscillations. The time scale on which
$f_{\rm Landau}(t,\vec{p})$ varies is $\mathop{\gsi} 1/(g^2T)$.

Higher order
corrections to ${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$,
which are not included in HTL effective action but are generated
radiatively within that theory, lead to
a damping of the plasmon oscillations.
The plasmon damping rate $\Gamma$
is of order $g^2T$ and
has been
computed for $\vec{p} = 0$
in ref.~\cite{braaten}.

These two different damping effects manifest themselves in
the correlator $C_{A_{\scriptstyle \rm t}}(t,{\bf p})$ in quite different ways, so that its
functional form is expected to be
\begin{eqnarray}
  C_{A_{\scriptstyle \rm t}}(t,{\bf p}) \sim
  A \exp({-\Gamma t})\cos (\omega_{W} t+\delta) +  f_{\rm Landau}(t,\vec{p}).
        \label{form}
\end{eqnarray}
The time dependence of $f_{\rm Landau}(t,\vec{p})$ becomes
non-perturbative for $t \mathop{\gsi} 1/(\hbar g^4T)$ \cite{arnold96}, where
$f_{\rm Landau}(t,\vec{p})$ is expected to vanish.  This means that
$C_{A_{\scriptstyle \rm t}}(t,{\bf p})$ can be computed perturbatively up to a
non-perturbative constant $C_{A_{\scriptstyle \rm t}}(0,{\bf p})$ as long as $t\ll 1/(\hbar
g^4T)$.

In the classical lattice gauge theory one expects a similar
qualitative behavior. In the order of magnitude estimates for
the quantum theory one has to replace $\hbar\to Ta$. The analytic
structure of ${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$ is more complicated than in the
quantum case. The HTL resummed ${\cal D}_{A_{\scriptstyle \rm t}}(\omega,\vec{p})$ depends not only
on the magnitude but also on the direction of $\vec{p}$
\cite{bodeker}.  In particular, there are
directions of $\vec{p}$ for which there is a discontinuity for
arbitrarily large values of $\omega$ \cite{arnold97}.  Nevertheless, the
qualitative behavior of $C_{A_{\scriptstyle \rm t}}(t,\vec{p})$ should be given by
eq.~\nr{form}.

Unfortunately, the numerical results for $C_{A_{\scriptstyle \rm t}}(t,\vec{p})$
in~\cite{ambjorn} have been normalized to $C_{A_{\scriptstyle \rm t}}(0,\vec{p})$ which
cannot be computed in perturbation theory. Comparing our perturbative
estimates with the non-perturbative results therefore requires some
model assumptions about  $C_{A_{\scriptstyle \rm t}}(0,\vec{p})$.

In order to account for the plasmon damping effects, we include the
leading order damping rate $\Gamma$ in the HTL resummed propagator
${\cal D}^{ab}_{ij}(\omega,{\bf p})$.  Since the momentum dependence
of $\Gamma$ is not known we use its value for $\vec{p} = 0$ which is
$\Gamma = 0.176 g^2T$ \cite{braaten}.  We can then write for the
transverse components in eq.~\nr{AA}, analogously to eq.~\nr{breit},
\begin{eqnarray}
   {\cal D}^{ab}_{11}(\omega+i\epsilon,{\bf p}) =
   \frac{\delta^{ab}}{
   -(\omega+i \Gamma)^2 + \vec{p}^2 +
   \Pi_{\rm HTL}^{11}(\omega, {\bf p})}. \la{breit2}
\end{eqnarray}

Consider first the zero external momentum case, ${\bf p}={\bf 0}$.
Eq.~\nr{breit2}
only makes sense when $\Gamma^2 \ll \vec{p}^2 +
\mathop{\rm Re} \Pi^{11}_{\rm HTL}(\omega,\vec{p})$.
In the latter term in eq.~\nr{Ctp} there should be no problem,
since for $\vec{p}=0$,
$\Pi^{11}_{\rm HTL}(\omega,\vec{p}) \to \omega_W^2$, but for
the first term
(i.e., for the static limit $t=0$) the inequality
is not satisfied
(remember that $\Pi_{\rm HTL}^{11}(0, {\bf p})=0$).
One might try to regulate the real part of the gauge fixed self-energy
phenomenologically with a ``magnetic mass'';
letting $M=\sqrt{m_{\rm magn}^2+\Gamma^2+\omega_W^2}$,
$\widetilde M=\sqrt{m_{\rm magn}^2+\omega_W^2}$, it follows
from eq.~\nr{Ctp} that this would give
\begin{eqnarray}
C_{A_{\scriptstyle \rm t}}(0) & = & \frac{T}{m_{\rm magn}^2+\Gamma^2}, \\
C_{A_{\scriptstyle \rm t}}(t)- C_{A_{\scriptstyle \rm t}}(0) & = &
-\frac{T}{M^2}+\frac{T}{M\widetilde M}
e^{-\Gamma t}\cos\!
\left(\widetilde M t-
\arctan\frac{\Gamma}{\widetilde M}
\right).
\end{eqnarray}
Parameterized this way, one can indeed find quite reasonable agreement
with Fig.~6 in~\cite{ambjorn}, but only if $\Gamma$ is chosen to have
a large value, $\Gamma \sim (0.8\ldots 1.0) g^2T$.  Otherwise one is
getting too many oscillations in $C_{A_{\scriptstyle \rm t}}(t)$, not observed
in~\cite{ambjorn}. For these large values of $\Gamma$, the magnetic
mass parameter is in fact favored to be small or zero. However, these
fits are quite phenomenological, and thus we will not consider them
any more.

For ${\bf p}\neq 0$, the $t=0$ --part of the leading order
correlator is still parametrically non-perturbative, but at least it is
formally finite for $\Gamma\to 0$ so that
the perturbative approximation
might be numerically reasonable.
To get a feeling about the momentum scales in question, note that
for the value $\beta_G=14$ considered in~\cite{ambjorn},
$|\vec{p}|=0.69g^2T\times k$ and
$\omega_W=0.87g^2T$ (the latter can be obtained from eq.~\nr{wbW}
with $m_W(T)=0$).
We have evaluated numerically both the HTL self-energy in
eq.~\nr{PiW} and
the remaining $\omega_1$-integral in eq.~\nr{Ctp},
for $k=1,\ldots,4$.
The resulting correlators are shown in Fig.~\ref{Acorr}.
This should be compared with Fig.~6 in~\cite{ambjorn}.

\begin{figure}[tb]

\vspace*{-1.0cm}

\hspace{1cm}
\epsfysize=18cm
\centerline{\epsffile{Acorr.eps}}

\vspace*{-6cm}

\caption[a]{
The gauge field correlator in the pure SU(2) theory.
To be compared with Fig.~6 in~\cite{ambjorn}. Here $t$
is expressed in lattice units: $\bar t\equiv t/a$.
Unless otherwise stated, $\beta_G=14$.}
\la{Acorr}
\end{figure}

The main effects to be seen
in Fig.~\ref{Acorr} are the following.
For $k=1$ one can see an oscillation in Fig.~\ref{Acorr},
in contrast to Fig.~6 in~\cite{ambjorn}. Thus
one could say that the non-perturbative
plasmon damping rate is larger than the perturbative estimate
$\Gamma = 0.176 g^2T$. Indeed,
one has to go to a much larger value, $\Gamma\sim 1.0 g^2T$
($>|\vec{p}|, \omega_W$),
to get enough plasmon damping. Another observation to be made
at $k=1$ is that in~\cite{ambjorn} the correlator is already
very close to zero at $\bar t/\beta_G^2=0.10$. This is not quite so in
Fig.~\ref{Acorr}, but one has to wait much longer for the
correlator to vanish. Thus the non-perturbative Landau damping
effects seem also to be larger than the leading order perturbative
HTL result. At $k=2$ there is no large difference in
Landau damping any more, but the plasmon damping appears
to be somewhat too weak even with $\Gamma\sim 1.0g^2T$.
Finally, at $k=4$ one gets reasonable
agreement between the perturbative and lattice results,
provided that $\Gamma\sim 1.0g^2T$. One can also
see that the $\beta_G$-dependence is reproduced;
the plasmon frequency thus diverges according to eq.~\nr{omegaWa}.

We have also tried the expression $-\omega^2-2i\omega\Gamma$
in the denominator of eq.~\nr{breit2}, so that the real part
is not modified by $\Gamma^2$. The qualitative conclusions
and the preferred value $\Gamma\sim 1.0g^2T$ remain the same.
Based on those curves, one would nevertheless say that even
at $k=2$ there is too little Landau damping in the
perturbative estimates, but at $k=4$ one again gets good
agreement.

\subsubsection*{Summary and Conclusions}

We have computed several quantities related to real time correlation
functions in the classical SU(2) and SU(2)+Higgs models on the lattice,
using hard thermal loop resummed perturbation theory.

Our results for the gauge field and scalar plasmon frequencies in the
broken phase are in remarkable agreement with the numerical lattice
simulations in ref.~\cite{tang}. We have reiterated that the classical
gauge field plasmon frequency is divergent in the continuum limit and
we have demonstrated that this is consistent with the results of
ref.~\cite{tang}, where
the plasmon frequency was claimed to be lattice spacing
independent.

For the symmetric phase we have computed gauge invariant scalar
and vector correlators as functions of time at the lowest order
in perturbation theory. Furthermore, we have estimated the effect of
higher order corrections.  Our results are in good
agreement with ref.~\cite{tang}.
We have shown that it is difficult to extract damping rates from
the measurement of these correlators.

Finally, we have studied the correlator of the transverse gauge field
in pure SU(2) gauge theory. While the qualitative features of the
numerical simulations in ref.~\cite{ambjorn} are consistent with our
perturbative estimates, there appear to be significant quantitative
discrepancies. The damping of the plasmon oscillations observed
in~\cite{ambjorn} appears to be much stronger than one would expect
from the perturbative result for the damping rate~\cite{braaten}.
This is puzzling because the damping rate is of the order $g^2 T$ and
should therefore have a classical continuum limit.  However,
one should keep in mind
that the perturbative estimates are reliable only if the
plasmon frequency is much larger than the damping rate which is not
the case for the lattice spacing used in ref.~\cite{ambjorn}.

{\bf Acknowledgments.}
We are grateful to E.~Berger, P.~Overmann, O.~Philipsen,
M.G.~Schmidt and
I.O.~Stamatescu for useful discussions.

\subsubsection*{Note added}

After this paper was submitted,
we were informed by J.Smit that
in the revised version of Ref.~\cite{tang},
Tang and Smit have weakened
their conclusions concerning the lattice spacing independence of
the $W$ plasmon frequency in the broken phase, so that their
statements are now in better accordance with ours. We thank
J.Smit for communication on this issue.

In view of the interpretation in~\cite{ay2},
let us stress that the discussion around eq.~\nr{breit}
is not meant to be a consistent quantitative
estimate of the higher order corrections. We just want to
see in which qualitative way the higher order corrections
might manifest themselves.


\section{Summary}
\label{sec:conclusion}

In this work, we explored the transferability of HINTS on different computational domains for differential equations. Specifically, we considered two methods: (1) directly employ the HINTS for an equation defined in an unseen domain; (2) adopt the transfer operator learning to fine-tune the HINTS trained on the source domain for the unseen/target domain with limited data. Through presenting the results for Darcy flow and linear elasticity, we demonstrate the effectiveness of the two methods based on HINTS on fast, accurate solutions of differential equations. In particular, HINTS with transfer learning, by leveraging both the knowledge from the source domain and the target domain, converges even faster than using the direct application of HINTS on the target geometry. Despite the faster convergence, it comes with a price that one still needs a small dataset on the target domain. While we have focused on a specific instance of $k(\boldsymbol{x})$ ($E(\boldsymbol{x})$ for elasticity) and $f(\boldsymbol{x})$ in \autoref{sec:experiments}, the performance is consistent for the entire test dataset. \autoref{tab:quantified_results} shows the mean, median and standard deviation (STD) of the number of iterations needed for convergence for 100 different cases in the test dataset.

\begin{table}[htbp!]
    \centering
    \begin{tabular}{||c | c | c | c | c ||}
        \hline
        Problem & Method & Mean & Median & STD \\ [0.5ex]
        \hline\hline
        \multirow{3}{*}{Darcy} & GS & 403 & 405 & 34.62 \\
        \cline{2-5}
        & HINTS-GS & 165 & 164 & 19.03 \\
        \cline{2-5}
        & Transferred HINTS-GS & \textbf{157} & \textbf{162} & \textbf{15.75} \\
        \hline
        \multirow{3}{*}{Elasticity} & GS & 1029 & 1020 & 64.41 \\
        \cline{2-5}
        & HINTS-GS & 257 & 253 & 34.56 \\
        \cline{2-5}
        & Transferred HINTS-GS & \textbf{176} & \textbf{173} & \textbf{15.68} \\
        \hline
    \end{tabular}
    \caption{Summary of the results for the two benchmark problems. Mean, median and standard deviation of the number of iterations it takes for each method to converge to machine precision. The statistical measures were computed for 100 target samples of each one of the Darcy and elasticity problems.}
    \label{tab:quantified_results}
\end{table}

The capability of the direct application of HINTS for unseen geometries is, to some extent, rather surprising. Seemingly, a DeepONet trained for a fixed geometry (e.g., L-shaped domain) should not be effective on another geometry (e.g., L-shaped domain with a cutout) that is not included in the training dataset. We attribute the functionality of HINTS for unseen geometry to the following two factors: $(1)$ DeepONet simply needs to provide an approximate solution within HINTS, while the task of achieving accuracy is accomplished by the embedded numerical solver; $(2)$ the differential equation defined on the unseen geometry, for the examples that we consider, is similar to the equation defined on the original geometry but with an input function $k(\boldsymbol x)$ ($E(\boldsymbol x)$) defined in an extended domain. For $(1)$, intuitively, the prediction error of the DeepONet caused by the mismatch between the original and the new geometry depends on the difference between the two geometries. Within a reasonable degree of similarity between the two geometries, DeepONet can still decrease the errors of the low-frequency modes. For $(2)$, using the case of Darcy flow as an example, it may be shown that the differential equation defined in the L-shaped domain excluding the cutout is equivalent to the same equation defined in the L-shaped domain, where (a) within the cutout $k(\boldsymbol x)$ is simply padded with zero, and (b) the boundary condition at the cutout boundary is zero Neumann boundary condition. Technically, our approach of padding $k(\boldsymbol x)$ inside the cutout with zeros conforms with such equivalence. Therefore, generalizing the L-shaped domain into a new geometry (L-shaped domain with a cutout) is transformed into the generalization of $k(\boldsymbol x)$ from GRF in the training dataset into an unseen $k(\boldsymbol x)$, where it is from GRF outside the cutout but equals to zero inside the cutout.

\section{Introduction}

Numerical simulations play a crucial role in scientific and engineering applications such as mechanics of materials and structures \cite{hughes2012finite,simo2006computational,hughes2005isogeometric,jing2002numerical,rappaz2003numerical, goswami2019adaptive,bharali2022robust}, bio-mechanics \cite{zhang2022g2,goswami2022neural}, fluid dynamics \cite{patera1984spectral,kim1987turbulence,cockburn2012discontinuous}, etc. The simulation approach is based on solving linear/nonlinear partial differential equations (PDEs). The efficiency and accuracy of a simulation approach is always comparable to conflict partners. This means that the quest for a more efficient numerical solver frequently results in lesser numerical accuracy. In engineering simulations, the main factor is to have an acceptable accuracy with feasible computational requirements for both memory utilization and computing time. Furthermore, the solution process must be stable and dependable. Therefore, determining an appropriate approach for the problem at hand is crucial, and usually determines the outcome of a simulation.

In traditional numerical solvers like the finite element method, we often reduce the complex differential equations defining the physical system to a system of linear equations of the form: $\left[\mathbf{K}\right]\{\boldsymbol{u}\} = \{\boldsymbol{f}\}$, where $\left[\mathbf{K}\right]$ is referred as the stiffness matrix; $\{\boldsymbol{f}\}$ is the force vector and $\{\boldsymbol{u}\}$ is the set of unknowns. A simple, yet not recommended way to solve for the set of unknown is to use the direct method by inverting the stiffness matrix and multiplying it with a force vector. However, the direct method fails in cases of large degrees of freedom (the stiffness matrix is in the order of a few millions) and/or the stiffness matrix is sparse. At this juncture, the iterative solvers come to the rescue. We start with an initial guess for $\{\boldsymbol{u}\}$, and gradually progress towards the true solution for $\{\boldsymbol{u}\}$. The solver iterates until a reasonable solution that meets the stopping criterion (typically an error tolerance value) is obtained. Iterative solvers are appropriate for large computing problems because they can frequently be parallelized more efficiently using algorithms. However, proper pre-conditioning of the stiffness matrix is a mandatory requirement. The solution's high oscillatory component can be solved efficiently using a dense mesh and few steps with a simple iterative method like Jacobi iteration or Gauss-Seidel (GS) method. It suffers from divergence for non-symmetric and indefinite systems over a coarse grid, as well as slow convergence associated with low-frequency eigen modes, restricting its application for large scale linear systems.

Recent advances in deep learning in addition to the developments in computational power have provided the means to employ neural networks as efficient approximators for PDEs. Their compositional character differs from the traditional additive form of trial functions in linear function spaces, where PDE solution approximations are built using Galerkin, collocation, or finite volume approaches. Their computational parametrization via statistical learning and large-scale optimization approaches makes them increasingly suitable for solving nonlinear and high-dimensional PDEs. However, the neural network often learns the low-frequency eigen modes, and tends to avoid the high-frequency modes. This phenomenon referred to as spectral bias is observed in  numerous applications of neural networks.

In one of the recent works \cite{zhang2022hybrid}, we proposed an efficient approach to integrate synergistically the iterative solvers with deep neural operators to exploit the merits of both the solvers in turn and to overcome the challenges of either of them. The approach acronym as \emph{HINTS}, improves the convergence of the solution across the spectrum of eigen modes by leveraging the spectral bias of a deep neural operator. As observed in the seminal work of HINTS, the solution is flexible with regards to the computational domain and is transferable to different discretizations. In this work, we investigate the transferability properties, i.e., the ``\emph{T}'' in HINTS, with respect to domain adaptation. Specifically, the information from a model trained on a specific domain (\textit{source}), is employed to infer the solution on a different but closely related domain (\textit{target}).

Additionally, we integrate HINTS with the seminal work of operator level transfer learning \cite{goswami2022deep} to improve the convergence rate of HINTS on the target domain. In this scenario, we use a small number of labelled data to fine tune the target model using samples from the target domain. The model is initialized with the learnt parameters of the source model and is trained under a hybrid loss function, comprised of a regression loss and the Conditional Embedding Operator Discrepancy (CEOD) loss, used to measure the divergence between conditional distributions in a Reproducing Kernel Hilbert Space (RKHS). The target model is trained only for the deeper layers, acknowledging the widely accepted fact that the shallow layers are responsible for capturing the more general features.

As a summary, we investigate the capability of HINTS trained on a source domain to operate on a target domain with the following two approaches:
\begin{itemize}
    \item direct application of HINTS to an unseen target domain without retraining the DeepONet;
    \item usage of transfer learning to fine-tune the HINTS with limited data associated with the target domain.
\end{itemize}
This is illustrated in \autoref{fig:approaches}.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=10cm]{media/Fig_method.png}
    \caption{Diagram showing the proposed approaches. After building the source HINTS, one can use it directly on the target domain (left branch). In addition, we propose using transfer learning to fine tune the DeepONet of the source HINTS to get even better performance (right branch).}
    \label{fig:approaches}
\end{figure}

In this manuscript, we have considered the Darcy's model on a L-shaped domain (source), and the same domain with a circular or a triangular inclusion (target), and the linear elasticity model on a square domain (source) and a square domain with a circular inclusion (target). The remainder of the manuscript is organized as follows. In \autoref{sec:related_work}, we review the existing traditional numerical methods for solving linear systems, along with the recently popular deep learning solvers. This section also briefly covers the state-of-the-art neural operators, the Deep Operator Network (DeepONet), and a small discussion on the seminal work of HINTS and operator level transfer learning. In \autoref{approach}, we present the methodology of HINTS followed by the integration of the transfer learning approach. The experiments carried out to show the efficiency of domain adaptation with and without the transfer learning are presented in \autoref{sec:experiments}. Finally, we summarize our observations and provide concluding remarks in \autoref{sec:conclusion}.

\section{Method} \label{approach}

Without any loss of generality, we consider a family of PDEs defined in a domain $\Omega$:
\begin{align}\label{eqn:diffeqn_general}
    \mathcal{L}_{\boldsymbol{x}}(\boldsymbol{u};k) &= \boldsymbol{f}(\boldsymbol{x}), \quad \boldsymbol{x} \in \Omega \\
    \mathcal{B}_{\boldsymbol{x}}(\boldsymbol{u}) &= \boldsymbol{g}(\boldsymbol{x}), \quad \boldsymbol{x} \in \partial\Omega \label{eqn:bound_general},
\end{align}
where $\mathcal{L}_{\boldsymbol{x}}$ is a differential operator, $\mathcal{B}_{\boldsymbol{x}}$ is a boundary operator, $k=k(\boldsymbol{x})$ parameterizes $\mathcal{L}_{\boldsymbol{x}}$,  $\boldsymbol{f}(\boldsymbol{x})$ and $\boldsymbol{g}(\boldsymbol{x})$ are the forcing terms, and $\boldsymbol{u} = \boldsymbol{u}(\boldsymbol{x})$ is the solution of the given PDE. With a well-trained DeepONet embedded, a HINTS is capable of solving for $\boldsymbol{u}$ corresponding to $k$ and $\boldsymbol{f}$, i.e., it captures the solution operator $\mathcal G$ of the family of PDEs specified by \autoref{eqn:diffeqn_general} and \autoref{eqn:bound_general}:
\begin{equation}
    \label{eqn:source_hints}
    \mathcal G: k, \boldsymbol{f} \mapsto \boldsymbol{u}\text{ s.t. \autoref{eqn:diffeqn_general} and \autoref{eqn:bound_general} are satisfied}.
\end{equation}
For detailed information on the implementation of HINTS, readers are suggested to refer to the work \cite{zhang2022hybrid}.

Herein, we first construct a source HINTS, i.e., a HINTS with DeepONet trained offline for \autoref{eqn:diffeqn_general} and \autoref{eqn:bound_general} defined in the source domain $\Omega=\Omega^S$. This is conducted by using $N_S$ labelled source data, $\mathcal D_S = \{k_i, \boldsymbol{f}_i, \boldsymbol{u}_i\}_{i = 1}^{N_S}$. In the DeepONet architecture, the branch network is a convolution neural network comprising of channels, each taking as input one of the mapping functions. The trunk network takes as inputs the spatial location of the points in the domain $\Omega^S$. This DeepONet is trained with a standard regression loss (relative mean squared error) to obtain the optimized weights and biases of the source network, $\boldsymbol{\theta}^S$. Next, we build HINTS by assembling the DeepONet and the numerical solver (e.g., GS). We discretize $\Omega^S$ with triangular elements. The simulation starts by assuming an initial solution of the dependent variable and at every iteration of HINTS it adopts either the pre-decided fixed relaxation method or the pre-trained DeepONet to approximate the solution. The iterative solver and DeepONet is chosen based on a pre-decided ratio. Among the available numerical iterative solvers, we have employed the GS approach.

With the source HINTS properly constructed, we now consider transferring it to become a target HINTS, i.e. a HINTS for solving the class of PDEs (\autoref{eqn:diffeqn_general} and \autoref{eqn:bound_general}) defined in a target domain $\Omega=\Omega^T$, with the following two methods:

\textbf{Direct Application: Use Source DeepONet Directly in Target HINTS}. To do this, the first approach is to directly apply the source HINTS for inference in the target domain $\Omega^T$. Specifically, when implementing the target HINTS (i.e., HINTS for the target domain), the trained DeepONet from the source HINTS is directly invoked in the workflow of the target HINTS.

\textbf{Transfer Learning: Fine-tune the DeepONet in Target HINTS}. In the second approach, we fine-tune the trained DeepONet from the source HINTS with a small number of labelled samples from the target domain. We generate $N_T$ labelled data, $\mathcal D_T = \{\tilde{k}_i, \tilde{\boldsymbol{f}}_i, \tilde{\boldsymbol{u}}_i\}_{i = 1}^{N_T}$ on the target domain, where $N_S \gg N_T$. In the examples presented in this work, $N_T \approx 0.01\times N_S$. We initialize a target model (having the same architecture as the source model), with the learnt parameters of the source model and fine tune the model by training the fully connected layers of the convolution neural network and the last layer of the trunk net under a hybrid loss function. The hybrid loss function reads as:
\begin{equation}
\label{eq:target-loss}
    \mathcal L(\theta^T) = \lambda_1 \mathcal L_r(\theta^T) + \lambda_2 \mathcal L_{\text{CEOD}}(\theta^T),
\end{equation}
where $\lambda_1 = 1$ and $\lambda_2\gg\lambda_1$ are trainable coefficients, which determine the importance of the two loss components during the optimization process \cite{kontolati2022influence}. In \autoref{eq:target-loss}, while the first term is a standard regression loss, the second term ensures the agreement between the conditional distributions of the target data. For details on the construction of the $\mathcal L_{\text{CEOD}}(\theta^T)$, readers are suggested to refer to \cite{goswami2022deep}. While employing the HINTS algorithm for the target domain, the source DeepONet is replaced by the target DeepONet.

Even though the HINTS solution is transferable between related domains, and integrating the transfer learning approach would essentially mean an additional training time, we argue that the convergence rate of a transfer-learning integrated HINTS solution is in faster than its counterpart (see section \autoref{sec:conclusion} for statistical analysis). It is worth noting that the target model is trained with much less iterations and hence is very fast. We mention that there is potential for performing the transfer learning even better, and producing a more accurate target DeepONet, but in this work we did not aim for achieving this. We focus on showing that the transfer learning mechanism works and produces even faster convergence for HINTS.

\section{Related work}
\label{sec:related_work}

\textbf{Classical Numerical Solvers of ODEs/PDEs}. Over the last few decades, many studies have been conducted for developing advanced numerical solvers mainly for approximating the solutions of ODEs and PDEs. Classical methods are the Jacobi and GS methods, which were proposed in the 19th century. Since then, although many advanced algorithms have been proposed, most of the state-of-the-art solvers still use some versions of these two algorithms. The current leaders of the field of numerical solvers are the family of MultiGrid (MG) methods \cite{briggs2000multigrid,bramble2019multigrid}, where one uses a set of discretizations to solve the system. It is common to use Jacobi or GS smoothing inside the MG iterations. Therefore, constraints such as positive definiteness that apply for the Jacobi and GS algorithms, also affect MG as well. There are methods to overcome this, such as the shifted Laplacian method \cite{van2007spectral}, but these suffer from other disadvantages. The solvers compete for the lowest number of iterations needed for convergence, and also other properties such as physical time per iteration, parallelization capabilities, and more. However, in some cases the solvers may diverge (as an example, approximating the solution of an indefinite system), and solvers that are robust and can solve all types of problems are sought after.

\textbf{Machine Learning-Based Numerical Solvers}. Many authors have been investigating the use of AI when designing efficient numerical solvers. Some focus on creating an AI based solver for solving PDEs, replacing the numerical solvers. A notable example is the Physics-Informed Neural Networks \cite{raissi2019physics,goswami2020transfer}, where one trains a network to infer the solution of the PDE in a domain, without the need to assemble a linear system and solve, nor use a complex meshing algorithm. Another direction is to enhance numerical solvers using AI (which is the main focus of this paper). Most recent studies aim to replace components of the MG algorithms with AI based components, for example training a neural network to replace the restriction and prolongation operations \cite{moore2022learning,luz2020learning}. Others try to achieve a better performing preconditioner using learning \cite{gotz2018machine}.

\textbf{DeepONet}. Another notable advancement in the field of AI is the invention of operator learning methods \cite{goswami2022operator}. In contrast to standard Machine Learning (ML) tasks, where one seeks to approximate a function that can connect input data to output data, in operator learning one seeks a mapping between a family of functions and another family of functions that satisfies an operator, hence the name deep operator learning. Learning the operator enables many possibilities, fir example, after the network has been trained and the operator has been learned, one does not need to neither re-train nor solve the system again for new conditions or parameters, but rather infer the solution using the trained system for the new problem definition. This dramatically lowers the cost of solving PDE related problems. In addition, the learned operator does not depend on a discretization and can be used to infer the solution on any given discretization. Several operator learning methods have been proposed, including \cite{li2021fourier}. In this work, we focus on the DeepONet \cite{lu2021learning,goswami2022physics}, and use this for the operator learning.

\textbf{HINTS}. A new method to enhance numerical solvers using operator learning has been proposed under the name HINTS: Hybrid Numerical Iterative Transferable Solver \cite{zhang2022hybrid}. The idea of HINTS is to use an iterative solver such as Jacobi or GS, and replace some of the iterations with a DeepONet trained to receive the problem parameters and infer the solution. This DeepONet can also be used to receive the problem parameters and the residual at the current iteration and produce the correction term for the solution, so it can be applied in a similar way to the numerical solvers. Experiments show that using the existing numerical solvers, they tend to face slow convergence due to their difficulty to smooth the error for low frequency modes (where for the high modes they operate well), while the DeepONet excels in smoothing the low modes errors (but may fail for the high modes). Using both the numerical methods and the DeepONet, uniform convergence of all modes is achieved and the solvers converge to machine precision much faster. HINTS showed promising results on many tasks, and also a lot of potential for extensions, and in this paper we discuss a very important extension of HINTS that mechanics simulations may benefit from - the transferability of HINTS to new geometries and discretizations.

\textbf{Transfer Learning}. The idea of transfer learning is to leverage the parameters of a model trained with sufficiently large labelled dataset to infer information on a related task with few labelled datasets and minimal training. In \cite{goswami2022deep}, an operator level transfer learning was proposed to lower the computational costs of training a DeepONet (from Scratch) for related tasks. In the categorization of TL approaches, one popular classification is based on the consistency between the distributions of source and the target input (or feature) spaces and output (or label) spaces. The shift between the source and target data distributions is considered the major challenge in modern TL. One type of distribution shifts include conditional shift, where the marginal distribution of source and target input data remains the same while the conditional distributions of the output differ (i.e., $P(\mathbf{x}_s) = P(\mathbf{x}_t)$ and $P(\mathbf{y}_s\vert\mathbf{x}_s) \ne P(\mathbf{y}_t\vert\mathbf{x}_t)$) and covariate shift, where the opposite occurs (i.e., $P(\mathbf{x}_s) \ne P(\mathbf{x}_t)$ and $P(\mathbf{y}_s\vert\mathbf{x}_s) = P(\mathbf{y}_t\vert\mathbf{x}_t)$). In this work, we have employed the TL model proposed in \cite{goswami2022deep} to work on conditional distribution discrepancy between the domains. In this work, the authors have reported that the target model essentially requires a smaller dataset to fine tune, which can thus be done in significantly less time.

\section{Computational experiments}
\label{sec:experiments}

In this section, we explore our method using two benchmark problems. The first problem involves the flow in heterogeneous porous media (Darcy's model) on a two-dimensional L-shaped domain (source). The target domains considered in this case are the same L-shaped domain with a circular and a triangular cutout. In the next problem, we have considered a thin rectangular plate (source) subjected to in-plane loading that is modeled as a two-dimensional problem of plane strain elasticity. The target domain considered in this case is the same rectangular plate with a central circular cutout. In both the examples, the DeepONet is trained using the Adam optimizer \cite{kingma2014adam}. The implementation has been carried out using the \texttt{PyTorch} framework \cite{paszke2019pytorch}. For both the examples we initialize the DeepONet parameters using Xavier initialization. Details on the data generation, network architecture, such as number of layers, number of neurons in each layer, and the activation functions are provided with each example.

\subsection{Darcy flow}
\label{subsec:darcy}

In the first example we consider the Darcy's flow on a L-shaped domain.
The problem is defined on a L-shaped domain (source) as:
\begin{align}
    \label{eqn:poisson_PDE}
    \nabla\cdot\left(k(\boldsymbol{x})\nabla u(\boldsymbol{x})\right) + f(\boldsymbol{x}) &= 0,\;\;\; \bm x = (x,y)\in\Omega^S_L:=(0,1)^2\backslash[0.5,1)^2 \\
    u(\boldsymbol x) &= 0, \quad \boldsymbol x\in\partial\Omega^S_L,
\end{align}
where $k(\boldsymbol{x})$ is a spatially varying hydraulic conductivity, $u(\boldsymbol x)$ is the hydraulic head, and $f(\boldsymbol x)$ is a spatially varying force vector. We use triangular elements to discretize the L-shaped domain, $\Omega^S_L$. The spatial discretization is shown in \autoref{fig:domains}(a).

\begin{figure}[!ht]
     \centering
     \begin{subfigure}[t]{0.32\textwidth}
         \centering
         \includegraphics[width=1\textwidth]{media/l_shaped.png}
         \caption{L-shaped domain.}
     \end{subfigure}
     \hfill
     \begin{subfigure}[t]{0.32\textwidth}
         \centering
         \includegraphics[width=1\textwidth]{media/l_shaped_circle.png}
         \caption{L-shaped domain with a circular hole.}
     \end{subfigure}
     \hfill
     \begin{subfigure}[t]{0.32\textwidth}
         \centering
         \includegraphics[width=1\textwidth]{media/l_shaped_triangle.png}
         \caption{L-shaped domain with a triangular hole.}
     \end{subfigure}
     \hfill
     \caption{Mesh discretization for the geometries considered in the Darcy's problem: (a) source domain, (b-c) target domains considered for domain adaptation.}
     \label{fig:domains}
\end{figure}

\bigbreak
\noindent
\textbf{Training Source HINTS}

In this example, the goal is to learn the operator of the system in \autoref{eqn:poisson_PDE}, which maps the random conductivity field and the random force vector to the output hydraulic head, i.e., $\mathcal{G}_\theta: k(\boldsymbol x), f(\boldsymbol x) \rightarrow h(\boldsymbol x)$, where $\mathcal{G}_\theta$ is the solution operator. To generate multiple samples of the conductivity fields and force vectors for training the source DeepONet, we describe $k(\boldsymbol x)$ and $f(\boldsymbol{x})$ as stochastic processes, the realizations of which are generated using a Gaussian Random Field (GRF), with a standard deviation of $0.3$ and $0.1$ for $k(\boldsymbol x)$ and $f(\boldsymbol x)$, respectively and a correlation length of $0.1$ for both the processes. To train the source DeepONet, we generate $N_S = 51\small{,}000$ samples as the labeled dataset of random fields and the corresponding responses, and an additional $N_S^{test} = 9\small{,}000$ samples to test the model.

The branch network of the DeepONet is a combination of a CNN (input dimension $31 \times 31$, number of channels $[2, 40, 60, 100, 180]$, kernel size $3\times 3$, stride $2$, the number of channels of the input 2 comes from the concatenation of $K(\boldsymbol x)$ and $f(\boldsymbol x))$ and a fully-connected network (dimensions $[180, 80, 80]$). The dimension of the trunk network (fully-connected network) is $[2, 80, 80, 80]$. We train the DeepONet for $25\small{,}000$ epochs with a fixed learning rate of $1e-4$, and a mini-batch size of $10\small{,}000$. For the branch network we employ the \texttt{ReLU} activation function and for the trunk network, we use \texttt{Tanh} activation. The last layers of both the branch and the trunk networks have linear activation functions. The source model converges with a mean relative error of $4\%$ on the test dataset.

The trained DeepONet is plugged into the HINTS algorithm and the HINTS iterations are executed until the error of the solution reaches machine zero. An example of such iterative solution is given in \autoref{fig:HINTS_Poisson}. When inspecting the solution, we focus mainly on the convergence efficiency of the solution to machine zero. For that, we observe the norm of the error per iteration, as demonstrated in \autoref{fig:HINTS_Poisson_error_norm}. The founding idea of HINTS is based on the uniform convergence of all the eigen modes, which is shown in \autoref{fig:HINTS_Poisson_modes}. To illustrate the solution of the problem at hand, we show \autoref{fig:HINTS_Poisson_sol}, which is the solution at the last iteration (after convergence). We also show the error between the approximate solution and the exact solution (obtained by directly solving the system instead of using an iterative method) in \autoref{fig:HINTS_Poisson_error}. Note that the scale of the error is machine precision limit, which means the desired convergence has been achieved.

\begin{figure}[htbp!]
     \centering
     \begin{subfigure}[t]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/error_per_iter.png}
         \caption{The error norm per iteration.}
         \label{fig:HINTS_Poisson_error_norm}
     \end{subfigure}
     \begin{subfigure}[t]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/mode_errors.png}
         \caption{The mode errors.}
         \label{fig:HINTS_Poisson_modes}
     \end{subfigure}
     \begin{subfigure}[t]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/approx_sol.png}
         \caption{The approximate solution at the last step.}
         \label{fig:HINTS_Poisson_sol}
     \end{subfigure}
     \begin{subfigure}[t]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/final_error.png}
         \caption{The error at the last iteration of HINTS.}
         \label{fig:HINTS_Poisson_error}
     \end{subfigure}
     \hfill
     \caption{Employing HINTS to solve the Darcy's problem on a L-shaped domain. It is interesting to note that in (d) the scale shows convergence on the error to machine zero at the last iteration of the proposed solver.}
     \label{fig:HINTS_Poisson}
\end{figure}

Next, to investigate the domain adaptation capabilities, we consider the following two tasks:
\begin{itemize}
    \item \textbf{Task1:} From a L-shaped domain to a L-shaped domain with a circular cutout. The target domain is defined as: $\Omega^{T1}_L = \Omega_L \backslash \{(x, y) | (x - 0.25) ^ 2 + (y - 0.25) ^ 2 \leq 0.15\}$.
    \item \textbf{Task2:} From a L-shaped domain to a L-shaped domain with a triangular cutout. The target domain is defined as: $\Omega^{T2}_L = \Omega_L \backslash \{(x, y) | (x, y) \in \bigtriangleup((0.2, 0.1), (0.6, 0.4), (0.3, 0.4))\}$.
\end{itemize}
The discretization of the target domains are shown in \autoref{fig:domains} (b) and (c). While the circular cutout has a smooth boundary and is easier to approximate, the triangular cutout has locations of singularity, and hence imposes a more challenging scenario for domain adaptation.

\textbf{Direct Application:}\\
To begin with, we first investigate the domain adaptation capabilities of source DeepONet to make extrapolated approximations on target domain discretization for target HINTS. Precisely, the source DeepONet is directly employed and is iterated with the relaxation methods to approximate the solution of the dependent variable on the target discretization. On the target domains, we define $k(\boldsymbol{x})$, and $f(\boldsymbol x)$ by setting the input function values to be zero for points within the cutouts. The convergence of the solution is attributed to the generalization ability of DeepONet. The results for the two target tasks are presented in \autoref{fig:HINTS_geometries}. We observe that for Task1, convergence to machine precision is obtained in $182$ iterations. For Task2, even though much slower (takes longer than $300$ iterations), we do achieve convergence as well. We attribute the slow convergence to the three vertices of the triangle, which are singular points and are considered more difficult to handle. The HINTS was able to operate on this geometry and show good performance.

\begin{figure}[htbp!]
     \centering
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/circle_hints.png}
         \caption{HINTS trained on $\Omega_L$, solving on $\Omega_L^{T1}$.}
         \label{fig:HINTS_circle}
     \end{subfigure}
     \hfill
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/triangle_hints.png}
         \caption{HINTS trained on $\Omega_L$, solving on $\Omega_L^{T2}$.}
         \label{fig:HINTS_triangle}
     \end{subfigure}
     \hfill
     \caption{Direct application of HINTS for domain adaptation. The results presented here are the plots of the target domain when HINTS has been directly used for inference without any re-training. Uniform convergence of the modes is observed as shown in the bottom left images for (a) and (b).}
     \label{fig:HINTS_geometries}
\end{figure}

\textbf{Transfer Learning:}\\
As an alternative to the above approach, we propose to fine-tune the DeepONet with a small number of samples from the target domain. To that end, we employ the operator level transfer learning approach proposed in \cite{goswami2022deep}, where the target DeepONet is initialized with the weights and biases of the source DeepONet. To fine tune the target DeepONet, only the fully-connected layers of the branch network and the last layer of the trunk network are re-trained with a hybrid loss function that takes into account the difference in the conditional distribution of the source and the target.

To train the target DeepONet for each of the two tasks, we generate $N_T = 500$ random samples of $k(\boldsymbol x)$ and $f(\boldsymbol x)$ and obtain the corresponding solution $u(\boldsymbol x)$. The target DeepONet is trained for $10\small{,}000$ iterations. When compared to the initial training of the network (from scratch), fine-tuning is faster. Once the target model is trained, we employ target DeepONet with the iterative solver on Target HINTS for the target domain.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=12cm]{media/compare_transfers.png}
    \caption{Convergence of the direct application of HINTS (left) and the transfer learning HINTS (right), done for the L-shaped Darcy problem with a circular cutout. The top figures show the error norm convergence and the bottom figures show the error norms of specific modes.}
    \label{fig:transfer_with_modes}
\end{figure}

Finally, a comparative study is carried out based on the number of iterations each approach takes to converge to machine precision. We randomly select a sample from the target dataset and compare the convergence of the error (both the norm of the error and the mode errors) over iterations, between the direct application of HINTS and the transfer learning HINTS. This is shown in \autoref{fig:transfer_with_modes}. In addition, a comparison of the convergence rate that includes the standard GS solver is shown in \autoref{fig:compare_transfer}. We conclude that the HINTS solution is transferable to different domain geometries, and integrating the transfer-learning approach to fine tune the source DeepONet on the target domain results in a $25\%$ faster convergence without any loss of accuracy for this example.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=10cm]{media/comparison.png}
    \caption{Convergence of the different methods for the L-shaped Darcy problem with a circular cutout. Comparison of the convergence of GS (without HINTS, shown in Blue Line), HINTS-GS (Orange Line) and the transfer learning HINTS-GS (Green Line) in terms of error decay.}
    \label{fig:compare_transfer}
\end{figure}

\subsection{Linear elasticity problem}

In the second example, we consider linear elasticity on a square domain under plane-strain conditions. The governing equation for the model is defined as:
\begin{align}
    \label{eqn:elasticity_PDE}
    \begin{cases}
        \epsilon_x(x, y) &= \frac{\partial u_x(x, y)}{\partial x} \\
        \epsilon_y(x, y) &= \frac{\partial u_x(x, y)}{\partial y} \\
        \gamma_{xy}(x, y) &= \frac{\partial u_x(x, y)}{\partial y} + \frac{\partial u_y(x, y)}{\partial x} \\
    \end{cases}
\end{align}
with subscripts $i,j\in\{1,2\}$ refer to the two in-plane directions, $\boldsymbol{u}_i$ is the displacement in the $i$ direction, $\varepsilon_{ij}$ is the strain component measured in $j$ direction due to displacement in $i$ direction, $\sigma_{ij}$ is the stress component, and $f_i$ is the body force in the $i$ direction. The Lamé parameters, $\mu=\frac{E}{2(1+\nu)}$ and $\lambda=\frac{\nu E}{(1+\nu)(1-2\nu)}$ describe the mechanical properties of the material, where $E$ and $\nu$ are the Young's modulus and the Poisson's ratio, respectively. In this example, we consider the square domain, $\Omega^S = [0, 1]\times [0, 1]$ as the source. The discretization of the domain is presented in \autoref{fig:square_domains}(a).

\begin{figure}[!ht]
     \centering
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/rect.png}
         \caption{Square mesh ($\Omega^S$).}
     \end{subfigure}
     \hfill
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/rect_circle.png}
         \caption{Square mesh with a circular hole ($\Omega^{T3}$).}
     \end{subfigure}
     \hfill
     \caption{Illustration of the meshes used for the elasticity numerical experiments.}
     \label{fig:square_domains}
\end{figure}

\bigbreak
\noindent
\textbf{Employing HINTS to solve the Elasticity problem on source domain}

The goal of the elasticity problem is to learn the operator of the system described by \autoref{eqn:elasticity_PDE}, which maps randomly generated spatially varying modulus of elasticity, $E(\boldsymbol x)$ and randomly varying force vector, $f(\boldsymbol{x})$ to the displacement vector, $\boldsymbol u (\boldsymbol x)$. In this example, we follow the same data generation approach as discussed in \autoref{subsec:darcy}, and the model was trained with $N_S = 85\small{,}000$ samples and tested with $N_S^{test} = 15\small{,}000$ samples. The branch net inputs two channels $(E(\boldsymbol{x})$ and $f(\boldsymbol x))$, and we adopt the same architecture for the convolution modules as discussed in the previous example. The fully-connected network following the convolution modules has a dimension $[256, 160]$. The dimension of the trunk network is $[2, 128, 128, 160]$. The network is trained for $25\small{,}000$ epochs to achieve roughly $4\%$  relative error, indicating a sufficiently well trained network. Now, we employ the HINTS algorithm to solve the elasticity problem on the source domain.

\textbf{Domain adaptation for the elasticity problem}

For investigating the domain adaptation capabilities, we define the following task:
\begin{itemize}
    \item \textbf{Task3:} From a square domain to a square domain with a circular cutout. The target domain is: $\Omega^{T3} = \Omega^S \backslash \{(x, y) | (x - 0.5) ^ 2 + (y - 0.5) ^ 2 \leq 0.15\}$.
\end{itemize}
The discretization of the target domains is shown in \autoref{fig:square_domains}. To begin with, we first employ the source DeepONet to infer on the target domain. In this scenario, no additional training is carried out on the target model. The HINTS algorithm is employed on the target domain for the iterative solver and use the source DeepONet to approximate the solution of displacement. The results presented in \autoref{fig:HINTS_elasticity_no_transfer}(a) show that using HINTS as is we can converge to machine precision after $205$ iterations on the target domain.

As discussed in the previous example, we now integrate the operator level transfer learning algorithm with the HINTS model. In this setup, to train the target model, we generate samples of the input function by appending zeros to the function values with the circular cutout. The target model is trained with $N_T = 100$ samples, where the model is initialized with the optimized parameters of the source model and while training all the layers except the fully-connected layers of the branch network and the last layer of the trunk network are frozen. The fine tuned target DeepONet is replaced in the HINTS algorithm to generate the solution for the target domain. The results obtained using transfer learning integrate HINTS are presented in \autoref{fig:HINTS_elasticity_transfer}(b). In this setup, we observe convergence to machine precision after $169$ iterations, a $22.5\%$ improvement over the previous setup of employing HINTS with the source DeepONet to approximate solution for the target domain.

\begin{figure}[!ht]
     \centering
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/elasticity_no_transfer.png}
         \caption{HINTS (direct application) trained on $\Omega^S$, solving on $\hat{\Omega}^S_{circle}$.}
         \label{fig:HINTS_elasticity_no_transfer}
     \end{subfigure}
     \hfill
     \begin{subfigure}[t]{0.49\textwidth}
         \centering
         \includegraphics[width=\textwidth]{media/elasticity_transfer.png}
         \caption{HINTS (transfer learning), solving on $\hat{\Omega}^S_{circle}$.}
         \label{fig:HINTS_elasticity_transfer}
     \end{subfigure}
     \hfill
     \caption{Results obtained using the direct application of HINTS (a) and transfer learning HINTS (b) for an example from the target data-set of the elasticity problem on the square domain with a circular cutout.}
     \label{fig:HINTS_elasticity}
\end{figure}