\section{Introduction }
\label{intro}

Majorana zero modes (MZMs) in topological superconductors~\cite{m1,m2,m4,m5,m6} have generated significant interest because of their potential
utility in topological quantum computation~\cite{nayakrmp}. For this
purpose, proximity-induced topological superconductors based on
semiconductors~\cite{fu-kane,*sau,*lutchyn,*oreg} have
been considered particularly convenient, because of the large tunability
resulting from the conventional nature of the constituents.
However, quasi-one-dimensional (quasi-1D) topological superconductors with the
potential for harboring multiple MZMs, while not ideal for quantum
computation applications, are interesting systems in their own right.
According to the classification table for topological systems~\cite{kitaev-classi,*Schnyder}, one-dimensional (1D) superconductors
can support Kramers pairs of Majoranas or multiple Majoranas,
where the systems are time-reversal symmetric (class DIII) \cite{kane-mele}
or chiral symmetric (class BDI) \cite{fidkowski,ipsita-sudip,Mandal2015,*Mandal2016a,*Mandal2016b}, respectively.
While proximity effect in wide semiconductor nanowires can lead to
multiple Majoranas in class BDI for the appropriate spin-orbit coupling~\cite{Tewari_2011}, class DIII Majorana Kramers pairs are found to require
interactions to generate from spin-singlet proximity effect~\cite{jelena,*ips-kramers}.
Multiple MZMs have turned out to be particularly interesting because of
novel phenomena that can result from their interplay with interactions.
The most direct addition of interactions in this case was shown to modify
the $Z$ invariant to $Z_8$~\cite{fidkowski}. Recently, more interesting physics has been shown to arise from the interplay of such multiple MZMs with random interactions in the form of the Sachdev-Ye-Kitaev (SYK) model~\cite{sy,*syk-kitaev,*marcel}. From a more pedestrian standpoint, details of experimental manifestations, such as quantization
of conductance or degeneracy of Josephson spectra, are expected to be
more intricate for systems with multiple MZMs as compared to the ones
with single MZMs (that have dominated experimental attention so far).
Specifically, it has been shown~\cite{beenakker} that the conductance into a
wire in the BDI symmetry class takes values that are integer multiples of the
quantum of conductance. Furthermore, perturbations that reduce the
symmetry to class D, also reduce the conductance to the characteristic single
quantum of conductance or vanishing conductance, associated with the
$Z_2$ topological invariant.

\begin{figure}
\includegraphics[scale=0.1]{general_system}
\caption{\label{gs}
Schematic representation of the generic system treated in Sec.~\ref{tomo}. The strength of the tunelling barrier is fixed at $\tau=5\,t_x$, where $t_x$ is the hopping along the chain direction. The superconducting system, as well as the leads, are quasi-1D in nature ($t_y << t_x$).}
\end{figure}

Quasi-1D superconductors, that may be viewed  as
weakly coupled 1D chains \cite{Kaladzhyan_2017} (as shown in Fig.~\ref{gs}), have been suggested in several potential spin-triplet superconductors, such as lithium
purple bronze~\cite{Mercure_12,Lebed_13}, Bechgaard salts~\cite{Lebed_00,Lee_2001,Lee_2003,Shinagawa_2007}, and even possibly SrRuO$_4$
~\cite{raghu}. Given the evidence for spin-triplet pairing in these systems,
in the form of high upper critical fields, these materials have been conjectured to
host MZMs at the ends~\cite{Yakovenko}. Their quasi-1D
structure, composed of many chains coupled by weak transverse
hopping, suggests the possibility of one MZM from each of the chains.
Recent work shows~\cite{Dumitrescu13} that in the cases of time-reversal
(TR) invariant superconductivity in the form of equal spin pairing (ESP),
these MZMs would not split, leading to the possibility of multiple MZMs
at the ends of these materials. Furthermore, spin-triplet superconductors can support
low-energy end-modes even in the absence of external perturbations such as
magnetic fields. This is different from topological superconductivity in
semiconductor systems~\cite{*sau,*lutchyn,*oreg}, which despite being
topologically equivalent to $p-$wave superconductors in certain limits,
cannot realize time-reversal invariant phases without magnetic fields.
 The flexibility of p-wave systems to realize highly symmetric MZMs, allows for topological superconductivity with a high degree
of symmetry. As shown earlier~\cite{Dumitrescu13}, the stability of the multiple
MZMs depends on the variety of symmetries of the systems, and therefore,
in principle, can be split by a variety of perturbations.

In this paper, we study the effect of various symmetry-breaking
perturbations on experimentally measurable signatures of the quasi-1D
spin-triplet superconductors, such as transport and Josephson response~\cite{yukio1,yukio2,jorge1,jorge2}.
In the first part of the paper (Sec.~\ref{tomo}), we numerically compute the zero and non-zero temperature conductances of quasi-1D nanowires, that host multiple MZMs
in the configuration depicted in Fig.~\ref{gs}. For the purposes of reference, we start by reviewing the results~\cite{beenakker} on the conductance of the quasi-1D s-wave Rashba nanowire, with parameters chosen such that the system is in the BDI symmetry class. In this case, we study how the conductance into the wire, as a function of density, tracks the bandstructure and the topological invariant -- it is shown to decouple into single nanowires with modified chemical potentials that belong to the BDI class. Following
this (Sec.~\ref{pwave}), we shift to the main focus, i.e., spin-triplet superconductors.
We extend the class of perturbations previously considered~\cite{Dumitrescu13},
and start with a model with mirror, chiral, and TR symmetries. We systematically break these symmetries by changing various spin-orbit coupling
terms that maybe controlled by gate voltages and magnetic fields. We study how
the conductance tracks the topological invariant and spectrum in all these cases.
In the second part of the paper (Sec.~\ref{cavity}), we study the effect of the symmetry-induced spectrum-breaking on the
Andreev spectra of Josephson junctions~\cite{Dumitrescu13}. Recent
measurements have demonstrated the ability to measure aspects of Andreev
state spectrum in a Josephson junction by two-tone spectroscopy~\cite{urbina,Devoret}. Similar to the case of conductance, we show that the spectrum shows
multiple zero-energy Andreev states in the highly symmetric case with mirror and
chiral symmetries. We study how applying gate voltages to the junction to change the symmetry of the spin-orbit coupling as well as magnetic fields, splits these states in the junction.

\section{Differential conductance with normal leads}
\label{tomo}

In this section, we analyze the behaviour of the differential conductance, that can be detected using normal leads connected to the first lattice sites of the system, as shown in Fig.~\ref{gs}. Let $t_{x}$ be the hopping strength between the neighboring sites in the same chain, and $t_{y}$ be the hopping strength between the neighboring sites in the neighboring chains. We consider the limit $t_y\ll t _x$ in order to model a quasi-1D chain.
The leads are modeled as having only hopping (of strength $t_x$) and chemical potential ($\mu_j$) terms corresponding to the single chains, and they are represented by the Hamiltonian
	\begin{align}
H_{leads} & =
-t_{x}  \sum_{i=1}^{N_\ell-1} \sum_{j=1}^{N_y}
	 {\Phi}^{\dagger}_{i+1, j} \,\tau_{z}\, {\Phi}_{i,j}
\nonumber \\ &
\hspace{1 cm}
-  \left(  \tau \,t_{x} \sum_{j=1}^{N_y}
	 {\Phi}^{\dagger}_{1, j} \,\tau_{z}\, {\psi}_{1,j} +  {\rm h.c.}  \right) .
\label{eqlead}
	 \end{align}
Here, $\tau $ is the strength of the tunnelling barrier in units of $t_x$, and is set at $5$ for the systems we analyze. Furthermore, ${\Phi}^{\dagger}_{i,j}= (d^{\dagger}_{i,j,\uparrow},d^{\dagger}_{i,j,\downarrow},d_{i,j,\downarrow},- d_{i,j,\uparrow})$ and
${\psi}^{\dagger}_{\ell,j}= (c^{\dagger}_{\ell,j,\uparrow},c^{\dagger}_{\ell,j,\downarrow},c_{\ell,j,\downarrow},- c_{\ell,j,\uparrow})$ are the spinors belonging to the lead and chain sites, respectively.
The site-indices $(\ell,j)$ label the fermions in the $(x,y)$ strip, such that $\ell \in [1,N_x]$ and $j \in [1,N_y]$.
Lastly, $ {\sigma}_\alpha $ and $ {\tau}_\alpha $  ($ \alpha \in \lbrace x,y,z \rbrace $) are the Pauli matrices which act on the spin and particle-hole spaces, respectively . In our numerics, we have taken the number of sites in each chain (lead)
to be  $N_x \, (N_\ell)  =  100$, while the number of chains is set to $N_y =3$, corresponding to a three-channel lead. Energy eigenvalues, voltages, and all the parameters in the Hamiltonian are expressed in units of $t_x$. The conductance is calculated in units of $\frac{e^2}{h}$.

 The zero temperature ($T=0$) conductance $G_0(V)$ has been computed using the KWANT package \cite{Groth_2014}, that uses the scattering matrix formalism. These results are extended to  non-zero temperature ($T>0$) conductance using the convolution:
	\begin{equation}
	G_T (V) = \int dE\, \frac{df_{\epsilon}(T)}{d\epsilon} \Big \vert_{(E-V)} G_0(E)\,,
\label{GT}
	\end{equation}
	where $f_{\epsilon} (T)=\frac{1}{e^{\epsilon/kT}+1}$ is the Fermi function at temperature $T$ and energy $\epsilon$, whose derivative with respect to energy is carried out at the value $(E-V)$, energy being in units of electron volts.

\subsection{Rashba nanowire}
\label{swave}

\begin{figure*}[]
{\includegraphics[width = 0.5 \textwidth]{F1a}} \qquad
{\includegraphics[width = 0.35\textwidth,height = 0.35\textwidth]{F1b}}
{\includegraphics[width = 0.35\textwidth,height = 0.35\textwidth]{F1c}} \qquad
{\includegraphics[width = 0.35\textwidth,height = 0.35\textwidth]{F1d}}
\caption{\label{1dnano}
We shows the results for the quasi-1D Rashba nanowire described by Eq.~\ref{nanoham}, with $\Delta_s=0.4,\, V_x=0.8,\,\alpha=0.25, \,,t_y=0.1 $. The first panel (including (a), (b), and (c)) shows the correspondence between the spectrum, the invariant, and the zero-bias differential conductance, all plotted as functions of the chemical potential $\mu$. The remaining three panels (i.e., (d), (e), anf (f)) show the differential conductance as a function of voltage, for $\mu=-0.1,\,0.725,$ and $0.8$ respectively, at different temperature values (as shown in the plot-legends).
All the temperature values are in units of $t_x$.}
\end{figure*}

We consider a multi-channel 1D Rashba nanowire aligned along the $x$-axis, brought in contact with an s-wave superconductor, in the presence of an external magnetic field of strength $V_x$ applied in the $x$-direction. This can be modeled as an array of 1D chains coupled by a weak hopping amplitude, with the Hamiltonian written below:
\begin{align}
\label{nanoham}
H = & \sum_{\ell=1}^{N_x}\sum_{j=1}^{N_y} {\psi}^{\dagger}_{\ell,j}
\left [
\left \lbrace -\mu+2 \left (t_x+t_y \right ) \right \rbrace \tau_{z}
+\Delta_s \tau_{x}+V_{x} \sigma_{x} \right ] {\psi}_{\ell,j}
\nonumber \\
&	- \sum_{\ell=1}^{N_x-1} \sum_{j=1}^{N_y}
	\left \{ {\psi}^{\dagger}_{\ell+1, j}
\left (t_{x}+ \mathrm{i}\,\alpha\, \sigma_{y}\right )\tau_{z}\, {\psi}_{\ell,j}+{\rm h.c.}  \right\}
\nonumber \\	&
- \sum_{\ell=1}^{N_x} \sum_{j=1}^{N_y-1}
	\left ( {\psi}^{\dagger}_{\ell, j+1} \,t_{y}\,\tau_{z}\, {\psi}_{\ell,j}
+{\rm h.c.} \right) .
\end{align}
Here, $\mu$ is the chemical potential, $\Delta _s$ is the magnitude of the s-wave superconducting gap, and $\alpha $ is the spin-orbit coupling.
The system has a chiral symmetry operator $\mathcal{S} = \sigma_y\, \tau_y$, which can used to off-block diagonalize the Hamiltonian~\cite{tewsau} to the form $\begin{pmatrix} 0 & A(k) \\ A^{T}(-k) & 0\end{pmatrix}$. The topological invariant is calculated as~\cite{tewsau}:
\begin{equation} \label{e1}
\mathcal{Z} = \frac{1}{2 \,\pi \, \mathrm{i}}\,\int_{k=-\pi}^{k = \pi}d\theta(k) \,,\quad
\theta(k)= \frac{\det \left (A(k) \right)}{|\det \left (A(k) \right )|}\,.
\end{equation}
More details of this calculation are presented in Appendix~\ref{app}.

The topological behaviour of the quasi-1D Rashba wire described by
Eq.~\ref{nanoham} can be understood from the evolution of the spectrum of
the nanowire with a Zeeman field, as shown in Fig.~\ref{1dnano}(a).
The gap-closures seen in this spectrum indicate the presence of a sequence of six
topological phase transitions. In addition, for chemical potential in the range $0.75\lesssim\mu\lesssim1.5$, one can see Andreev bound states \cite{DEGENNES,PhysRevB.86.100503,PhysRevB.86.180503,PhysRevB.97.165302,PhysRevB.96.075161,PhysRevB.98.155314,Vuik_2019} as states that have ``peeled off'' below the continuum of states. These states are similar to those obtained in single channel nanowires in the absence of any lead quantum dot, and have an energy that approaches zero at the phase transitions~\cite{PhysRevB.86.100503,PhysRevB.86.180503,PhysRevB.97.165302,PhysRevB.96.075161,PhysRevB.98.155314,Vuik_2019}.

The topological phase transitions seen in Fig.~\ref{1dnano}(a) are not accompanied by a change in symmetry. Rather, they correspond to changes in the topological invariant that is calculated using Eq.~\ref{e1}, and plotted in Fig.~\ref{1dnano}(b). We see that the topological invariant changes between consecutive integer values. As the chemical potential crosses each phase transition, we observe the corresponding gap closures in (a), as expected for topological phase transitions. The range of integer values (i.e., from zero to three) for the topological invariant in this system can be understood from the fact that the normal state (i.e., $\Delta_s=0$) of the isolated system can be decomposed into a sequence of three sub-bands with different wave-function profiles in the $y$ direction. Adding superconductivity does not couple these bands, and hence the Hamiltonian in Eq.~\ref{nanoham} describes a stack of decoupled topological nanowires, with their normal state bands shifted relative to each other. The applied Zeeman field splits the spin components of each of these sub-bands in a way such that they are topological in a range of chemical potentials, where one of these spin-split bands is occupied. Because the Zeeman splitting of the electrons for $V_x=2$
is larger than this separation between the various sub-bands,
changing chemical potential can sequentially drive all of them into a topological phase,
before the lowest sub-band gets both its spins occupied. This leads to a situation where
the number of topological bands can increase to three before decreasing back to zero
\cite{PhysRevB.84.144522}. While the results for the topological invariant in this sub-section can be understood from the sub-band decomposition of the Hamiltonian in Eq.~\ref{nanoham}, in later sub-sections we will find that the presence of chiral symmetry protects the topological invariant from small perturbations that couple the sub-bands.

While the topological invariant, plotted in Fig.~\ref{1dnano}(b), is not directly measureable in experiments, one can see the evidence for this invariant in the zero-bias conductance, plotted in Fig.~\ref{1dnano}(c).
We observe that the zero temperature zero-bias conductance tracks the integer topological invariant in (b) quite closely. The quantization of the conductance here represents the topological invariant, despite the fact that the tunneling Hamiltonian from the lead (Eq.~\ref{eqlead}) couples the different channels, which we used in the previous paragraph to understand the integer values. Thus, measuring the zero-bias conductance at low-enough temperatures can provide insights into the topological phase diagram of such a wire.
Furthermore, the ABS states do not appear to affect the value of the zero-bias conductance peak (ZBCP) for the parameters of our calculation. Fig.~\ref{1dnano}(c) tracks the change of the conductance with a rise in temperatures. We find that the conductance values are lowered as the temperature is increased. However, even at $T =4.3 \times 10^{-3}\,t_x$, the phase transition points can be identified from the conductance plots, though the conductance is significantly reduced from the correct quantized value.

The thermal-suppression of the zero-bias conductance, as seen in the Fig.~\ref{1dnano}(c), can be understood by considering the conductance as a function of voltage for different temperatures. One sees from Fig.~\ref{1dnano}(d) (plotted for $\mu=0.1$) that the temperature-suppression arises from a broadening of the zero-bias peak with temperature. The conductance as a function of voltage at two other values of $\mu$ (viz. $\mu=0.725$ and $0.8$), plotted in Fig.~\ref{1dnano}(e) and ref{1dnano}(f), shows a similar thermal-suppression.
Interestingly, this thermal-suppression effect appears to be weaker in the case of the smaller conductance peaks, that are associated with fewer MZMs. This is accompanied by narrower zero-bias peak widths for the case of larger number of MZMs. These observations suggest that the extra MZMs, that occur in the case of larger number of modes, are coupled to the lead with a weaker tunneling amplitude.

This behaviour is found in all the cases examined in this paper. The $T=0$ plot in Fig.~\ref{1dnano}(e) shows sharp peaks away from the zero voltage, which are associated with the bulk states that are quantized by finite size effects. The width of these resonances are suppressed because of the weak tunneling
of these states across the tunnel barrier. These narrow peaks associated with the
sub-gap states are washed away at higher temperatures.
Furthermore, Fig.~\ref{1dnano}(f) shows additional broad peaks, away from zero-energy but below the superconducting gap. These peaks are associated with finite energy ABSs, the evidence of which is seen in the spectrum in Fig.~\ref{1dnano}(a).
The ABSs that result from splitting of the MZMs, are localized near the end of the wire, and therefore couple strongly to the leads resulting in larger broadening compared to the sub-gap states. As expected, these extra states do not change the zero-bias conductance, which is controlled by the topological invariant.

\subsection{p-wave superconductors}
\label{pwave}

We will now consider the main focus of this work, i.e., TR-invariant topological superconductors~\cite{Dumitrescu13} that can be realized by spin-triplet pairing, exhibiting equal spin pairing (ESP) $p$-wave superconductivity. These properties are conjectured to be present in the quasi-1D transition metal oxide Li$_{0.9}$Mo$_6$O$_{17}$, and some organic superconductors~\cite{limo1,triplet1,triplet2,Lebed_00,Mercure_12,Lebed_13,Lee_2001,
Lee_2003,Shinagawa_2007}. The hopping integrals along the crystallographic directions of these materials vary as, $t_x \gg t_y \gg t_z$, making them quasi-1D conductors.

 The triplet ($S=1$) superconductivity of the p-wave wire can be represented by a matrix pair
potentital $\Delta_{\alpha\beta}(k)=\Delta\left[\textbf{d}(k)\cdot \bm{\sigma} \right ] _{\alpha\beta}$, where
$k=k_x$ is the 1D crystal-momentum.
The nature of the triplet component characterized by the vector $\mathbf d$ is odd (i.e. $\textbf{d}(k)=-\textbf{d}(-k)$), while $\Delta$ represents the magnitude.
Here, we choose $\mathbf d(k) =d(  k)\left(0,0,1\right )$, i.e., along
the $z$-direction in the spin space. The superconducting term in real space is then of the form: $ \mathrm{i}\,\Delta \left ( c_{ \ell+1,\uparrow}^{\dagger}\, c_{ \ell,\uparrow}^{\dagger}\,+\,c_{\ell+1,\downarrow}^{\dagger}\,
c_{ \ell,\downarrow}^{\dagger} \right )+ {\rm h.c.}$
This choice of the $\bm d$-vector represents a TR-invariant superconductor containing the ESP spin-triplet $p$-wave state.

The bulk Hamiltonian for a p-wave superconducting chain, with the order parameter described
in the previous paragraph, can be written in the Nambu basis (defined with the spinor $\Psi_{k}=(c_{k,\uparrow},c_{k,\downarrow},c_{-k,\downarrow}^{\dagger},-c_{-k,\uparrow}^{\dagger})^{T}$) as:
\begin{align}
\label{eq:H1DK}
&	{\cal{H}}^{1D}_{k}(\mu,\Delta, \mathbf V_Z,\alpha_R) \nonumber \\
& = \left [\, \epsilon(k)-\mu+2\,t_x\,   \right ]\sigma_{0}\tau_z + \Delta\, \sin k \,\sigma_z\, \tau_{x}
 + {\cal{H}}^{Z}\,+ {\cal{H}}^{SO}\,.
\end{align}
Here, $\epsilon(k)=-2\,t_x \cos k $ is the single-particle kinetic energy,
and $ \Delta\,\sin k$ is the $p$-wave superconducting order parameter.
In addition to p-wave superconductivity, the above Hamiltonian allows us to consider the
effect of an electric-field-induced inversion-symmetry breaking spin-orbit interaction (SOI) term aligned in an arbitrary direction ${\mathbf{a}}$ in the spin space. This is written as
\begin{align}{\cal{H}}^{SO}=\alpha_R \,\sin k
\left ({\mathbf{a}} \cdot \bm{\sigma}\right ) \tau_z\,.
\end{align}
The Hamiltonian also allows us to consider the breaking of TR symmetry ,
through the Zeeman effect of an applied magnetic field, that is
captured by the term
\begin{align}
{\cal{H}}^{Z}= \left( \bm{V} \cdot \bm{\sigma} \right ) \tau_0 \,.
\end{align}

The single-chain p-wave Hamiltonian described above can be generalized
to model a quasi-1D system that is more relevant to the experimental materials (such as Li$_{0.9}$Mo$_6$O$_{17}$~\cite{Lebed_00,Mercure_12,Lebed_13,Lee_2001,
Lee_2003,Shinagawa_2007}), by coupling
multiple copies of Eq.~\ref{eq:H1DK} into a multi-channel Hamiltonian that is written as:
\begin{align}
  H^{Q1D}_{k;jj'} = & \,{\cal{H}}^{1\text{D}}_{k,j} \delta_{j, j'}+{\cal{H}}^{y}_{j,j'} \,,
\nonumber \\
{\cal{H}}^{1\text{D}}_{k,j}
= & \, {\cal{H}}^{1D}_{k}
\left (\mu_j,\Delta_j,\mathbf V^j,\alpha_{R}^j \right ).
\label{eq:HQ1D}
\end{align}
The different chains in this quasi-1D bundle are coupled together
with an amplitude $t_y$, given by the y-directional Hamiltonian
\begin{align}
 {\cal{H}}^{y}_{j,j'}
= &- t_y \, \tau_z   \left (\delta_{j,j'+1}+\delta_{j,j'-1} \right )
 \nonumber \\ & \quad
+ \mathrm{i}\,\alpha_R'\, \sigma_y \,
\tau_z \left (\delta_{j,j'+1}-\delta_{j,j'-1} \right ),
\end{align}
 where $\alpha_R'$ represents the magnitude of the inter-chain Rashba SOI.
As we saw in the last subsection, coupling identical chains leads to an artificial decoupling
of sub-bands, that can lead to non-generic results. For this reason, we have introduced
a $j$-dependence of the parameters of the single chains $\mathcal{O}_j=\mu_j,\Delta_j,\mathbf V^j,\alpha_{R}^j$, which is assumed to be of the form
\begin{align}
&\mathcal{O}_j=\bar{\mathcal{O}}\left (1-\tilde{j}\, \gamma \right ) ,
\end{align}
where $\bar{\mathcal{O}}$ is the average value of the parameters, $\gamma=0.1$, and $\tilde{j} = j-2$.

Combining all the ingredients discussed in this subsection so far, the total Hamiltonian in Eq.~\ref{eq:HQ1D} can be explicitly written out in the position space as:
\begin{widetext}
\begin{equation}
\label{nanoham2}
\begin{split}
H =
&\sum_{\ell=1}^{N_x}
\sum_{j=1}^{N_y}
{\psi}^{\dagger}_{\ell,j}
\left [  \left \lbrace 2(t_x+t_y) -\mu_{j} \right \rbrace \tau_{z}
+ \mathbf{V}^j\cdot \bm{\sigma} \right ] {\psi}_{\ell,j}
- \sum_{\ell=1}^{N_x-1} \sum_{j=1}^{N_y}
\left [ {\psi}^{\dagger}_{\ell+1, j}
\left \lbrace t_{x}\, \tau_{z}\,
+ \frac{ \mathrm{i}\, \Delta_j}{2}\, \sigma_z\, \tau_x +
\frac{ \mathrm{i}\, \alpha_{R}^{j}}{2}
\left (\mathbf{a}\cdot \bm{\sigma}  \right ) \tau_z \right \rbrace
{\psi}_{\ell,j}+{\rm h.c.}  \right ]
\nonumber \\ &- \sum_{\ell=1}^{N_x} \sum_{j=1}^{N_y-1}
\left \{ {\psi}^{\dagger}_{\ell, j+1}
\left  ( \,t_{y}\,\tau_{z}\, -\mathrm{i}\,\alpha_R'\,\sigma_y\,\tau_z
\right  ){\psi}_{\ell,j} +{\rm h.c.} \right \} .
\end{split}
\end{equation}
\end{widetext}

The above Hamiltonian, in addition to obeying particle-hole symmetry $\mathcal{P}=\sigma_y\tau_y K$ (and taking $k\rightarrow -k$) that applies to any superconductor, obeys
a TR symmetry for $\bm{V}=0$. The corresponding TR operator is $\mathcal{T}= \mathrm{i}\,\sigma_y\, K$ (plus performing the operation $k\rightarrow -k$), $K$ here denotes complex conjugation. In addition, depending on the presence or absence of spin-conservation (the latter caused by SOI), or inversion symmetry, the system can possess various
mirror or chiral symmetries. This makes the p-wave system particularly interesting because
it allows, in principle, turning the various symmetries on or off. by applying electric and magnetic fields, while all the while remaining in the topological phase.

We now study the signatures of the symmetry breaking phenomena in this topological p-wave
superconductor in the subsubsections below. Specifically, we will compute the results for the spectrum, topological invariant, and conductance, similar to what we reviewed in the more familiar and simpler case of the Rashba nanowire in the previous subsection (illustrated in Fig.~\ref{1dnano}).
For our numerics, we will choose the superconducting pairing amplitude to be $\Delta=0.5$ (in units where $t_x=1$), in addition to other parameters specified in the relevant subsubsection.

\subsubsection{Time-reversal and chiral symmetric case: $\mathbf V =\alpha_R'=0$}

The p-wave system of Eq.~\ref{eq:HQ1D}, with the restriction $\bm{V} =\alpha_R' = 0 $, is in the BDI symmetry class, similar to the Rashba nanowire studied in Sec.~\ref{swave}. This has a chiral symmetry operator $ \mathcal{S} = \sigma_z\, \tau_y$, which is different though, compared to the Rashba nanowire case.

The spectrum of the model as a function of $\mu$ is shown in Fig.~\ref{F2}(a). This illustrates a sequence of bulk-gap closings, together with a set of zero-energy states,
similar to the Rashba nanowire case of Sec.~\ref{swave}. However, in this case, we do
not see significant sub-gap ABSs associated with the ends. The evolution
of the spectrum can again be understood in terms of changing of the filling of sub-bands.
Unlike the Rashba nanowire case, both spin components of a sub-band with p-wave
filling are topological, with the same sign of the topological invariant. This can be understood from the chiral topological invariant, which is calculated analogous to the Rashba nanowire case, and is plotted in Fig.~\ref{F2}(b). From this plot, we see that the topological invariant jumps from zero to two, as $\mu$ increases from $\mu\sim 0$. This regime corresponds to both spin components of the lowest sub-band starting to fill. The fact that the topological invariant increases from zero to two indicates that both spin components of the sub-band contribute the same value to the topological invariant, which is different from the case of the Rashba nanowire.
The difference can be understood in the simpler case with $\alpha_R=0$, where
the Hamiltonian and the chiral symmetry $\mathcal{S}$ commute with the mirror symmetry operator $M = \mathrm{i} \left (\hat{\mathbf d}
\cdot \boldmath{\sigma} \right )\tau_0=\sigma_z\tau_0$. $M$ can thus be used to define a mirror-invariant as well~\cite{tudor15}. Thus a chiral topological invariant can be computed for each $\sigma_z=\pm 1$. Furthermore, the two sectors $\sigma_z=\pm 1$ can be mapped into
each other by $\sigma_x$, that commutes with $\mathcal{S}$. This explains why the two sectors have the same topological invariant. The inclusion of non-zero $\alpha_R$ does not break the chiral symmetry (though it breaks the mirror symmetry), and therefore it cannot change the topological invariant. In fact, numerical results for $\alpha_R=0$ (not included here) appear qualitatively identical to Fig.~\ref{F2}.
The topological invariant begins to decreases at $\mu \gtrsim 4$, as the Fermi level crosses the tops of the bands.

The topological invariant indicates the number of MZMs that appear at each end
of the system, which appear as zero-energy states in the spectrum shown in Fig.~\ref{F2}(a).
The even-parity of the topological invariant can be understood to be a consequence of
the TR symmetry, that constrains the MZMs to appear in Kramers pairs. The
even number of MZMs appear as zero-bias conductance plotted in Fig.~\ref{F2}(c). Similar
to the case of the Rashba nanowire, the zero-bias conductance closely tracks the
number of MZMs, and provides a measurable indication of the change of the
topological invariant (that cannot be measured directly). As in the case of the Rashba nanowire, the temperature dependence of the zero-bias conductance provides a sense of the energy scale with which the MZMs couple to the leads. As before, we find that the higher conductance peaks are more sensitive to temperature.

\begin{figure}[htb]
	\includegraphics[width = 0.4 \textwidth]{F3}
\caption{\label{F2}
For quasi-1D p-wave superconductors of Eq.~\ref{eq:HQ1D}, descibed by the parameters $ t_y=0.1,\,\alpha_R =0.25, \, \alpha_R'=0.0,\,\mathbf V =0$, and ${\mathbf{a}}\,=\,(0,0,1)$, the figure shows the correspondence between the spectrum, the BDI invariant (associated with the chiral symmetry operator $\sigma_z\,\tau_{y}$), and the
zero-bias differential conductance, all of which have been plotted as functions of the   chemical potential $\mu$. All the temperature values are in units of $t_x$.}
\end{figure}

\begin{figure}[htb]
{\includegraphics[width = 0.4 \textwidth]{F4}}
\caption{\label{F3}For quasi-1D p-wave superconductors of Eq.~\ref{eq:HQ1D}, descibed by the parameters $ t_y=0.1,\,\alpha_R =0.25, \, \alpha_R'=0$, ${\mathbf{a}}=(0,0,1)$, and $\mathbf V= (0,0.2,0)$, the figure shows the correspondence between the spectrum, the BDI invariant (associated with the chiral symmetry operator $\sigma_{z}\,\tau_{y}$), and the zero-bias differential conductance, all of which have been plotted as functions of the chemical potential $\mu$. All the temperature values are in units of $t_x$.}
\end{figure}

\subsubsection{Time-reversal broken chiral symmetric case:
$|{\mathbf{a}} \times {\hat{z}}|=\alpha_R'=0$}

The TR symmetry of the system, discussed in the previous sub-subsection,
can be broken by applying a Zeeman field $\mathbf V= (0,0.2,0)$.
The nanowire, however, still has chiral symmetry, encoded by the operator $\mathcal{S}=\sigma_z\tau_y$. The breaking of the TR symmetry splits the Kramer's degeneracy of the bulk states near the phase transitions seen in Fig.~\ref{F3}, such that the three phase transitions for $\mu<1$ (seen in Fig.~\ref{F2}(a)) are now split into six phase transitions
(see Fig.~\ref{F3}(a)). For the parameters chosen, the splitting of some of the higher chemical potential ($\mu>3$) transitions are too small to resolve. As seen
in Fig.~\ref{F3}(b), these split-transitions are indeed topological phase transitions, as
they are accompanied by a change of the topological invariant. The topological
invariant, which is identical to the one calculated in the last sub-subsection,
shows integer jumps for $\mu<1$, as opposed to the double jumps seen in the
TR symmetric case discussed in the last sub-subsection.

The MZMs that appear at the end because of the non-trivial topological invariant are
no longer required to be Kramers degenerate. This allows both even and odd
number of MZMs. This is apparent from the numerical result for the zero bias
conductance shown in Fig.~\ref{F3}(c), where we see that many of the conductance
steps that are multiples of $4e^2/h$ split into steps with height $2e^2/h$
associated with individual MZMs. Unfortunately, many of the smallest steps are washed out
at the lowest temperature of $T\sim 10\,K$.

\begin{figure}[]
\includegraphics[width = 0.4 \textwidth]{F5}
\caption{\label{F4} For quasi-1D p-wave superconductors of Eq.~\ref{eq:HQ1D}, descibed by the parameters $t_y=0.3, \, \alpha_R=0.25$, $ \alpha_R^\prime=0 $, ${\mathbf{a}}\,=\,(1,1,0)$, and $\mathbf V= (0,0.1,0.1)$, the figure shows the correspondence between the spectrum, the Pfaffian invariant (as the system belongs to class D), and the zero-bias differential conductance, all of which have been plotted as functions of the chemical potential $\mu$.
All the temperature values are in units of $t_x$.}
\end{figure}

\subsubsection{Superconductors without symmetry}
\label{classD}

Changing the electric field symmetry of the system such that either the intra-chain
Rashba coupling picks up a non-zero component (causing $|\mathbf {a}\times \hat{z}|\neq 0$), and/or generates an inter-chaing $\alpha_R'\neq 0$, breaks all the chiral symmetries of the system. This places the system in the symmetry class D~\cite{kitaev-classi,*Schnyder}, which is the minimal symmetry for a superconductor, showing only particle-hole symmetry.  As seen from the spectrum plotted in Fig.~\ref{F4}(a), this leads
to a structure of bulk-gap closings that is similar to Fig.~\ref{F3}(a), in the sense
of showing six phase transitions at low chemical potential values ($\mu<2$). In this case, we also find six phase transitions above $\mu>2$.

However, unlike Fig.~\ref{F3}(a), the zero-energy states in Fig.~\ref{F3}(a) both appear or disappear at the subsequent phase transitions. This is consistent  with the fact
that the topological invariant for class D is in the group $Z_2$, and therefore it takes only two values given by \cite{Kitaev_2001} $Q =\frac{1-\nu}{2}$,
where
\begin{align}
& \nu = \text{sgn} \left[ Pf(Q_1) \,Pf(Q_2) \right ],\nonumber \\
& Q_1 = \mathcal{H}^{Q1D}(k)  \Big \vert_{k=0} \sigma_y\, \tau_y \,,\nonumber \\
& Q_2 = \mathcal{H}^{Q1D}(k) \Big \vert_{k=\pi}
\sigma_y\, \tau_y \,.
\end{align}
Here, $Pf(A)$ represents the Pfaffian of the matrix $A$ \cite{pf}.
The resulting topological invariant $Q$ is plotted in Fig.~\ref{F4}(b), which shows
that the system alternates between a topological phase (with one MZM at each end), and
a trivial phase. Most of the trivial phases, seen in Fig.~\ref{F4}(b), correspond to
the chemical potential range of Fig.~\ref{F4}(b) with an even value of the chiral topological invariant. These regions in the spectrum of Fig.~\ref{F4}(a) contain sub-gap states, that result from splitting the even number of MZMs by the breaking of chiral symmetry with SOI.

As in the earlier cases, the plot of the zero-bias conductance shown in Fig.~\ref{F4}(c)
indicates the topological invariant. The topological region with invariant $Q=1$
shows a conductance of $2e^2/h$, corresponding to a single MZM at each end of the wire.
The zero-bias conductance in this case shows a stronger suppression with temperature compared to its chiral symmetric counterpart in Fig.~\ref{F3}(c). This indicates that the residual MZM is the one that couples weakest to the lead.

\begin{figure}[]
\includegraphics[width = 0.4 \textwidth]{F6}
\caption{\label{F5}For the quasi-1D p-wave superconductors of Eq.~\ref{eq:HQ1D}, descibed by the parameters $t_y=0.3, \alpha_R=0.25$, $ \alpha_R^\prime=0.1 $, ${\mathbf{a}}\,=\,(1,1,0)$, and $\mathbf{V}= 0$, the figure shows the correspondence between the spectrum, the DIII invariant ($Q$), and the zero-bias differential conductance, all of which have been plotted as functions of the chemical potential $\mu$.
All the temperature values are in units of $t_x$.}
\end{figure}

\subsubsection{Time-reversal preserving case}
\label{sectr}

Applying an electric field to generate an inter-chain Rashba SOI ($\alpha_R'\neq 0$),
as well as an intra-chain SOI ($\alpha_R\neq 0$), in the absence of a Zeeman field,
breaks all chiral and spin symmetries without breaking TR symmetry.
This leads to a superconductor in the symmetry class DIII. The symmetry class
of the system becomes apparent from the spectrum shown in
Fig.~\ref{F5}(a), which exhibits three phase transitions for $\mu\lesssim 1$, similar to
the spectrum of the TR symmetric superconductor shown in
Fig.~\ref{F3}(a). However, the chiral symmetry present in the case of Fig.~\ref{F3}(a), is broken here by the applied electric field. As a result,
the MZMs disappear at alternate phase transition points,
similar to the spectrum of the class D nanowires shown in Fig.~\ref{F4}(a).
Similar to the case of Sec.~\ref{classD}, the alternating presence of
MZMs can be understood from the topological invariant for this case, which can take
two values (see Fig.~\ref{F4}(b)). As in the other cases, the topological
invariant only changes at the phase transitions.
The topological invariant for the symmetry class DIII, shown in Fig.~\ref{F5}(b), is calculated as $Q=\frac{1-\nu}{2}$~\cite{DIII}, where
\begin{equation}
\nu = \det \left({\mathcal{U}}^K \right )\,\frac{Pf \left(\hat{\theta} (0) \right )}
{ Pf \left (\hat{\theta} (\pi) \right )}\,.
\end{equation}
Here, $\hat{\theta} (0)$ and $\hat{\theta} (\pi)$ represent the matrix elements of
the TR operator $\mathcal{T}$, in the basis of the occupied states,
at $k=0$ and $k = \pi$, respectively. The matrix ${\mathcal{U}}^K $ in this basis
is given by the so-called Kato Propagator~\cite{DIII}:
\begin{align}
{\mathcal{U}}^K(0,\pi) = \lim_{n\rightarrow \infty} \,\prod
\limits_{\lambda=0}^{n} \, \mathcal{P}_{o}(k_\lambda)\,,
\end{align}
where $\mathcal{P}_{o}(k_\lambda)$ is the projector into the occupied bands (negative energy) and $k_\lambda = \frac{ \lambda \, \pi}{n}$.

The zero-bias conductance as a function
of $\mu$, which is plotted in Fig.~\ref{F5}(c), provides a measurable indication
of the topological invariant. Similar to the previous sub-subsection, this correspondence
is apparent from the fact that the zero-bias conductance takes two values in the
tunneling limit, corresponding to the two values of the topological invariant.
The conductance in the topologically non-trivial regime takes a value $4e^2/h$
corresponding to two MZMs at each end. The doubled value of the quantized
conductance, in contrast with the case in Sec.~\ref{classD}, is a
result of the Kramers degeneracy associated with TR symmetry.
Relative to the spectrum of the multiple Kramers pairs of MZMs shown in
Fig.~\ref{F2}(a) (for the chiral and TR symmetric p-wave superconductor),
Kramers pairs of ABSs or MZMs seen in the spectrum in Fig.~\ref{F5}(a).
Thus the DIII class topological invariant and the structure of ABSs in Fig.~\ref{F5} can be understood from the chiral symmetric topological invariant at the corresponding
values of chemical potential in Fig.~\ref{F2}(b).

\section{Signatures of multiple MZMs in Andreev spectroscopy}
\label{cavity}

In this section, we focus on the effect of the symmetry-breaking on the ABS spectra of Josephson junctions (JJs) of superconducting nanowires.
A JJ is created by introducing a weak link with tunneling amplitude
$\gamma$, which has negligible conductance compared to the rest of the superconducting wire.
We can therefore assume that the supercurrent has a negligible contribution to the superconducting phase drop around the wire, which can be controlled by a flux-loop~\cite{felix}.
The ABS spectrum of a JJ generates features in the ac absorption, that can be measured by several techniques such as microwave-spectroscopy~\cite{van_Woerkom} and two-tone spectroscopy~\cite{urbina,Devoret}.

Following the argument in the last paragraph, the superconducting phase
difference $\phi$ across the JJ, generated by the magnetic flux-loops, is introduced through a modified Hamiltonian corresponding to Eq.~\ref{eq:HQ1D}:
\begin{widetext}
 \begin{align}
\label{nanoham3}
	H = & \sum_{\ell=1}^{N_x}\sum_{j=1}^{N_y} {\psi}^{\dagger}_{\ell,j}\left ((-\mu+2(t_x+t_y)) \,\tau_{z}+\Delta_s\, \tau_{x}^{\ell}+V_{x} \,\sigma_{x} \right ){\psi}_{\ell,j}
- \sum_{\ell=1}^{N_x-1} \sum_{j=1}^{N_y}
	\left \{ {\psi}^{\dagger}_{\ell+1, j}
\left (t_{x}+  \mathrm{i}\,\alpha\, \sigma_{y}\right )\tau_{z}\, {\psi}_{\ell,j}+{\rm h.c.}  \right\}
	\nonumber \\	&
	- \sum_{\ell=1}^{N_x} \sum_{j=1}^{N_y-1}
\left ( {\psi}^{\dagger}_{\ell, j+1} \,t_{y}\,\tau_{z}\, {\psi}_{\ell,j} +{\rm h.c.} \right ) - \gamma \sum_{j=1}^{N_y}
\left ( {\psi}^{\dagger}_{N_x, j} \,\tau_{z}\, {\psi}_{1,j}+{\rm h.c.} \right )\,,
\end{align}
\end{widetext}
where $\tau_x^{\ell} =\begin{pmatrix} 0 &
e^{\mathrm{i}\, \ell\,\phi/N_x}
\\ e^{-\mathrm{i}\, \ell\,\phi/N_x}
& 0\end{pmatrix}$. Furthermore, the phase difference $\phi$ across the JJ is controlled by the magnetic flux $\Phi$ in the superconducting loop through the relation
$\phi=2\,\pi\,\Phi/\Phi_0$, where $\Phi_0= \frac{h\,c} {2\,e}$ is the superconducting flux quantum.
The $\ell$ dependence of the matrix $\tau_x^{\ell}$ can be understood as a winding of the superconducting phase around the loop, which is needed to ensure that the superconducting phase difference between the ends $\ell=1$ and $\ell=N_x$ of the JJ is equal to $\Phi$.

\begin{figure}[]
\includegraphics[width =0.4 \textwidth]{Rf}
\caption{\label{sring}
ABS spectrum of a quasi-1D Rashba nanowire, with the same parameters as in Fig.~\ref{1dnano}, as a function of the flux across a Josephson junction. The ends of the multi-chain ring has a weak coupling strength of $\gamma=0.1$.
The three panels have different values of the chemical potential:
$\mu=0.1$ (top), $\mu= -0.52 $ (middle), and $\mu = 0.4$ (bottom). The spectrum shows degeneracies of $6$, $4$, and $2$, at flux $\Phi=\Phi_0/2$. These correspond to $3$, $2$, and $1$ MZM(s) at each end of the nanowire in the open boundary case, as seen in Fig.~\ref{1dnano}.}
\end{figure}

The ABS spectrum of the above Hamiltonian is plotted in Fig.~\ref{sring} for
several chemical potential values, and it shows zero-energy crossings at
$\frac{\phi}{2\,\pi} = 1/2$. The class BDI topological invariant for
the bulk Hamiltonian, with the same parameters as used to plot the spectrum in Fig.~\ref{1dnano}, shows clearly that the number of zero-energy ABSs at
$\frac{\phi}{2\,\pi}=1/2$, for the different chemical potentials, is twice the topological invariant. This can be understood by noting that, prior to introducing the tunnel coupling (i.e., at $\gamma=0$), the topological invariant in the BDI symmetry class equals the number of MZMs at each end of the JJ. Introducing the tunnel coupling $\gamma$ across the JJ splits the pairs of MZMs into ABSs with energies that are typically non-zero, except at $\phi=\pi$.

We can understand the above argument more explicitly by applying a gauge transformation, $U=e^{ \frac{\mathrm{i}\,\ell \,\phi\,\tau_z}{2\,N_x}}$, to each site $\ell$ of the lattice. This eliminates the $\ell$ dependence of the SC phase in $\tau_x^\ell$, in favour of introducing a phase $e^{\mathrm{i}\,\phi\,\tau_z/2}$ into the tunneling
term proportional to $\gamma$ (in Eq.~\ref{nanoham3}). Then this term becomes
proportional to $\mathrm{i}\,\gamma$ for phase $\phi=\pi$. Since this term commutes with the
chirality operator $\mathcal{S}=\sigma_y\,\tau_y$ (see Sec.~\ref{swave}), and
the MZMs for the BDI class are eigenstates of $\mathcal{S}$ with eigenvalues $\mathcal{S}=\pm 1$ in the absence of tunneling (i.e., $\gamma=0$), these MZMs cannot be hybridized by the coupling $\gamma$ at phase $\phi=\pi$. As a result, they appear as zero-energy ABSs, as shown in Fig.~\ref{sring}.

We now analyze the p-wave system treated earlier in Sec.~\ref{pwave}. Introducing a JJ into the system with a phase difference $\phi$ (similar to Eq.~\ref{nanoham3}), leads to the modified Hamiltonian:
\begin{widetext}
\begin{align}
\label{nanoham4}
H =  &\sum_{\ell=1}^{N_x}
	\sum \limits_{j=1}^{N_y}
	{\psi}^{\dagger}_{\ell,j} \left[
	\left \lbrace (-\mu_j +2 \left (t_x+t_y \right ) \right \rbrace
\tau_{z} + \mathbf{V}^j\cdot\mathbf{\sigma} \right ] {\psi}_{\ell,j}
- \sum_{\ell=1}^{N_x-1} \sum_{j=1}^{N_y}
	\left [ {\psi}^{\dagger}_{\ell+1, j}\left \lbrace t_{x}\,\tau_{z}
+ \frac{\mathrm{i}\, \Delta_j}{2} \,\sigma_z \, \tau_x^{\ell}
+ \frac{  \mathrm{i}\, \alpha_{R}^{j}}{2}\left (\mathbf{a}\cdot\mathbf{\sigma} \right )
\tau_z \right \rbrace {\psi}_{\ell,j}+{\rm h.c.}  \right ] \nonumber \\
&- \sum_{\ell=1}^{N_x} \sum_{j=1}^{N_y-1}
	\left \{ {\psi}^{\dagger}_{\ell, j+1}
	\left( t_{y}\,\tau_{z}\, -\mathrm{i}\,\alpha_R'\,\sigma_y\tau_z \right)
	{\psi}_{\ell,j} +{\rm h.c.} \right \} - \gamma \sum_{j=1}^{N_y}
\left ( {\psi}^{\dagger}_{N_x, j} \,\tau_{z}\, {\psi}_{1,j}+{\rm h.c.} \right ) ,
	\end{align}
\end{widetext}
with  $\tau_x^{\ell}$ having the same meaning as in Eq.~\ref{nanoham3}.
We consider the same parameters as in Sec.~\ref{sectr}.
Similar to the case of the s-wave superconductor with Rashba SOI
studied in Fig.~\ref{sring}, the tunnel coupling proportional to $\gamma$ splits
the end MZMs into finite-energy ABSs, as seen in Fig.~\ref{pring}. However, unlike the s-wave case, we see the ABSs cross zero energy at two points: at $\Phi = \Phi_0/4$ and $3\,\Phi_0/4$.
The number of ABSs merging at zero energy varies with the
chemical potential, as shown in the three different panels of Fig.~\ref{pring}.
Comparison of the values of the chemical potential with the topological invariant in Fig.~\ref{F2}, shows clearly that the number of zero-energy ABSs at $\Phi=\Phi_0/4$ and $ 3\Phi_0/4$ is twice the topological invariant, similar to the case of the Rashba nanowire.
The middle panel also shows low energy ABSs, in addition to the zero-energy
crossings associated with the MZMs.

\begin{figure}[]
\includegraphics[width =0.4\textwidth]{Jf}
\caption{\label{pring}
ABS spectrum of a quasi-1D p-wave superconductor, with the same parameters as in Fig.~\ref{F5}, as a function of the flux across a Josephson junction. The ends of the multi-chain ring has a weak coupling strength of $\gamma=0.1$.
The three panels have different values of the chemical potential:
$\mu=0.0$ (top), $\mu= 2.0$ (middle), and $\mu =5.0$ (bottom). The spectrum shows degeneracies of $0$, $4$, and $4$, at flux $\Phi=\Phi_0/2$. These correspond to $0$, $2$, and $2$ MZMs at each end of the nanowire in the open boundary case, as seen in Fig.~\ref{F5}. The middle panel shows additional states due to the presence of ABSs (also seen in Fig.~\ref{F5}(a)), which also exist in the open boundary case at low energies.}
\end{figure}

The results shown in Fig.~\ref{sring} and Fig.~\ref{pring} for the highly symmetric cases show that the Andreev spectra also show multiple zero-energy ABSs similar to the corresponding conductance plots. Therefore, we expect these results to be somewhat fragile in the sense that breaking the symmetries will lower the number of Andreev crossings (similar to what is seen for ZBCPs).

\section{Discussion and conclusion}
\label{end}

In this paper, we have studied the effect of symmetry-breaking field for
multi-channel p-wave superconductors that have a high degree of symmetry.
These symmetries allow the possibility of topological superconductivity
with integer topological invariants with an integer number of MZMs.
We find that the ZBCP reflects this topological invariant, as both vary
synchronously with the chemical potential. Breaking the symmetries systematically, by applying strain and magnetic fields, leads to reducing the conductance to
lower integer values by splitting some of the MZMs. The integer topological
invariants also appear to manifest as zero-energy crossings of Andreev spectra for the highly symmetric topological superconductors. For the examples we consider, the number of
zero-energy crossings of ABSs corresponds to the topological invariant,
similar to the zero-bias conductance.  The crossings of the Andreev spectra in
topological Josephson junctions is expected to be measurable through
recent advances in Andreev spectrocopy~\cite{Geresdi-Kouwenhoven,tosi-urbina,dev}. The changes in the signatures (such as zero-bias conductance and Andreev spectroscopy) of the topological invariant can elevate the fingerprints of the MZMs to a rich structure, where the observables vary over several quantized values in a multi-dimensional phase space.

While the signatures for integer topological invariants appear to be quite
robust to variations in the Hamiltonian, the numerical examples we considered
so far do not include disorder. How far these predictions hold up to realistic
disorder in these systems will be an interesting direction for future work. In
addition, it will be interesting to see if the zero-bias conductance with normal
leads translates to a robust signature for superconducting leads, as with the
case of non-degenerate Majorana modes~\cite{glazman-oppen}.
Explicit computation of the detection of the ABSs through a
cavity-response experiment is also left for future work.

\section{Acknowledgments}

  We thank Pablo San-Jose for collaboration in the initial stages of the project. We also thank Denis Chevallier for useful discussions. Pfaffians were calculated using a code written by Bas Nijholt.
 ABR acknowledges the computational facilities provided by IIT-KGP, where parts of the work were carried out.
	JS acknowledges support from 	NSF
DMR1555135 (CAREER).


\section{Introduction}
\label{sec:intro}
Face hallucination
(FH) refers to the task of recovering high-resolution (HR) facial images from corresponding low-resolution (LR) inputs~\cite{baker_pami2002, chen2018fsrnet, grm2018face}. Solutions to this task have applications in face-oriented vision problems, such as face editing and alignment, 3D reconstruction or face  attribute estimation~\cite{Super-FAN,chen2018fsrnet, jourabloo2017pose, li2017generative, lin2007super,QLC,roth2016adaptive, yu2018super} and are used to mitigate performance degradations caused by input images of insufficient resolution. One particularly popular use of FH models is for LR face recognition tasks\cite{gunturk2003eigenface,lin2007super, zhao2015heterogeneous}, where LR probe images are super-resolved to reduce the dissimilarity with HR gallery data.
\begin{figure}[t!]
\centering
\begin{minipage}{0.96\columnwidth}
\includegraphics[width=\textwidth]{downsampling_method_comparison.pdf}\vspace{1mm}
\end{minipage}
\begin{minipage}{0.03\columnwidth}
\begin{turn}{270}
  \scriptsize{ MS \hspace{8mm} NMS}
  \end{turn}
\end{minipage}
\begin{minipage}{\columnwidth}
\vspace{-1mm}
\scriptsize{\hspace{0.4mm} LR ($24$ px) \hspace{2mm} URDGN \hspace{3mm} LapSRN \hspace{3.2mm} SRResNet \hspace{3.5mm} CARN \hspace{4.8mm} C-SRIP}
\end{minipage}
\caption{Hallucination examples  ($8\times$)
for the five FH models used in this work (see Sec.~\ref{Sec: Method} for details).
The top row shows results for a LR image generated with a degradation procedure matching (MS) the one used during training and the bottom row shows results for an image produced by a non-matching degradation function (NMS). Note the difference in the reconstruction quality.
In this paper, we study the bias introduced into FH models by the training data, which has so far received limited attention in the literature.
}
\label{fig: sr_bias}\vspace{-3mm}
\end{figure}
Formally, face hallucination is defined as an inverse problem described by the following observation model~\cite{nasrollahi2014super}:
\begin{equation}
\mathbf{x}=\mathbf{H}\mathbf{y}+\mathbf{n},
\label{Eq: Inverse_SR_problem}
\end{equation}
where $\mathbf{x}$ denotes the observed low-resolution face image, $\mathbf{H}$ stands for a composite down-sampling and blurring operator, $\mathbf{n}$ represents an additive i.i.d. Gaussian noise term with standard deviation $\sigma_n$, and $\mathbf{y}$ is the latent high-resolution face image that needs to be recovered~\cite{nasrollahi2014super}.
Recent techniques increasingly approach the FH problem in~\eqref{Eq: Inverse_SR_problem} using machine learning methods~\cite{ahn2018carn,bulat2018learn,lai2017lapsrn,nguyen_SR_2018,yu2018face,  yu2016ultra} and try to learn a direct (non-linear) mapping $f_{\theta}$ from the LR inputs to the desired HR outputs, i.e.,
$f_{\theta}: \mathbf{x} \rightarrow \mathbf{y}.$

This mapping is commonly implemented with a parameterized regression model, e.g., a convolutional neural network (CNN), and the parameters of the model $\theta$ are learned through an optimization procedure that minimizes a selected training objective (e.g., an $L_p$ loss) over a set of corresponding LR-HR image pairs. Because the learning procedure is supervised, the image pairs needed for training are constructed by artificially degrading HR training images using a selected degradation function, i.e., a known operator $\mathbf{H}$ and noise level $\sigma_n$. Such an approach ensures that all generated LR images have corresponding HR ground truth faces available for training, but also implicitly defines the type of image degradations the learned model is able to handle. If the actual degradation function encountered with (real-world) test data differs from the one used during training, the result of the face hallucination model may be far from optimal - as  illustrated in Fig.~\ref{fig: sr_bias} for five recent state-of-the-art
FH models~\cite{ahn2018carn,grm2018face,lai2017lapsrn,ledig2016photo,yu2016ultra}.

As can be seen from the presented examples, the HR images recovered from a LR input that matches the characteristics of the training data (Fig.~\ref{fig: sr_bias}, top row) are of significantly better quality than those produced from a non-matching LR input image (Fig.~\ref{fig: sr_bias}, bottom row). While all models are able to convincingly ($8\times$) upscale the example $24\times 24$  face  with a matching LR image, the hallucination results exhibit considerable artifacts when a small change in terms of blur and noise is introduced into the degradation procedure.
These examples show that the bias introduced into the FH models by the training data has a detrimental effect on the quality of the super-resolved faces and may adversely effect the generalization ability of the trained models to data with unseen characteristics.

Surprisingly, the problem of (face hallucination) model bias has received  little attention from the research community so far. Nevertheless, it has important implications for the generalization abilities of FH models as well as for the performance of high-level vision tasks that rely on the generated hallucination results, most notably face recognition. The existing literature on the generalization abilities of FH techniques is typically focused on generalization across different facial characteristics, such as pose, facial expressions, occlusions or alignment, and less so on the mismatch in the degradation functions used to produce the LR test data or qualitative experiments with real-world imagery. Difficulties with model bias are, therefore, rarely observed. Similarly, when used to improve performance of LR face recognition problems, FH models are predominantly applied on artificially degraded images, leaving the question of generalization to real-world LR data unanswered.

In this paper, we aim to address these issues and study the problem model bias in the field of face hallucination. We try to answer obvious research questions, such as: How do different image characteristics affect the reconstruction quality of FH models? How do FH models trained on artificially degraded images generalize to real-world data? Do FH models ensure improvements in LR face recognition when applied as a preprocessing step? Are there differences in recognition performance when using either artificially generated or real-world LR data? To answer these and related questions we conduct a rigorous analysis using five recent state-of-the-art FH models and examine in detail: \textit{i)} the mismatch between the degradation procedure used to generate the LR-HR training pairs and the actual degradation function encountered with LR data, \textit{ii)} changes in classifier-independent separability measures before and after the application of FH models, and \textit{iii)} face recognition performance with hallucinated images and a state-of-the-art CNN recognition model. We make interesting findings that point to open and rarely addressed problems in the area of face hallucination
and provide insights into future research challenges in this area.

\section{Related Work}

\textbf{Bias in computer vision.} Machine learning techniques are known to be sensitive to the characteristics of the training data and typically result in models with sub-optimal generalization abilities if the training data
is biased towards certain data characteristics.
The
effect of dataset bias can, for example, be seen in \cite{buolamwini2018gender}, where commercial gender classification systems are shown to have a drop in gender-classification accuracy on darker-skinned subjects compared to lighter-skinned subjects, indicating insufficient training data coverage of the latter. Torralba and Efros~\cite{torralba2011unbiased} dem\-onstrate that image datasets used to train classification models  are heavily biased towards specific appearances of object categories, causing poor performance in cross-dataset experiments.
Zhao et al. \cite{zhao2017men} show that datasets for semantic role labeling tasks, contain significant gender bias and introduce strong associations between gender labels and  verbs/objects (e.g., \textit{woman} and \textit{cooking}) that lead to biased models for certain labeling tasks.
These examples show that understanding dataset bias is paramount for the generalization abilities of machine learning models.
Our work is related to these studies, as we also explore dataset bias. However, different from prior work, we focus on the task of face hallucination, which
has not been studied from this perspective so far.

\textbf{Face hallucination for face recognition.} Face recognition performance with LR images tends to degrade severely in comparison to HR face data.
To mitigate this problem, a significant body of work resorts to FH models and tries to up-sample images during pre-processing~\cite{farrugia2017face,gunturk2003eigenface, lin2007super,su2016supervised}
or to devise models that jointly learn an upscaling function and recognition procedure~\cite{hennings2008simultaneous,jian2015simultaneous, wu2016deep}.
While performance improvements are reported with these works, experiments are commonly limited to artificially down-sampled images, findings are then simply extrapolated to real-world data and potential issues due to dataset bias are often overlooked.

Experiments with real LR images, on the other hand, are scarce in the literature and the usefulness of FH models for face recognition with real-world LR imagery has not received much attention by the research community. As part of our analysis, we study this issue and explore the effect on FH models on data separability and recognition performance on artificially down-sampled and real-world LR data.

\section{Methodology}\label{Sec: Method}

\subsection{Experimental setup}

We conduct our analysis with several  state-of-the-art FH models and LR images of size $24\times 24$ pixels. Since there is no clear distinction on what constitutes a LR image, we select the LR image data to be smaller than $32\times 32$ pixels, which represents an image size, below which most computer vision models are known to deteriorate quickly in performance~\cite{grm2017strengths, torralba200880,wang2016studying}. Given this rather small size, we use an upscaling factor of $8\times$  with the FH models and generate $192\times 192$ images that are  used as the basis for our analysis.

\subsection{Face hallucination (FH) models}

Using the presented setup, we study the effect of dataset bias using five recent FH (or super-resolution) models, i.e.: the Ultra Resolving Discriminative Generative Network (URDGN,~\cite{yu2016ultra}), the Deep Laplacian Super-Resolution Network (LapSRN,~\cite{lai2017lapsrn}),   the Super-Resolution Residual Network (SRResNet,~\cite{ledig2016photo}), the Cascading Residual Network (CARN,~\cite{ahn2018carn}), and the Cascading Super Resolution Network with  Identity Priors (C-SRIP,~\cite{grm2018face}). The selected models differ in the network architecture and training objective, but are all considered to produce state-of-the-art hallucination results as shown in Fig.~\ref{fig: sr_bias}. We also include an interpolation-based method in the experiments to have a baseline for comparisons. A short summary of the models is given below:
\begin{itemize}[leftmargin=*]\vspace{-0.5mm}
    \item \textbf{Bicubic interpolation}~\cite{bicubic} is a learning-free approach that up-samples images by interpolating missing pixel values using Lagrange polynomials, cubic splines, or other similar functions. Unlike  FH models, it does not rely on domain knowledge when generating HR faces. \vspace{-1mm}
    \item \textbf{URDGN} consists of a generator and a discriminator network, and is trained using the generative adversarial network (GAN~\cite{goodfellow2014generative}) framework, where the discriminator is trained to tell apart real and generated HR images, whereas the generator is trained to minimize an $L_2$ reconstruction loss and the accuracy of the discriminator.\vspace{-1mm}
   \item \textbf{LapSRN} represents a CNN-based  model that progressively up-samples LR images by factors of $2$ through bilinear deconvolution and relies on a feature prediction branch to calculate the high-frequency residuals at each scale. Because of the progressive up-sampling, multi-scale supervision signals are used during training. \vspace{-1mm}
   \item \textbf{SRResNet} is a variant of the SRGAN~\cite{ledig2016photo} model that incorporates many of the recent tweaks used in CNN-based super-resolution, such as adversarial training, pixel shuffle up-sampling, batch normalization and leaky ReLU activations. SRResNet represents the generator network of SRGAN trained with the $L_2$ loss. \vspace{-1mm}
  \item \textbf{CARN} consists of a light-weight CNN, which is able to achieve state-of-the-art performance for the general super-resolution problems using an efficient cascading architecture that combines the design principles of densely connected networks~\cite{huang2017densely} and res nets~\cite{He_2016_CVPR}. We use the variant with local and global cascading connections, as opposed to the lighter variants of the network.\vspace{-1mm}
  \item \textbf{C-SRIP} is a CNN-based FH model that incorporates explicit face identity constraints into the training procedure in addition to the main reconstruction objective. The model has a cascaded architecture that allows it to use supervision signals at multiple scales during training.
\end{itemize}

To incorporate face-specific domain knowledge into the models and ensure a fair comparison, we train all models on the CASIA Webface~\cite{yi2014learning} dataset using $494,414$ images of $10,575$ subjects. We crop the $192\times 192$ central part of the images and generate the HR-LR data pairs for training by  blurring the HR images with a Gaussian kernel of $\sigma_b=\frac{8}{3}$ and then downscaling them $8\times$ using bicubic interpolation.

\begin{figure*}[!tb]
\begin{minipage}{0.02\textwidth}
 \end{minipage}
 \hfill
\begin{minipage}{0.29\textwidth}
  \centering

  \includegraphics[width=1\textwidth]{downsampling_model_comparison_lr.pdf}\vspace{0.1mm}
  \text{\footnotesize (a) LR inputs ($\sigma_n$ vs. $\sigma_b$)}
 \end{minipage}
 \hfill
 \begin{minipage}{0.29\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{downsampling_model_comparison_bicubic.jpg}\vspace{0.1mm}
  \text{\footnotesize (b) Bicubic ($\sigma_n$ vs. $\sigma_b$)}
 \end{minipage}
 \hfill
  \begin{minipage}{0.29\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{downsampling_model_comparison_C-SRIP.jpg}\vspace{0.1mm}
  \text{\footnotesize (f) C-SRIP ($\sigma_n$ vs. $\sigma_b$)}
 \end{minipage}
 \hfill
 \begin{minipage}{0.02\textwidth}
 \end{minipage}\vspace{2mm}
\caption{Reconstruction capabilities of the learning-free bicubic interpolation a selected FH model. The image block on the left (with samples of size $24 \times 24$ pixels) illustrates the effect of increasing noise ($\sigma_n$, increases vertically) and blur ($\sigma_b$, increases horizontally) for a sample LR LFW image, the second and third block show $192 \times 192$ reconstructions generated by bicubic interpolation and C-SRIP, respectively. Images marked green are generated with a degradation function matching the one used during training. For the FH model good HR reconstructions are achieved only with images degraded similarly as the training data, whereas interpolation ensures reasonable reconstructions with all input images. Results for the remaining FH models are shown in the Appendix. Best viewed zoomed in.}\vspace{0mm}

\label{fig:SR_grids_noise}
\end{figure*}

\begin{figure*}[!tb]
 \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_bicubic.pdf}\vspace{-2mm}
  \text{\footnotesize (a) Bicubic}
 \end{minipage}
 \hfill
  \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_URDGN.pdf}\vspace{-2mm}
  \text{\footnotesize (b) URDGN}
 \end{minipage}
 \hfill
  \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_LapSRN.pdf}\vspace{-2mm}
  \text{\footnotesize (c) LapSRN}
 \end{minipage}
 \hfill
  \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_CARN.pdf}\vspace{-2mm}
  \text{\footnotesize (d) CARN}
 \end{minipage}
 \hfill
   \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_srresnet.pdf}\vspace{-2mm}
  \text{\footnotesize (d) SRResNet}
 \end{minipage}
 \hfill
   \begin{minipage}{0.162\textwidth}
  \centering
  \includegraphics[width=1\textwidth,trim=6mm 75mm 6mm 6mm, clip]{reconstruction_grid_C-SRIP.pdf}\vspace{-1mm}
  \text{\footnotesize (d) C-SRIP}
 \end{minipage}\vspace{2mm}
\caption{Reconstruction capabilities with mismatching degradation functions due to different blur and noise levels. The heat maps show the average SSIM values computed over artificially degraded LFW images. The points marked in the heat maps correspond to the sampled levels of noise ($\sigma_n$, increases vertically) and blur  ($\sigma_b$, increases horizontally). The value of $\sigma_n$ and $\sigma_b$ that was used for training is marked green. Note that all FH models achieve good reconstructions only around values that match the training setup. Best viewed in color.}\vspace{-1.5mm}
\label{fig:heat_maps}
\end{figure*}

\subsection{Datasets.} \label{subsec:datasets}

We conduct experiments on the Labeled Face in the Wild (LFW~\cite{huang2007labeled}) and SCFace~\cite{grgic2011scface} datasets. We introduce artificial down-sampling to simulate  low image resolutions with LFW and use the SCFace images to explore the effect of training data bias on real-world LR images.
\begin{itemize}[leftmargin=*]\vspace{-0.5mm}
    \item \textbf{LFW} is one of the most popular face dataset available, mainly due to the unconstrained settings in which the images were captured. The dataset~\cite{huang2007labeled} consists of $13,233$ face images of size $250\times 250$ pixels belonging to $5749$ subjects. For the experiments, we use only the central crop of the images to have faces of similar proportion to the ones used during FH model training. \vspace{-1mm
    \item \textbf{SCface} contains images of $130$  subjects that are split between a gallery set, containing $130$ high-resolution frontal mug\-shots ($1$ per subject), and a larger probe set of surveillance-camera images. The daylight camera set, which we use for our experiments, consists of images from $5$ different security cameras.
Each subject is recorded by each camera at $3$ different distances, resulting in a total of $130\times 5\times 3 = 1950$ probe set images. We crop facial areas from all images based on the provided facial landmarks prior to the experiments.
\end{itemize}

\subsection{Bias exploration with synthetic LR data}

We start our analysis by exploring the sensitivity of FH models to a controlled mismatch in the degradation function. We first crop the ($192\times 192$) central part of the LFW images and generate baseline LR test data using the same degradation function as used during training. To simulate the mismatch, we generate additional sets of LR data from LFW by varying the standard deviations of the Gaussian blurring kernel $\sigma_b$ and Gaussian noise term $\sigma_n$, which define $\mathbf{H}$ and $\mathbf{n}$ in~\eqref{Eq: Inverse_SR_problem}. We consider five different values for each parameter and select $\sigma_b$ from $[0.75, 1.5, 2.25, 3, 3.75]$ and $\sigma_n$ from $[0, 5, 10, 15, 20]$. Because the LR test data is generated artificially, the HR ground truth can be used to evaluate the reconstruction capabilities of the FH models for each combination of $\sigma_b$ and $\sigma_n$. Note that it is in general infeasible to include all possible data variations in the training procedure, so there will always be image characteristics that have not been accounted for by data augmentation. The selected noise and blur levels are therefore as reasonable factors as any to simulate the mismatch.

From the hallucination examples in Fig.~\ref{fig:SR_grids_noise} we see that visually convincing results for the FH model are produced only for LR images generated with blur and noise levels similar to those used during training (close to the images marked green), and deteriorate quickly as the difference to the training blur and noise levels gets larger (see Appendix for additional results). The interpolation baseline produces less convincing results compared to the best hallucinated image of C-SRIP, but  does also introduces lesser distortions with images of other blur and noise levels. A similar observation can also be made for the remaining FH models based on the results in Fig.~\ref{fig:heat_maps}, where average structural similarity (SSIM) values computed over the entire LFW dataset are shown for different levels of noise and blur. Here, the computed SSIM scores are shown in the form of interpolated heat maps for all five FH models and the baseline (bicubic) interpolation procedure.
The first thing to notice is that the degradation in reconstruction quality is also visible for the (learning-free) interpolation method. This suggests that the reconstruction problem gets harder with increased noise and blur levels
and the worsened reconstruction quality is not linked exclusively to the mismatch in the degradation function. However, the heat maps also clearly show that performance degrades much faster for the FH models than for the interpolation approach and that the degradation is particularly extreme for the C-SRIP model, which otherwise results in the highest peak SSIM score among all models.

In general, all FH models achieve significantly higher SSIM scores with matching degradation functions (see  green point in Fig.~\ref{fig:heat_maps}) than the interpolation approach, but their performance falls below bicubic interpolation at the highest noise and blur levels - see lowest heat map part in Fig.~\ref{fig:heat_maps}.
This is an important finding and implies that for imaging conditions that are difficult to model  and challenging to reproduce using \eqref{Eq: Inverse_SR_problem}, interpolation may still be a better choice for recovering HR faces than FH models, which require representative HR-LR image pairs for training.

The presented results are consistent with recent studies~\cite{stutz2018disentangling,su2018robustness}, which suggest that the performance of CNN models may come at the expense of robustness and that trying to learn models that are more robust to varying imaging conditions leads to less accurate results. We observe similar behaviour with the tested FH models (compare the heat maps of C-SRIP and URDGN, for example) and hypothesize that the relatively narrow focus of the models on specific degradation functions may be one of the reasons for the convincing performance of recent CNN-based FH models.

\subsection{Bias exploration with synthetic and real data}

Next, we explore the impact of dataset bias with synthetic LR images from LFW and with real-world surveillance data from SCFace, where the observed image degradations due to the acquisition hardware are not well modelled by the training degradation function. Since there is no HR ground truth available for the SCFace data, measuring the reconstruction quality is not possible with this dataset. We therefore focus on face recognition, which is regularly advocated in the literature as one of the main applications for FH models~\cite{farrugia2017face,gunturk2003eigenface,lin2007super,su2016supervised}, and use it as a proxy for face hallucination performance.
Because this task is different from the reconstruction task studied above, we first run experiments with artificially degraded LFW images  to have a baseline for later comparisons with results obtained on real-world SCFace data. We note that recognition experiments add another dimension to our analysis, as we now also explore the impact of the dataset bias on the semantic content of the reconstructed HR data and not only on the perceived quality of the hallucinated faces.

For the experiments, we use a ResNet-$101$ model~\cite{He_2016_CVPR} and train it for face recognition on a dataset of close to $1.8$ images and $2622$ identities~\cite{VGGface}. We select the mdel because of its state-of-the-art performance~\cite{masi2016we,ranjan2017hyperface} and that fact that an open-source implementation is readily available. We perform network surgery on the trained ResNet-$101$ and use the activations from the penultimate network layer as a $512$-dimensional descriptor of the input face images.

\begin{figure}[t!]
\centering\vspace{-0.7mm}
\begin{minipage}{0.51\columnwidth}
\centering
  \includegraphics[width=\textwidth]{lr_samples.pdf
\end{minipage}
\begin{minipage}{0.48\columnwidth}
\centering
  \includegraphics[width=\textwidth]{dist1_size_hist.pdf
\end{minipage
 \caption{Examples of LR LFW and SCFace images used in the experiments. Left: the first row shows LFW samples degraded using the \textit{matching} scheme (MS), the next row shows LFW images degraded with the \textit{non-matching} scheme (NMS) and the last row shows images from SCFace. Right: distribution of SCFace image widths/heights (in $px$) for faces captured at the largest distance.\vspace{-2.5mm}}
 \label{fig:face_samples_scface_lfw}
\end{figure}
\begin{figure*}
\centering\vspace{-0.5mm}
 \begin{minipage}{0.239\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_lfw_192px_hr_.pdf}\vspace{0.3mm}
  \text{\footnotesize (a) HR images}
 \end{minipage}
  \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_optimal_downsampling_bicubic.pdf}
  \includegraphics[width=\textwidth]{tsne_adverse_downsampling_bicubic.pdf}\vspace{0.3mm}
  \text{\footnotesize (b) Bicubic}
 \end{minipage}
   \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_optimal_downsampling_URDGN.pdf}
  \includegraphics[width=\textwidth]{tsne_adverse_downsampling_URDGN.pdf}\vspace{0.3mm}
  \text{\footnotesize (c) URDGN}
 \end{minipage}
   \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_optimal_downsampling_LapSRN.pdf}
  \includegraphics[width=\textwidth]{tsne_adverse_downsampling_LapSRN.pdf}\vspace{0.3mm}
  \text{\footnotesize (d) LapSRN}
 \end{minipage}
   \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_optimal_downsampling_CARN.pdf}
  \includegraphics[width=\textwidth]{tsne_adverse_downsampling_CARN.pdf}\vspace{0.3mm}
  \text{\footnotesize (e) CARN}
 \end{minipage}
   \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=\textwidth]{tsne_optimal_downsampling_srresnet.pdf}
  \includegraphics[width=\textwidth]{tsne_adverse_downsampling_srresnet.pdf}\vspace{0.3mm}
  \text{\footnotesize (f) SRResNet}
 \end{minipage}
   \begin{minipage}{0.118\textwidth}\centering
  \includegraphics[width=1\textwidth]{tsne_optimal_downsampling_C-SRIP.pdf}
  \includegraphics[width=1\textwidth]{tsne_adverse_downsampling_C-SRIP.pdf}\vspace{0.3mm}
  \text{\footnotesize (g) C-SRIP}
 \end{minipage}\vspace{2.2mm}
 \begin{minipage}{0.015\textwidth}

  \begin{turn}{270}
  \footnotesize{\hspace{-5mm} MS \hspace{14mm} NMS}
  \end{turn}
 \end{minipage}
 \caption{Visualization of ResNet-$101$ features extracted from hallucinated HR images using t-SNE~\cite{maaten2008visualizing}. Results are shown for the $10$ largest classes of  LFW. The plots show distributions for: (a) the original HR images, and (b-g) hallucinated HR face images images down-sampled using the matching (MS) or non-matching (NMS) degradation schemes. Best viewed in color and zoomed in.\vspace{-1.6mm}}
 \label{fig:tsne}
\end{figure*}

For the experiments with artificially down-sampled LFW data, we consider two different degradation schemes:
\begin{itemize}[leftmargin=*]\vspace{-0.5mm}
\item \textbf{A matching scheme (MS),} where each full-resolution LFW image is first filtered with a Gaussian kernel of $\sigma_b= 10$ and
the generated image is then decimated to the target size using bicubic interpolation. No noise is added. This scheme matches the training setup. \vspace{-1mm
\item \textbf{A non-matching scheme (NMS)}, where $\sigma_b$ is selected randomly from a uniform distribution, i.e., $\mathcal{U}\left(0.5, 4\right)$, for each LFW image.
After filtering and down-sampling, images are corrupted through additive Gaussian noise with standard deviation $\sigma_n$, drawn randomly from $\mathcal{U}(0, 20)$. This ensures a mismatch between the applied degradation function and the one used during training. Furthermore, it results in a different degradation for every test image.\vspace{-0.5mm}
\end{itemize}
The two schemes generate $24\times 24$ LR data of size and different characteristics as shown in Fig.~\ref{fig:face_samples_scface_lfw}. The generated images are then fed to the FH models for up-sampling and the HR results are used as inputs for ResNet-$101$.

For the experiments with the SCFace data, we use a subset of $650$ images captured by the five surveillance cameras at the largest of all recorded distances. After removing the interlaced rows from the images as well as a corresponding number of columns to ensure a correct aspect-ratio, we end up with images, where the facial area covers an image region close in size to the $24\times 24$ pixels expected by the FH models - a distribution for the SCFace face widths/heights is shown on the right of Fig.~\ref{fig:face_samples_scface_lfw}. We rescale  all images to the correct input size (using bicubic interpolation) and then feed the hallucination results produced by the FH models to ResNet-$101$ for descriptor computation.

\textbf{Experiments on data separability.} Using the experimental setup described above, we explore whether data separability is improved when facial details are hallucinated
how the separability is affected by the mismatch in the degradation function. To this end, we visualize the distribution of ResNet-$101$ feature descriptors extracted from hallucinated HR images of the $10$ largest LFW classes (i.e., the $10$ subjects with the highest number of images) using t-SNE~\cite{maaten2008visualizing} in Fig.~\ref{fig:tsne}. In order to quantitatively evaluate the separability of the presented distributions, we also compute a separability measure in the form of the Kullback-Leibler (KL) divergence between the distribution of a given class and joint distribution of all remaining classes in the 2D t-SNE embedding space and report average values calculated over all $10$ considered LFW classes in Table~\ref{tab: KL divergence}.

We observe that the for the original HR images (before down-sampling) the classes are well separated and show no overlap. After down-sampling with the matching scheme (MS) and subsequent up-sampling (top row in Fig.~\ref{fig:tsne}), we see considerable overlap in the class distributions for bicubic interpolation. The FH models, on the other hand, improve the data separability over the interpolation-based baseline and result in significantly higher KL-divergence scores. C-SRIP performs particularly well and generates compact class clusters with very little overlap.
\begin{table}[!tb]
\renewcommand{\arraystretch}{1.05}
\caption{Average KL divergence for the $10$ largest LFW classes with the MS and NMS degradation schemes estimated in the 2D space generated by t-SNE. Arrows indicate an increase or decrease in value compared to the baseline bicubic interpolation method.\vspace{1mm}}
\label{tab: KL divergence}
\centering
\footnotesize
\begin{tabular}{ l  cccc}
\hline
\multirow{2}{*}{Approach}  & & \multicolumn{3}{c}{LFW}   \\\cline{3-5}
 && MS & NMS & Change \\ \hline
Bicubic (baseline) & &$0.5389$ &  $0.2135$ & $-0.3254$   \\
URDGN   & &$0.5561$ $\blu{\uparrow}$ & $0.2143$ $\blu{\uparrow}$ & $-0.3418$  \\
LapSRN   && $0.6346$ $\blu{\uparrow}$ & $0.2087$ $\red{\downarrow}$ & $-0.4259$ \\
CARN   & &$0.6851$ $\blu{\uparrow}$ & $0.1957$ $\red{\downarrow}$ & $-0.4894$ \\
SRResNet  & & $0.7148$ $\blu{\uparrow}$ & $0.1962$ $\red{\downarrow}$ & $-0.5222$  \\
C-SRIP   && $0.7676$ $\blu{\uparrow}$ & $0.1972$ $\red{\downarrow}$ & $-0.5704$  \\ \hline
\end{tabular}\vspace{-3mm}
\end{table}

With the non-matching scheme (NMS) all mo\-dels perform noticeably worse, as shown in the bottom row of Fig.~\ref{fig:tsne}. Similarly as with the reconstruction experiments, we again see a drop in performance for bicubic interpolation, which is a learning-free  approach and was hence not trained for specific image characteristics. This suggests that ensuring good data separation is a harder task for LR images generated by NMS and that the drop in the KL divergence is not only a result of mismatched degradation functions. However, if we take the performance drop of the interpolation approach as our baseline, we observe that the FH models are much more sensitive to the characteristics of the LR data. The KL divergence of all models drops to a comparable value around $0.2$ and for the majority (except for URDGN) even falls slightly behind bicubic interpolation.

To further analyze the separability of the ResNet-$101$ descriptors of the hallucinated images, we report values for another non-parametric separability measure. i.e., Thornton's Geometric Separability Index (GSI), however, this time for the entire LFW and SCFace datasets and all FH models in Table~\ref{tab: GSI}. The index is defined as the fraction of data instances of a given dataset, $\mathcal{S}$, that has the same class-labels as their nearest neighbors, i.e.~\cite{thornton1998separability}:
$GSI = \frac{1}{n}\sum_{i=1}^nf(\mathbf{z}_i,\mathbf{z}'_i)$,
where $n$ stands for the cardinality of $\mathcal{S}$ and $f$ is an indicator function that returns $1$ if the $i$-th ResNet-$101$ descriptor $\mathbf{z}_i$ and it's nearest neighbor $\mathbf{z}'_i$ share the same label and $0$ otherwise. GSI is bounded between $0$ and $1$, where a higher value indicates better separability. We use the cosine similarity to determine nearest neighbors.
\begin{table}[!tb]
\renewcommand{\arraystretch}{1.05}
\caption{GSI values achieved by the FH models in the ResNet-$101$ feature space. Note the decrease in the data separability due to mismatched degradation functions. Arrows indicate an increase or decrease in value compared to the baseline bicubic interpolation.\vspace{1mm}}
\label{tab: GSI}
\centering
\footnotesize
\begin{tabular}{ l  ccccc}
\hline
\multicolumn{1}{l}{\multirow{2}{*}{Approach}}  & \multicolumn{3}{c}{LFW}  &\multirow{2}{*}{SCFace} \\\cline{2-4}
  & MS & NMS & Change & \\ \hline
  Bicubic (baseline) & $0.6283$ &  $0.5032$ &{$-19.9\%$}  & $0.5963$\\
 URDGN   & $0.6481$ $\blu{\uparrow}$ & $0.4866$ $\red{\downarrow}$ & {$-24,9\%$}   & $0.5346$\\
 LapSRN   & $0.6657$ $\blu{\uparrow}$ & $0.4906$ $\red{\downarrow}$& {$-26.3\%$}   & $0.6218$\\
 CARN   & $0.7130$ $\blu{\uparrow}$ & $0.4858$ $\red{\downarrow}$& {$-31.8\%$}  & $0.5691$\\
 SRResNet   & $0.7084$ $\blu{\uparrow}$ & $0.4927$ $\red{\downarrow}$& {$-30.4\%$}   & $0.5840$\\
 C-SRIP   & $0.7104$ $\blu{\uparrow}$ & $0.4893$ $\red{\downarrow}$& {$-31.1\%$}   & $0.5712$\\ \hline
\end{tabular}\vspace{-3mm}
\end{table}

The results in Table~\ref{tab: GSI} again show that the data separability is improved with all FH models compared to the baseline with the MS scheme on LFW. With the NMS scheme all models perform worse than the baseline and also exhibit a larger drop in separability than simple bicubic interpolation. On SCFace we see a similar picture. Only LapSRN results in better separability than the interpolation-based baseline, while all other FH models decrease separability.
These results again point to the importance of suitable training data, as FH models do not generalize well to unseen image characteristics and perform different than expected when applied on real-world imagery.

\textbf{Recognition experiments.} In our last series of experiments we look at the recognition performance ensured by the FH models and extracted ResNet-$101$ descriptors on LFW and SCFace. For LFW we follow the so-called ``unrestricted outside data'' protocol and use the $6000$ pre-defined  image pairs in verification experiments.
We keep one of the images in each pair unchanged (at the original resolution), and down-sample the second one using either the MS or NMS scheme. The LR images are then upscaled with the FH models and used to extract ResNet-$101$ descriptors. Matching is done with the cosine similarity. We report verification accuracy for the $10$ predefined experimental folds. For SCFace we perform a series of identification experiments, where we try to recognize subjects in the upscaled HR probe images based on the HR gallery data.
\begin{figure}[!tb]
\begin{minipage}{0.49\columnwidth}
  \centering
  \includegraphics[width=1\textwidth]{recog_results_lfw_resnet101.pdf}
\end{minipage}
\hfill
\begin{minipage}{0.49\columnwidth}
  \centering
  \vspace{1.5mm}
  \includegraphics[width=1\textwidth]{recog_results_scf_resnet101.pdf}

  \end{minipage}\vspace{1mm}
  \caption{Recognition results on LFW (left) and SCFace (right). With a matching degradation function all models improve upon interpolation. The results are less predictable with image characteristics not seen during training. Best viewed in color.\vspace{-2mm}}
  \label{fig: LFW_SCFace_results}
\end{figure}

Fig.~\ref{fig: LFW_SCFace_results} shows that on HR LFW images the ResNet-$101$ model achieves a median verification accuracy of $95.1\%$. When the image size is reduced to $24\times24$ pixels with the MS scheme and the LR images are upscaled with bicubic interpolation, the performance drops to $84.5\%$. The FH models improve on this and achieve significantly better results. The highest median accuracy of $91.8\%$ comes from C-SRIP, which is the top performer in this setting. With the NMS scheme the drop in performance is larger for all methods  compared to the HR data. URDGN, LapSRN and CARN are only able to match the performance achieved by bicubic interpolation, while SRResNet and C-SRIP degrade results.

Results for SCFace  are shown  separately for each of the five cameras and in the form of the overall mean identification accuracy (i.e., rank-$1$) in Fig.~\ref{fig: LFW_SCFace_results}. We see that none of the FH models outperforms the bicubic baseline on all cameras. Overall, LapSRN offers a slight improvement over bicubic interpolation considering the average identification accuracy, but the performance gain is modest and in the range of $3\%$. The ranking of the models is also not consistent across different cameras, which generate LR data with very different characteristics. Observe, for example, C-SRIP, which performs worst with images from camera $2$, but is one of the top performers on camera $4$, where it gains around $10\%$ in performance over bicubic interpolation.
These results show that without suitable mechanisms that are able to compensate for the bias introduced into FH model by the training data, hallucination results with real-world images are unpredictable and findings made with artificially down-sampled images cannot simply be extrapolated to real-world data.

\section{Conclusion, discussion and outlook}

We have studied the impact of dataset bias on the problem of face hallucination and analyzed five recent CNN-based FH models on artificially degraded as well as real-world LR images. Below is summary of the main findings:
\begin{itemize}[leftmargin=*]\vspace{-0.5mm}
    \item \textbf{Reconstruction and robustness:} FH models achieve better reconstruction performance than the learning-free interpolation baseline on LR images matching the training data in terms of characteristics. However, their superiority fades away quickly
    as the LR image characteristics diverge from the training setting.
    The rather sudden drop in reconstruction quality points to an accuracy-robustness trade-off with FH models not present with learning-free approaches, as also observed for other CNN-based models by recent studies~\cite{stutz2018disentangling,su2018robustness}.\vspace{-0.5mm}
    \item \textbf{Separability and recognition:} We observe statistically significant improvements in data separability and face recognition performance, when LR image sre pre-processed with FH models (as opposed to interpolated), but \textit{only} for LR images degraded with the same approach as used during training. For mismatched image characteristics (with real-world data) we found no significant improvements in separability or recognition performance for any of the FH models, which in most cases fall behind simple interpolation
\end{itemize}
Overall, our results suggest that despite recent progress, FH models are still very sensitive to the characteristics of the LR input data. We found limited empirical proof of their usefulness for higher-level vision tasks (e.g., recognition) beyond improvements in perceptually quality -- which might be important for representation-oriented problems, such as alignment or detection. Our analysis shows that we, as a community, need to move away from the standard evaluation methodology involving artificially degraded LR images and focus on more challenging real-world data when developing FH models for specific vision problems.

A common way to mitigate the effects of dataset bias in CNN-based models from the literature are domain adaption (DA) techniques or ensemble approaches \cite{csurka2017domain}. These have not been explored extensively for the problem of face hallucination yet (see~\cite{bulat2018learn} for initial attempts), but seem like an good starting point to improve the generalization abilities of FH models and make them applicable to real-world data.

{\small
\bibliographystyle{ieee}