\section{Introduction}

The introduction of a conducting surface into a region disturbs the electromagnetic field.  As a general
measure of the disturbance we can calculate the mode sum
\begin{equation}
\label{modesum}
S(\Omega) = \sum_{\alpha} \left (e^{-\omega_{\alpha}/\Omega} - e^{-\overline{\omega}_{\alpha}/\Omega}\right )
\end{equation}
where $\overline \omega_{\alpha}$ are the mode frequencies before introduction of the surface \cite{note1}
and $\omega_{\alpha}$ is the resulting spectrum.  For a general spectrum, the mode sum $S(\Omega)$ is divergent for large values of the parameter $\Omega$ (for example, for a massless scalar field, it diverges as $\Omega^{d-1}$, where $d$ is the dimensionality); however,
for the electromagnetic modes disturbed by a thin perfectly conducting cylindrical shell (of any crosssection), the mode sum per unit length is finite
\cite{BD}.
In this case (the subject of this paper) the quantities of particular interest are the
Kac number per unit length ${\cal K}/L$ (the change in the number of possible modes \cite{kac,BD})
and the Casimir energy per unit length ${\cal C}/L$ (the change in the zero point energy \cite{casimir} formally calculable from the sum
of $\frac {1}{2} (\omega_{\alpha} - \overline\omega_{\alpha})$ \cite{note2}).   These can be extracted from $S$ by
an expansion in powers of $1/\Omega $:
\begin{equation}
\label{kacnumber}
S(\Omega)/L = {\cal K}/L - \frac {2}{\Omega} {\cal C}/L + ...
\end{equation}
From here the Kac number $\mathcal{K}$ can be recognized as the coefficient $B_{3/2}$ of the heat kernel expansion widely used in the Casimir calculations \cite{heat}.

For the case of a cylinder (of any cross section) the modes have the form
\begin{equation}
\label{cylindermodes}
\omega_{\alpha}(q) = \sqrt{q^2+ w_{\alpha}^2}  ,
\end{equation}
where $q$ is the continuous wave vector of the modes in the axial direction and $w_{\alpha}$ are the cutoff frequencies for the modes.  There are two types of modes, in which either the magnetic field or the electric field is purely transverse \cite{Jackson}.  The cutoff frequencies $w_{\alpha}$ are determined by the
solutions of the two-dimensional Helmholtz problem for a scalar field representing the component of the electric or magnetic field parallel to the axis of the cylinder, with Dirichlet and Neumann boundary conditions for the two mode types.  The mode sum (\ref{modesum}) is over both types of modes.

The sum over the cutoff frequencies (\ref{modesum}) can be done with the aid of a contour integral representation \cite{contour}.  Let $\psi(w)$ be a function that is zero for each cutoff frequency, and let $\overline \psi(w)$ be the
corresponding function for the undisturbed space.  Then
\begin{equation}
\label{contour}
\frac{S(\Omega)}{L} = \frac {1} {2 \pi i} \oint dw
\int_{- \infty}^{\infty} \frac {dq}{2 \pi} e^{-\sqrt{q^{2}+w^{2}}/\Omega} \frac{d}{dw}
\ln \frac{\psi (w)}{\overline \psi (w)}
\end{equation}
where the first integral is over a contour enclosing the real axis, and $L$ is the (infinite) length of
the cylinder.  The advantage to this
representation is that the contour can be deformed to lie along the imaginary $w$ axis, where the
functions $\psi(w)$ and $\overline \psi(w)$ are slowly varying, instead of highly oscillatory.  The integral over $q$ is
\begin{equation}
\int_{- \infty}^{\infty} e^{-\sqrt{q^2+w^2}/\Omega} dq = 2w K_{1}\left (\frac{w}{\Omega}\right )
\end{equation}
where $K_{n}$ is the modified Bessel function of order $n$. Using this and making the change in variables $w \rightarrow iy$ gives
\begin{eqnarray}
\label{contour2}
\frac{S(\Omega)}{L} &=& \frac {1} {2 \pi^2} \oint dy \
 y K_{1}\left (\frac {iy}{\Omega}\right ) \frac {d}{dy}
\ln \frac{\psi (iy)}{\overline \psi (iy)}
\nonumber
\\&=&- \frac {1} {2 \pi^2 \Omega} \int_{-\infty}^{\infty} dy \
i y K_{0}\left (\frac {iy}{\Omega}\right )
\ln \frac{\psi (iy)}{\overline \psi (iy)}
\end{eqnarray}
In the second representation an integration by parts has been performed, and the contour integral has been converted to an integral over the $y$ axis.

In the cases to be considered, the function $\ln[\psi (iy)/\overline \psi (iy)]$
can be taken to be an even function of $y$.  Then we can replace $i y K_{0}(iy/\Omega)$ by its even part, $(\pi/2) |y| J_{0}(y/\Omega)$ ($J_{n}$ is the Bessel function of order $n$), and reduce the range of integration to the positive $y$ axis, arriving at the representation
\begin{equation}
\label{contour3}
\frac{S(\Omega)}{L} = - \frac {1} {2 \pi \Omega} \int_{0}^{\infty} ydyJ_{0}\left (\frac{y}{\Omega}\right )  \ln \frac{\psi (iy)}{\overline \psi (iy)}
\end{equation}

The quantity $\ln[\psi (iy)/\overline \psi (iy)]$ becomes small for large $y$.
In general the leading term is of order $y^{-1}$; this determines the Kac number per unit length, as will now be shown.
Define
\begin{equation}
\label{kacdef}
\frac{{\cal K}}{L} = -\frac{1}{2\pi} \lim_{y \rightarrow \infty} y \ln \frac{\psi (iy)}{\overline \psi (iy)}
\end{equation}
and separate Eq.(\ref{contour3}) into two terms:
\begin{eqnarray}
\label{contour4}
\frac{S(\Omega)}{L} &=& \frac{{\cal K}}{L} \int_{0}^{\infty} \frac {dy}{\Omega} J_{0}\left (\frac{y}{\Omega}\right )\nonumber\\
 &-& \frac {1}{\Omega} \int_{0}^{\infty}  dy \
 J_{0}\left (\frac{y}{\Omega}\right ) \left (\frac {y} {2 \pi} \ln \frac{\psi (iy)}{\overline \psi (iy)} + \frac{{\cal K}}{L}\right )
\end{eqnarray}
The first integral converges and is equal to unity, independent of the cutoff $\Omega$. The second integral is convergent without the Bessel function, so that we may take the limit $\Omega \rightarrow \infty$ in the integrand.  Then according to Eq.(\ref{kacnumber}), the Casimir energy per unit length is
\begin{equation}
\label{casmir}
\frac{{\cal C}}{L} =  \frac {1}{2} \int_{0}^{\infty} dy
  \left (\frac {y} {2 \pi} \ln \frac{\psi (iy)}{\overline \psi (iy)} + \frac{{\cal K}}{L} \right)
\end{equation}

\section{Circular Cylinder}

The Kac number for an arbitrarily shaped conductive shell has been given previously \cite{BD}.  The Casimir energy of the circular cylindrical shell has been also determined \cite{deraad}.  Here we will give a new route to the known result in the familiar setting of the circular cylinder, in preparation for the extension to the case of the cylinder of elliptical cross section.

We consider the effect on the electromagnetic spectrum of the introduction of a cylindrical conducting boundary of radius $A$ into another cylinder of radius $Z$ (which will be taken to infinity shortly \cite{note1}).   The cutoff frequencies of the system interior to the larger cylinder are specified by a scalar function $\varphi(\rho,\phi)$, which satisfies the two-dimensional Helmholtz equation.  The variables can be separated, so that $\varphi$ is a combination of angular factors $\exp(i n \phi)$ and Bessel functions of corresponding order.   There are both $TE$ and $TM$ modes, for the regions $\rho < A$ and $A < \rho < Z$, so that there are four mode conditions: $J_{n}(w A)=0$; $J_{n}'(w A)=0$; $H_{n}^{(1)}(w A) J_{n}(w Z) - H_{n}^{(1)}(w Z) J_{n}(w A)= 0$;
and $H_{n}^{(1)}{'}(w A) J_{n}'(w Z) - H_{n}^{(1)}{'}(w Z) J_{n}'(w A) = 0$ ($H_{n}^{(1)}$ is the Hankel function of order $n$).  Then a candidate for $\psi$ is the product for all $n$ of the left-hand sides of these four expressions, and $\overline \psi$ is the product of the factors $J_{n}(w Z) J_{n}'(w Z)$.  However, for $w = iy$, the Bessel functions are increasing or decreasing exponentially, and in the limit of large $Z$ the exponentially small $H_{n}^{(1)}(iZA)$ terms can be dropped so that $ \psi(iy)/\overline \psi (iy) $
reduces to a product of modified Bessel functions
\begin{equation}
\label{cylinder1}
\frac{\psi(iy)}{\overline \psi (iy)} = \prod_{n}
-(2 A y)^2 I_{n}(A y) K_{n}(A y) I_{n}'(A y) K_{n}'(A y)
\end{equation}
The factor $-(2Ay)^2$ has been introduced so that the factors in the product approach unity for large $y$.
This product can be written in a simpler form with the aid of the Wronskian relation $r I_{n}'(r)K_{n}(r)- r I_n(r)K_{n}'(r)= 1$:
\begin{equation}
\label{cylinder2}
\frac{\psi(iy)}{\overline \psi (iy)} = \prod_{n} \left (1 - \sigma_{n}^{2}\right )
\end{equation}
where
\begin{equation}
\label{sigmadefinition}
\sigma_{n} = y \frac {d}{dy} I_{n}(A y) K_n(A y)
\end{equation}

There is an alternate route to (\ref{cylinder1}) that sheds some light on its meaning \cite{KSLZ}.  The path integral representation of quantum mechanics turns three-dimensional quantum mechanics at zero temperature into a four-dimensional classical statistical mechanics problem where the ratio $H/T$ that determines the Boltzmann weight at a temperature $T$ is represented by the ratio of the action to Planck's constant.  The introduction of a boundary suppresses $TM$ modes (because the longitudinal electric field must vanish at the boundary) but introduces new $TE$ modes (because the longitudinal magnetic field can be discontinuous across the boundary).  In either case the effect is localized near the boundary, and can be described by solutions of the modified Helmholtz equation $(\triangle-y^{2})u(\textbf{r},y) = 0$ with sources on the boundary.  These solutions have radial parts that are described by the modified Bessel functions, and the amplitudes of the sources are proportional to $I_{n}(yA)$, $K_n(yA)$, $I_{n}'(yA)$, and $K_{n}'(yA)$ -- the quantities that enter into (\ref{cylinder1}). This approach gives a route to the calculation of the Casimir
energy entirely in terms of functions defined on the
imaginary $w$ axis.

Combining (\ref{casmir}) and (\ref{cylinder2}) gives
\begin{equation}
\label{casimircylinder}
\frac{{\cal C}}{L} =  \frac {1}{2}
\int_{0}^{\infty}
\left ( \frac{{\cal K}}
{L} + \frac {y} {2 \pi}  \sum_{n=-\infty}^{\infty}
 \ln (1-\sigma_{n}^2) \right ) dy
\end{equation}
The logarithm can be expanded in a sum of powers of $\sigma_{n}$.   Then  ${\cal C}/L =  \sum T_{m}$, where
\begin{eqnarray}
\label{cylinderexpansion}
T_{1}&=& \frac {1}{4 \pi } \int_{0}^{\infty}   \left (\frac{2\pi {\cal K}}{L} - y \sum_{n=-\infty}^{\infty}
\sigma_{n}^{2}\right ) dy
\nonumber
\\
T_{m}&=& -\frac {1}{4 \pi m} \int_{0}^{\infty} y dy  \sum_{n=-\infty}^{\infty}
\sigma_{n}^{2m}
\end{eqnarray}

\subsection{Klich Expansion}

Klich \cite{klich} has pointed out a useful trick for evaluating the sums that appear in these equations.
The Green function for the modified Helmholtz problem can be expanded in modified Bessel functions about an arbitrary origin, and by equating representations we can derive the identity \cite{JDJ}
\begin{equation}
\label{green}
K_{0}\left (yA \rho(\phi-\phi')\right ) = \sum_{n=-\infty}^{\infty} I_{n}(Ay) K_{n}(Ay) e^{in(\phi-\phi')}
\end{equation}
where
\begin{equation}
\rho(\phi) = \sqrt{2 - 2 \cos\phi}= 2 \left |\sin \frac{\phi}{2}\right |
\end{equation}
Performing the operation $y d/dy$ on both sides of (\ref{green}) gives
\begin{eqnarray}
\label{Gdef}
H(1,2)&\equiv& y \frac {d}{dy} K_{0}\left (2 y A \left |\sin \frac{\phi_{1}-\phi_{2}}{2}\right |\right )
\nonumber
\\
&=& \sum_{n=-\infty}^{\infty} y \frac{d}{dy} \left (I_{n}(yA) K_{n}(yA)\right ) e^{in(\phi_{1}-\phi_{2})}
\nonumber
\\
&=&\sum_{n=-\infty}^{\infty} \sigma_{n} e^{in(\phi_{1}-\phi_{2})}
\end{eqnarray}
By multiplying this expression times itself $2m$ times, identifying the $\phi_{i}$ in a chain of pairs, and integrating over all $\phi_{i}$ we may derive a series of identities \cite{CM}
\begin{eqnarray}
\label{klichidentities}
\sum_{n=-\infty}^{\infty} \sigma_{n}^{2m} &=& \int_{-\pi}^{\pi}\frac{d\phi_{1}}{2\pi} \ldots
\int_{-\pi}^{\pi}\frac{d\phi_{2m}}{2\pi} H(1,2)\nonumber\\
&\times&H(2,3)\ldots H(2m,1)
\end{eqnarray}
Substituting these into (\ref{cylinderexpansion}) gives an extension of the representation for ${\cal C}$ similar to that given by Balian and Duplantier \cite{BD}.

\subsection{Kac number}

For large $y$ the $\sigma_{n}$, Eq.(\ref{sigmadefinition}),  are small, and Eq.(\ref{kacdef}) gives
\begin{eqnarray}
\label{cylK}
 \frac {{\cal K}}{ L} &= &- \lim_{y \rightarrow \infty}
 \frac{y}{2\pi} \ln\frac{\psi(iy)}{\overline \psi (iy)}\nonumber\\
&=& - \lim_{y\rightarrow \infty} \frac{y}{2\pi} \sum_n \ln (1-\sigma_n^2)
=   \lim_{y\rightarrow \infty} \frac{y}{2\pi} \sum_n \sigma_n^2\nonumber\\
&=&  \lim_{y\rightarrow \infty} \frac{y}{2\pi} \int_{-\pi}^{\pi} \frac {d \phi_1}{2\pi}\int_{-\pi}^{\pi} \frac {d \phi_2}{2\pi} H(1,2) H(2,1)
\nonumber
\\
&=&  \lim_{y\rightarrow \infty} \frac{y}{\pi^{2}} \int_{0}^{\pi} d \beta \left [ 2yA\sin \beta K_{1}\left (2yA \sin \beta \right )\right ]^2
\end{eqnarray}
For large $y$, this integral is dominated by the contribution
from small $\beta$, so that we may replace $\sin\beta$
by its argument, leading to \cite{integral}
\begin{eqnarray}
\label{cylK2}
 \frac { {\cal K}}{L}
&=&  \frac {1}{2\pi^{2}A} \lim_{y\rightarrow \infty}\int_{0}^{2\pi yA} d(2yA\phi) (2yA\phi K_{1}(2yA \phi))^2\nonumber\\
&=& \frac{1}{2\pi^{2}A}\int_{0}^{\infty}dx (xK_{1}(x))^{2}=\frac {3}{64A}
\end{eqnarray}
This result was given previously by Balian and Duplantier \cite{BD}.  They also showed that with this choice $T_{1}$ in (\ref{cylinderexpansion}) vanishes.

\subsection{Casimir energy}

We evaluated the expressions for $T_{m}$ (\ref{cylinderexpansion}) by a Monte Carlo integration.   The first step is to change variables to $y = (\ln z)^{4}$, which compresses the range of integration to the interval $z \in [0,1]$ (the choice of the fourth power is somewhat arbitrary, but was found to give better sampling of the function in the Monte Carlo integration).  Then the multiple integral over the angles and $z$ is given by the average of the integrand (in the new variable) evaluated at uniformly and randomly sampled values for $z$ and the $\phi_{i}$.  Averaging over $10^8$ configurations we find
\begin{eqnarray}
\label{results}
T_2 = \frac{-0.00758 \pm 0.00002}{A^{2}}
\nonumber
\\
T_3= \frac{-0.002264 \pm 0.000002}{A^{2}}
\nonumber
\\
T_4= \frac{-0.001080 \pm 0.000001}{A^{2}}
\nonumber
\end{eqnarray}
For large $m$ the integral is almost entirely due to the terms involving $\sigma_{0} = y d/dy (I_{0}(Ay)K_{0}(Ay))$:
\begin{eqnarray}
\label{results2}
T_{20} =
-\frac{1}{8\pi}\int_{0}^{\infty} \sigma_{0}^4 ydy =-\frac{0.00685}{A^{2}}
\nonumber
\\
T_{30} =
-\frac{1}{12\pi}\int_{0}^{\infty} \sigma_{0}^6 ydy = -\frac{0.002258}{A^{2}}
\nonumber
\\
T_{40} =
-\frac{1}{16\pi}\int_{0}^{\infty} \sigma_{0}^8 ydy = -\frac{0.001081}{A^{2}}
\end{eqnarray}
This series ${ T_{m0} }$ is found to fall off as $0.0346 \times (2m)^{-2.5}$ rather accurately, and it is also possible
to evaluate the sum of the series in the form
\begin{equation}
S_0=
\frac{1}{4\pi} \int_{0}^{\infty} \left (\ln(1-\sigma_{0}^{2}) + \sigma_{0}^{2}\right ) dy = -\frac{0.012799}{A^{2}}
\end{equation}
Therefore only the first few $T_{m}$ need to be calculated;  we thus arrive at the evaluation
\begin{eqnarray}
\frac{{\cal C}}{L} &=& S_0-T_{20}-T_{30}-T_{40}+T_{2}+T_{3}+T_{4}
\nonumber
\\
&=& \frac{- 0.01354 \pm 0.00002}{A^{2}}
\end{eqnarray}
in agreement with the accepted value ${\cal C}/L = -0.013561343/A^{2}$ \cite{deraad}.  The Monte Carlo program is simple in structure and only requires modest computational resources.  The accuracy is limited by the $T_2$ term, which is difficult to evaluate because the integrand has significant contributions from large $y$ when the four angles $\phi_{i}$ are nearly the same.

\section{elliptical cylinder}

\subsection{Mode expansion}
Elliptical coordinates are related to the rectangular coordinates $X, Y$ by
\begin{eqnarray}
\label{elliptical}
X = h \cosh \xi \cos \eta
\nonumber
\\
Y = h \sinh \xi \sin \eta .
\end{eqnarray}
The surfaces of constant $\xi$ are confocal ellipses with axes $A = h \cosh \xi$ and $B = h \sinh \xi$.
The  distance between the foci is $2h$; the eccentricity of the ellipse is $\epsilon = {\text {sech}} \  \xi$.
The case of the circle is recovered in the limit $\xi \rightarrow
\infty$,  $h \cosh\xi = \sqrt{X^{2}+Y^{2}}$ (which implies $h \rightarrow 0$).

The two-dimensional modified Helmholtz equation can be separated in these variables.  The solutions have the form of products $P(\eta)Q(\xi)$, where $P$ and $Q$ satisfy the Mathieu equations
\begin{equation}
\label{de-eta}
\frac{d^2 P}{d \eta^2} = -(a +h^2 y^2 \cos^2 \eta) P(\eta)
\end{equation}
\begin{equation}
\label{de-xi}
\frac{d^2 Q}{d \xi^2} = (a + h^2 y^2 \cosh^2 \xi) Q(\xi)
\end{equation}
where $a$ is the separation constant.   The functions $P(\eta;a,y)$ play a role similar to that of the trigonometric functions in the case of the cylinder, and become them in the circle limit.  The condition that $P(\eta;a,y)$ be periodic restricts $a$ to a discrete set of values, but unlike the case of the circle the allowed values are not integers and depend on $y$; we will refer to the set of these as $\{a_{n}(y)\}$,
where $n$ is a counting label (the
modes can be ordered
 so that $a_{n+1} > a_n$, and so that $P(\eta;a_{0},y)$ is
the nodeless function of $\eta$).
For any $y$, the corresponding set of functions $P(\eta;a_{n}(y),y)$ forms a complete set for expression of a general function of $\eta$; they will be normalized so that for any
$a$ and $b$ in $\{a_n\}$,
\begin{equation}
\label{normaliztion}
\int_{-\pi}^{\pi} P(\eta;a,y) P(\eta;b,y) d\eta = \delta_{ab}
\end{equation}
Then it follows
from the completeness of the $P(\eta; a,y)$ that
\begin{equation}
\label{complete}
\sum_{a \in \{a_n\}} P(\eta;a,y) P(\eta';a,y) = \delta ( \eta-\eta')
\end{equation}

The function $Q(\xi;a,y)$ plays a role analogous to a modified Bessel function; there are again two solutions: $Q_{1}$, which is regular at small $\xi$ (and increasing); and $Q_{2}$,  which is small at large $\xi$ (and decreasing).  These are related by the Wronskian relationship
\begin{equation}
\label{MathieuWronskian}
Q_{2}(\xi;a,y) \frac {d}{d\xi} Q_{1}(\xi;a,y)-
Q_{1}(\xi;a,y) \frac {d}{d\xi} Q_{2}(\xi;a,y) = 1
\end{equation}
However, there are some important differences from the case of the circular cylinder.  Eq.(\ref{de-xi}) differs from the Bessel equation in that there is no term in the first derivative.  This implies that the Wronskian is constant, and we are choosing the normalization so that it is unity, as stated in (\ref{MathieuWronskian}).  This also means that the product of the two solutions is asymptotically unity at $\xi$ large, and (via Eq.(\ref{de-xi})) at $y$ large as well.   Additionally the Bessel functions depend on radial position $r$ only through the $ry$ combination, and on what happens in the angular sector only through the order $n$, while the Mathieu function $Q(\xi;a,y)$ depends on $y$, $\xi$, and $a$ separately.  We can make the mathematics of the ellipse look more like the mathematics of the circle if we define the restricted functions $I_{n}(y;\xi) = Q_{1}(\xi;a_{n}(y),y)$ and $K_{n}(y;\xi) = Q_{2}(\xi;a_{n}(y),y)$.  For real cutoff frequency $w$ we can similarly define the oscillatory functions $J_n(w;\xi)$ and $H_{n}(w;\xi)$ that correspond to the usual Bessel functions.

The cutoff frequencies for the elliptical cylinder whose shape and size are determined by $h$ and $\xi$ are determined by the values of $w$ for which either
$J_{n}(w;\xi)= 0$ or $d/d\xi (J_{n}(w,\xi)) = 0$  for the modes inside the cylinder and with a similar but more complicated relationship for the modes outside the cylinder.  The quantity $\psi (w)/\overline \psi (w)$ needed for the contour integral relationship (\ref{contour}) for the mode sum is constructed from these elements as in the circular cylinder case.  After the change of variables $w \rightarrow iy$, we arrive at
\begin{eqnarray}
\label{ellipse1}
\frac{\psi (iy)}{\overline \psi (iy)}&=& -4 I_{n}(y;\xi) \frac {d}{d\xi} I_{n}(y;\xi) K_{n}(y;\xi) \frac {d}{d\xi} K_{n}(y;\xi)
\nonumber
\\
&=&1-\left (\frac {d}{d\xi} (I_n(y;\xi)K_{n}(y;\xi)\right )^{2}
\nonumber
\\
&\equiv&1-\sigma_{n}^2
\end{eqnarray}
where we have used the Wronskian relationship (\ref{MathieuWronskian}) in the second line.
The cautious reader will have reason to be suspicious about the analytic continuation of $J_{n}(w;\xi)$ to $I_{n}(y;\xi)$.  In the case of the circular cylinder the order $n$ of the Bessel function is an integer and unaffected by the frequency (its argument $w$); here, the $a_{n}$ depend on the frequency, and take on different values on the real and imaginary axes. For complex values of $y$, the $a_{n}(y)$ are surely also complex.  However, the point is that the Mathieu function $Q(\xi,y,a)$ is analytic in $y$ for any choice of $\xi$ and $a$, and then the restricted function $I_{n}(y;\xi)$ is also analytic;  the way that the set $a_{n}$ evolves as $y$ is varied is built into the definition of the restricted functions.  Finally, we will note that the path integral approach \cite{KSLZ} gives an alternate route to the representation (\ref{ellipse1}).

With this foundation, the determination of the Kac number and the Casimir energy for the cylinder of elliptic cross section is similar to that for the circular cylinder.  In particular, Eqs. (\ref{kacdef})-(\ref{casmir}), (\ref{casimircylinder}), and (\ref{cylinderexpansion}) are unaltered, except that the sums are now over the set $\{a_n(y)\}$.

\subsection{Klich expansion}
In elliptic variables the Green function satisfies the partial differential equation
\begin{eqnarray}
\label{PDE}
\left ( \frac {\partial^2}{\partial \xi^2} +\frac {\partial^2}{\partial \eta^2}
-y^2 h^2 (\sinh^2 \xi + \sin^2 \eta )\right ) G(\xi,\eta;\xi'
 ,\eta')
\nonumber
\\
=  - 2\pi \delta (\xi-\xi') \delta (\eta - \eta')
\end{eqnarray}
We can represent it in two ways: by direct transcription from the polar variables
\begin{equation}
\label{green1}
G(\xi,\eta;\xi',\eta') =  K_0(y \rho) ,
\end{equation}
where
$\rho$
is the distance between the points $(\xi,\eta)$ and $(\xi',\eta')$; or as an
expansion in Mathieu functions
\begin{eqnarray}
\label{green2}
G(\xi,\eta;\xi',\eta') &=& \sum_{a \in \{a_n\}}
g_n(y)
P(\eta;a,y) P(\eta';a,y)\nonumber\\
 &\times& I_n(\xi,y) K_n(\xi',y)
\end{eqnarray}
for $\xi < \xi'$ (and with $\xi$ and $\xi'$ interchanged for $\xi > \xi'$).
The coefficients $g_n(y)$ are determined by the conditions that $G$ and $\partial G/\partial\xi$ be
continuous at $\xi_1 = \xi_2$ (for $\eta_1 \ne \eta_2$), and that the singularity at $(\xi_1,\eta_1) = (\xi_2,\eta_2)$
has the amplitude implied by (\ref{PDE}).
The result of this analysis is that $g_n(y) = 2\pi$ for all $n$ and $y$.
Thus we have the identity
\begin{eqnarray}
\label{identity}
K_0(y\rho) &=& 2 \pi \sum_{a \in \{a_n(y)\}} P(\eta_1;a,y)P(\eta_2;a,y)\nonumber\\
&\times& I_n(\xi_1,y) K_n(\xi_2,y)
\end{eqnarray}

We only need this result for the case $\xi_1 = \xi_2 = \xi$, where $\rho$ reduces to
\begin{eqnarray}
\label{rhodef}
\rho &=& h\sqrt {\cosh^{2}\xi (\cos \eta - \cos \eta')^2 +\sinh^{2} \xi (\sin \eta - \sin \eta')^2}
\nonumber
\\
&=& h \left |\sin \frac {\eta -\eta'}{2} \right | \Gamma \left (\frac {\eta+\eta'}{2}\right )
\end{eqnarray}
where
\begin{equation}
\label{Gamma}
\Gamma(\alpha) =  \sqrt{\sinh^2\xi+\sin^2 \alpha}
\end{equation}

Take the $\xi$ derivative of (\ref{identity}) to get
\begin{eqnarray}
\label{ident2}
\frac {\partial}{\partial \xi} K_0\left (2 y h \left |\sin\frac {\eta_1-\eta_2}{2}\right | \Gamma\left (\frac {\eta_1+\eta_2}{2}\right )\right )
\nonumber
\\
= 2 \pi  \sum_{a \in \{a_n(y)\}} P(\eta_1;a,y)P(\eta_2;a,y) \sigma_n
\end{eqnarray}
Thus by defining
\begin{equation}
\label{Gdef2}
H(\eta_1,\eta_2) =  \frac {\partial}{\partial \xi} K_0\left (2 y h \left |\sin\frac{\eta_1-\eta_2}{2}\right | \Gamma \left (\frac{\eta_1+\eta_2}{2}\right ) \right )
\end{equation}
we obtain an identity equivalent to (\ref{klichidentities}).
In the circular case (large $\xi$),
$\Gamma \approx \sinh \xi$ and
$\sinh \xi \approx \cosh \xi $, so that (\ref{Gdef2}) reduces to (\ref{Gdef}) with $A = h \cosh \xi$.
For the circular case,
the Klich expansion was a convenience that improved the rate of convergence by
summing over all orders of the Bessel functions.  In the elliptical case it is even
more useful, because it allows us to do
the sums equivalent to (\ref{cylinderexpansion}) without constructing
the sets $\{a_n(y)\}$ or the Mathieu functions.

\begin{figure}
\includegraphics[width=1.0\columnwidth,keepaspectratio]{kacfigure3.eps}
\caption{How the Kac number per unit length depends on the shape of the cylinder. }
\end{figure}

\subsection{Kac number}

Reprising the argument given above, the Kac number per unit length determines the leading
contribution to $\ln[\psi(iy)/\bar \psi(iy)]$ at large $y$.   The analogue to Eq. (\ref{cylK}) is
\begin{eqnarray}
\label{ellipseK}
 \frac { {\cal K}}{L} &=& \lim_{y \rightarrow \infty}
\frac {y}{2\pi}
 \int_{-\pi}^{\pi}
\frac {d\eta_1}{2\pi} \int_{-\pi}^{\pi}\frac {d\eta_2}{2\pi}
H^{2}(\eta_1,\eta_2)
\nonumber
\\
&=&
\lim_{y\rightarrow \infty}\frac {y}{\pi^{3}} \int_{0}^{\pi}
d\alpha \int_{0}^{\pi}d\beta
\left (\frac {d}{d\xi} K_{0}\left (2yh \sin\beta \Gamma(\alpha)\right ) \right )^2\nonumber\\
\end{eqnarray}
where we have changed variables to $\alpha=(\eta_1+\eta_2)/2$ and
$\beta = (\eta_1 -\eta_2)/2$.  As was previously explained, for large $y$
we can replace $\sin\beta$ by $\beta$; then the integral over $\beta$ is
the same as was considered in Eqs.(\ref{cylK}) and (\ref{cylK2}).  The result is
\begin{eqnarray}
\label{ellipseK2}
\frac {\cal K}{L} &=& \frac{3}{64h}\int_0^{\pi} \frac{d\alpha}{\pi}
\frac {\sinh^2(\xi) \cosh^2(\xi)}{\Gamma^{5}(\alpha)}
\nonumber
\\
&=& \frac {3}{64} \int_0^{\pi} \frac {d\alpha}{\pi}
\frac {A^2 B^2}
{(A^2 \sin^2 \alpha + B^2 \cos^2 \alpha )^{5/2}}
\end{eqnarray}
which can be evaluated in terms of the complete elliptic integrals ($A > B$ are the
axes of the ellipse).  This agrees from the general expression given
by Balian and Duplantier \cite{BD}.
In the limit of small $B$, this expression is of order $B^{-2}$.  Therefore in Figure 1 we plot ${\cal K}B^2/LA$, which is
independent of the size of the cylinder and only weakly dependent on its shape.  The horizontal axis is the eccentricity $\epsilon = {\text {sech}} \xi$.

For large eccentricity, the cylindrical shell is approximately a pair of planar strips of width $A$ and area $LA$ separated by a distance $B$.  Then its Kac number must be proportional to the area ($\mathcal{K}\propto LA$) while the Kac number per unit area can only depend on the strip separation $B$.   Since the Kac number itself is dimensionless, the combination of these two arguments predicts a dependence $\mathcal{K}\simeq LA/B^{2}$ thus explaining the divergence of the Kac number at large eccentricity.

\subsection{Casimir energy}

The term $T_1$ in Eq.(\ref{cylinderexpansion}) again vanishes for the case of the cylinder of elliptical cross-section:
substituting (\ref{Gdef2}) and (\ref{klichidentities}) into
(\ref{cylinderexpansion}) and then changing variables from
$\eta_i$ to $\alpha$ and $\beta$ leads to an integral
that differs from the circular case only by a scale factor.

The expansion (\ref{cylinderexpansion}) continues to be relevant, as does (\ref{klichidentities}), except
that now $H(\eta_{1},\eta_{2})$ is given by
(\ref{Gdef2}).  It follows from Eq. (\ref{cylinderexpansion}) that all of the $T_{m}$ are negative, and thus that the Casimir energy is negative.
The numerical evaluation of the $T_{m}$ can be done as above, using almost the same program.
\begin{figure}
\includegraphics[width=1.0\columnwidth, keepaspectratio]{rints3.eps}
\caption{The dependence of $|T_{m}|$ on $m$.
The curves are drawn for $\epsilon = 0.05, 0.50, 0.80, 0.90, $ and $0.95$, in ascending order.
The nearly constant slope of these lines indicates that
$-T_{m}$ is decreasing as $m^{-2.5}$.
}
\end{figure}
Figure 2 shows that it is again true that $-T_{m}$ decreases as $m^{-2.5}$, so that the series
sum converges.
We evaluated $T_{2}$ through $T_{10}$ by Monte Carlo integration, and applied a truncation correction based on the $m^{-2.5}$ law.  The resulting dependence of ${\cal C}/L$ on the eccentricity $\epsilon$ is shown in Figure 3.
Also shown in Figure 3 is $\sqrt{1-\epsilon^{2}} \times {\cal C}/L$.  The lack of dependence on $\epsilon$ for small $\epsilon$
agrees with the results of Kitson and Romeo \cite{KR}, who discussed the Casimir energy of a cylinder of elliptical cross-section perturbatively.

As was already observed, for large eccentricity the cylindrical shell is approximately a pair of planar strips of width $A$ separated by a distance $B$.  The Casimir attraction between these would give an energy ${\cal C} \simeq - LA/B^3$ \cite{casimir} ($LA$ is the area of a strip); this can be rewritten in terms of the eccentricity as ${\cal C} A^{2}/L \simeq - (1-\epsilon^{2})^{-1.5}$.  The dash-dotted line in Figure 2 shows that this crude argument successfully accounts for the divergence of the Casimir energy at large eccentricity.
\begin{figure}
\includegraphics[width=1.0\columnwidth, keepaspectratio]{casenrint5.eps}
\caption{How the Casimir energy per unit length varies with the eccentricity of the cylinder.
The dash-dotted line is $(1-\epsilon^{2})^{1.5} \times |{\cal C}| A^{2}/L$; this removes the
divergence at large eccentricity.
The dashed line is $\sqrt{1-\epsilon^{2}} \times |{\cal C}| A^{2}/L$, which removes the leading order dependence on $\epsilon$ for small $\epsilon$.
}
\end{figure}

\section{Discussion}

The case of a cylindrical shell with circular cross-section holds a special place in the physics of the Casimir effect.  At zero temperature Casimir forces are known to be attractive for parallel plates \cite{casimir}, repulsive for the sphere \cite{Boyer68, BD} and nearly zero (very weakly attractive) for a long cylindrical shell \cite{BD,deraad}.   Thus the latter is approximately the intermediate case.  The solution of the elliptical cross-section case allows us to follow the evolution of the Casimir attraction with eccentricity $\epsilon$ as the shell cross-section evolves from circular to highly eccentric, resembling the parallel plate geometry.  Therefore it is not surprising that the Casimir energy is found to decrease with eccentricity with the circular case corresponding to the energy maximum.  Naively one might have expected that the Casimir attraction in the $\epsilon\rightarrow 1$ limit would be stronger than its parallel plate counterpart \cite{casimir}. Our results indicate the opposite, however:  even though both attractions have the same order of magnitude, the parallel plate geometry generates stronger attraction.  This illustrates the non-additive character of the Casimir forces.

Since the interactions are attractive with the circular cross-section corresponding to the energy maximum and the large eccentricity $\epsilon=1$ limit being the minimum, with only Casimir stresses present a fixed area shell would be unstable with respect to collapse onto itself.  We argue that this remains the equilibrium state of the system at arbitrary temperature.  Indeed the high-temperature thermodynamics of the system is dominated by the Kac number as the latter determines the form of the free energy \cite{BD}
\begin{equation}
\label{fenergy}
\mathcal{F}\simeq -\mathcal{K}T\ln(Tl)
\end{equation}
at a temperature $T\gg 1/l$ where $l$ is a length scale approximately corresponding to the largest eigenfrequency of the problem.  For $\mathcal{K}>0$, as is the case in the problem under study, the equilibrium configuration of the shell must have the largest Kac number $\mathcal{K}$.  But we found (Figure 1) that the Kac number diverges in the large eccentricity $\epsilon=1$ limit which corresponds to the collapsed state.  However, as this limit is approached we have $l\simeq B\rightarrow 0$, thus inevitably leaving the range of applicability of the high-temperature expression (\ref{fenergy}) and entering the range of applicability of the zero-temperature theory where the ground state is still collapsed.  This allows us to argue that the equilibrium state of fixed area shell in the presence of Casimir stresses only is collapsed at arbitrary temperature.

\section{Acknowledgements}

EBK is supported by the US AFOSR Grant No. FA9550-11-1-0297. GAW gratefully acknowledges an Australian Postgraduate Award and the J. L. William Ph. D. scholarship.


\section{Introduction}
\label{sec:intro}
With the rapid growth of society and the modern logistics industry, road infrastructure has been greatly increased in today's world. There are a total of more than 64,000,000 kilometers of roads in the world~\cite{length}, which leads to massive operational requirements for pavement maintenance. The pavement inspection is one of the key steps~\cite{c1}. Generally speaking, the cameras are often utilized as pavement inspection equipment due to their low cost and the powerful data representational ability of images. Therefore, the pavement inspection task is often translated into a pavement distress analysis task based on the acquired pavement images, and then this task is accomplished manually by the proficient workers. Clearly, such an operation consumes plenty of time and labor resources due to an enormous amount of pavement images produced daily~\cite{c2}. Therefore, automating the pavement distress analysis can play a critical role in improving efficiency, reducing cost and also avoiding the labeling errors in manual pavement inspection.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.33]{first_page.pdf}
    \caption{The illustrations of pavement distress detection (finding out the distress images) and recognition (recognize the pavement distresses).}
    \label{fig.firstpage}
    \vspace{-0.5cm}
\end{figure}

Pavement distress detection and recognition are two fundamental tasks for pavement distress analysis, which aim at identifying the distressed pavement images and classifying the distressed images into specific categories, respectively, as shown in Figure~\ref{fig.firstpage}. In recent decades, many classical approaches have been proposed to address these two tasks, and they can be roughly categorized into two groups.

The first one is to utilize the image processing, hand-craft features and conventional classifiers to recognize the pavement distress~\cite{c3,c27,c28,c29,c30}. For example, Zhou et al.~\cite{c29} developed a two-step method that conducts the wavelet transform followed by a random transform to classify the pavement distress. Sun et al.~\cite{c3} proposed a crack classification method based on topological properties and chain code. The main drawback of these methods is that they often optimize the feature extraction and classification step separately or even do not involve any learning process which leads to poor performance. Moreover, it usually needs plenty of sophisticated image pre-processing.

The second group is comprised of those using deep learning-based methods. Inspired by the advance of deep learning approaches, it is more and more popular to apply different deep learning-based visual learning models for pavement distress detection and recognition~\cite{c8,c9,c10,c11,c16}. For example, K.Gopalakrishnan et al.~\cite{c8} leveraged VGG-16 pre-trained on ImageNet\cite{c9} to identify whether the specific pavement image is "crack" or "non-crack". Laha et al.~\cite{rddretina} detected road damages with RetinaNet~\cite{retinanet}. Compared to the conventional approaches, deep learning-based approaches often achieve better performance. However, most of these approaches only regard the pavement distress detection or recognition problem as common object detection or image classification problem and directly apply the classical deep learning approaches. They seldom paid attention to the specific characteristics of pavement images, such as the high image resolution, the low distress area ratio, and uneven illumination, in the model design phase.

To address the issue, instead of directly classifying the pavement images, IOPLIN~\cite{c16} performed the histogram equalization for suppressing the negative effects from illumination, and tackled the pavement distress detection task via inferring the labels of patches from pavement images with a Patch Label Inference Network (PLIN) for fully exploiting the high-resolution image information. IOPLIN is able to be iteratively trained with only the image label via the Expectation-Maximization Inspired Patch Label Distillation (EMIPLD) strategy and achieves promising detection performances for various categories of pavement distress. The main drawback of IOPLIN is that its optimization process is quite complex and time-consuming. Moreover, IOPLIN is not end-to-end trainable and is also unable to be further extended to the pavement distress recognition scenario.

To address the aforementioned issues, we present a novel pavement image classification framework named Weakly Supervised Patch Label Inference Network (WSPLIN)~\cite{wsplin} for both pavement distress detection and recognition. Similar to the IOPLIN, our method also accomplishes the pavement image classification via inferring the labels of patches from the pavement images with Patch Label Inference Networks (PLIN). Therefore, WSPLIN inherits the merits of IOPLIN, such as the better image information utilization and result interpretability, but also suffer from the obstacle of training PLIN only with image labels. Compared to IOPLIN, WSPLIN solves this model training issue via introducing a more concise end-to-end weakly supervised learning framework. Such a framework endows WSPLIN with better efficiency and greater flexibility, enabling the pavement distress recognition application.

In WSPLIN, the pavement image is divided into patches with different patch collection strategies under different scales for exploiting both global and local information. Then, a CNN is implemented as PLIN for inferring the labels of patches with a sparsity constraint. Finally, the patch label inference results are fed into a Comprehensive Decision Network (CDN) for completing the classification. We integrate PLIN and CDN as an end-to-end deep learning model. In such a manner, the PLIN can be optimized by the guidance of CDN and the patch label sparsity constraint in a cleaner and more efficient fashion. Moreover, three different strategies, namely Sliding Window (SW), Image Pyramid (IP), and Sparse Sampling (SS), are adopted for collecting patches from images. We name these corresponding WSPLIN versions, WSPLIN-SW, WSPLIN-IP and WSPLIN-SS, respectively. As same as IOPLIN, WSPLIN-SW has not considered any scale information during patch collection. It can be deemed as a naive version of WSPLIN. Different to WSPLIN-SW, WSPLIN-IP incorporates the scale information via dividing images into patches from coarse to fine based on an image pyramid. It is the default version of WSPLIN. WSPLIN-SW conducts a sparse patch sampling to collect only a few of patches from the image pyramid for improving the efficiency of WSPLIN. It can be seen as the speedy version of WSPLIN. We evaluate WSPLIN on a large-scale pavement image dataset named CQU-BPDD~\cite{c16} under different settings, including distress detection, one-stage recognition, and two-stage recognition. The experimental results show that WSPLIN outperforms extensive baselines and demonstrate the prominent advantages over IOPLIN in both efficiency and performance.

The main contributions of our work are summarized as follows:
\begin{itemize}
    \item We propose a novel end-to-end weakly supervised deep learning model named WSPLIN for addressing both pavement distress detection and recognition issues. WSPLIN not only inherits the merits of IOPLIN, but also enjoys faster training speed, better classification performance, and wider application scenarios over IOPLIN.
    \item Different from IOPLIN and the conventional CNN-based image classification methods, we introduce image pyramid to WSPLIN-IP for exploiting scale information. Moreover, we design a sparse patch sampling strategy in the image pyramid for further speeding up WSPLIN. The model training time of this faster WSPLIN version (WSPLIN-SS) is only one-fourth of the training time of IOPLIN while they share similar performance in pavement distress detection.

    \item We design a patch label sparsity constraint based on the prior knowledge of distress distribution and leverage the CDN to guide the training of PLIN in a weakly supervised way. The patch labels produced by PLIN provide interpretable intermediate information, such as the rough location and the type of distress.
    \item We empirically evaluate our model against the current state-of-the-art  CNN methods and some classic transformer methods as baselines in both the pavement distress detection and recognition tasks. Extensive results show that WSPLIN outperforms them in both tasks under different settings.

\end{itemize}

\section{Related Work}
\label{sec:rela_work}
\subsection{Image-based Pavement Distress Analysis}
The traditional pavement distress analysis approaches mainly include filter-based methods, and hand-crafted feature-based classical classifiers. For example, in~\cite{zhou2006wavelet}, wavelet transform is used to decompose a pavement image into different-frequency subbands. Hu et al.~\cite{hu2010novel} propose a novel Local Binary Pattern (LBP) based operator for pavement crack detection. In~\cite{shi2016automatic}, a random structured forest named CrackForest, which is combined with the integral channel features is proposed for automatic road crack detection. Kapela et al.~\cite{kapela2015asphalt} propose a crack recognition system based on the Histograms of Oriented Gradients (HOG). Pan et al.~\cite{pan2017object} use the four popular supervised learning algorithms (KNN, SVM, ANN, RF) to discern pavement damages. However, the traditional methods usually have weak performance owing to numerous artificial design factors, separate optimization procedures, and they cannot be adapted to a large number of data currently.

Inspired by the recent remarkable successes of deep learning in extensive applications, simple and efficient convolutional neural networks (CNN) based pavement distress analysis methods have gradually become the mainstream in recent years. In general, these methods can be divided into three parts according to the task objective: pavement distress segmentation~\cite{c11,crack500,yang2019feature,zou2018deepcrack}, pavement distress location~\cite{ibragimov2020automated,ZHU2022103991}, and pavement distress classification~\cite{c16,few_shot,dong2021automatic}. Among them, pixel-based pavement distress segmentation is a hot research field. Zhang et al.~\cite{crack500} leverage CNN to classify the image patch for segmenting pavement distress. In~\cite{c11}, a CNN is used to learn the structure of the cracks from raw images, then the segmentation result is generated by the obtained structure information. Based on the fully conventional network (FCN), Yang et al.~\cite{yang2019feature} fuse multiscale features from top-to-down for pavement crack segmentation. In DeepCrack~\cite{zou2018deepcrack}, multiscale deep convolutional features learned at hierarchical convolutional stages are fused together to capture the line structures. For distress localization, Ibragimov et al.~\cite{ibragimov2020automated} propose a method for localizing signs of pavement distress based on faster region based conventional neural network (Faster R-CNN). Zhu et al.~\cite{ZHU2022103991} compare the performance of three state-of-the-art object-detection algorithms on an Unmanned aerial vehicles(UAV) pavement image dataset, which includes six types of distress. Because pavement distress annotation requires professional knowledge and a large amount of time, the datasets used in the above methods are low-resolution and small-scale. However,  it remains to be determined whether models derived from small-scale datasets can be applied to real-world practice.

For pavement distress classification, Dong et al.~\cite{few_shot} propose a metric-learning based method for multi-target few-show pavement distress classification on the dataset which includes 10 different kinds of distress. In~\cite{dong2021automatic}, discriminative super-features constructed by the multi-level context information from the CNN is used to determine whether there is distress in the pavement image and recognize the type of the distress. All of these methods do a good job of classification on the dataset they use, which is small and only contains distressed images, and on which the test accuracy even achieves 100\%~\cite{dong2021automatic}. There have been few works to systematically evaluate the model's performance on a difficult large-scale multi-type dataset. Moreover, these approaches only regard the pavement distress detection or recognition problem as a common image classification problem and directly apply the classical deep learning approaches. In~\cite{c16}, patch-based weakly learning model IOPLIN and large-scale distress datasets CQU-BPDD are proposed to solve these problems. However, the main drawback of IOPLIN is that the patch label inference strategy based on the pseudo label makes IOPLIN incompatible with pavement recognition and its optimization process is quite complex and time-consuming. Our approach takes inspiration from IOPLIN but operates with different patch inference strategy, and uses more effective and hierarchical patch collection strategies.

\vspace{-0.2cm}
\subsection{Deep Learning-based Image Classification}
In recent years, due to the popularity of the ImageNet Large Scale Visual Recognition Challenge (ILSVRC)~\cite{ilsvrc}, many computer vision algorithms based on deep learning have emerged. Among them, a series of convolutional neural networks (CNNs) play a leading role in the field of image classification. I.e., AlexNet~\cite{alexNet} first applies the structure of convolutional neural networks to large-scale image classification datasets. Simonyan et al. first propose the deep and large-scale convolutional neural network VGGNet~\cite{c17} (i.e., the VGG19 model has 19 layers and more than 130 million parameters, while the previous convolutional neural network has less than 10 layers and millions to tens of millions of parameters). In InceptionNet~\cite{c18}, $1 \times 1$ convolution kernel application is proposed for the first time. He et al.~\cite{c19} propose a residual structure and network extension strategy to construct network family for the first time. Zoph et al.~\cite{mobilenets} bring CNN into the embedded mobile terminal and propose MobileNet specially designed for low computing power and low memory computing platform. Then based on MobileNet and Neural architecture search (NAS)~\cite{nas}, Tan et al. propose an efficient CNN, dubbed EfficientNet~\cite{c21}. In the past year, inspired by the field of Natural Language Processing (NLP), Dosovitskiya et al.~\cite{vit} propose a visual classification model based on Transformer~\cite{attention}, named as ViT.

However, the general classification models based on CNN and Transformer are difficult to be directly used in the field of pavement distress analysis. This is mainly because the pavement images have many specific characteristics, such as  high image resolution, the low distress area ratio, and uneven illumination, compared with object-centric natural images. This paper aims to use a patch collection strategy to incorporate both local and multiscale information for pavement distress analysis.

\section{Methodology}
\label{sec:format}
\begin{figure*}[t]
    \centering
    \includegraphics[scale=0.46]{Network.pdf}
    \caption{The overview of WSPLIN approaches. WSPLIN divides the images under varied scales into patches with different collection strategies, and then leverages a PLIN to produce the pseudo patch labels. Finally, CDN is used to yield the final pavement image label based on these patch labels.}
    \label{fig.network}
    \vspace{-0.5cm}
\end{figure*}
In this section, the network architecture of the Weakly Supervised Patch Label Inference Network (WSPLIN) is shown in Figure~\ref{fig.network}. We first introduce the problem formulation and overview of WSPLIN in section~\ref{sec:formulation}. Then, we introduce the involved patch collection strategies in section~\ref{sec:pc}. After that, the core modules of WSPLIN, Patch Label Inference Network (PLIN), and Comprehensive Decision Network (CDN) are detailed in section~\ref{sec:plin} and section~\ref{sec:cdn} respectively. Finally, we will show how to apply WSPLIN to detect and recognize the pavement distress in section~\ref{sec:pavement DR}.

\vspace{-0.2cm}
\subsection{Problem Formulation and Overview}
\label{sec:formulation}
Both pavement distress detection and recognition can be deemed as an image classification task from the perspective of computer vision. Let $X=\{x_1,\cdots,x_n\}$ and $Y=\{y_1,\cdots,y_n\}$ be the collection of pavement images and their pavement labels respectively. $y_i$ is a $C$-dimensional one-hot vector where $C$ is the number of categories and $y_{ij}$ indicates the $j$-th element of $y_i$. In the detection case, such a classification task is a binary image classification issue (distressed or normal) where $C=2$. In the recognition case, this classification task is a multi-class image classification problem where $C>2$. In a pavement label $y$, if the $j$-th element is the only nonzero element, it indicates that the corresponding pavement image belongs to the $j$-th category.
The pavement distress detection or recognition is to learn a classifier $F_{det}(\cdot)$ or $F_{reg}(\cdot)$ can label the pavement image correctly, $y_i\leftarrow F(x_i)$.

There are two strategies for accomplishing the pavement distress recognition task. One is the two-stage recognition flow path and the other is the one-stage recognition flow path. The two-stage recognition is to identify distressed images first via pavement distress detection, and then apply the pavement distress recognition to further classify each distressed image into a specific type of pavement distress. The one-stage recognition is to directly consider the normal case as an additional category in the recognition procedure, and therefore the pavement distress detection and recognition tasks are jointly tackled with one image classification model.

Similar to Iteratively Optimized Patch Label Inference Network (IOPLIN), WSPLIN is a patch-based pavement image classification method whose main obstacle is to train Patch Label Inference Network (PLIN) only with the image label. WSPLIN introduces an additional module named Comprehensive Decision Network (CDN) to guide the optimization of PLINs in an end-to-end weakly supervised learning manner. The flow path of WSPLIN is very concise. In WSPLIN, the pavement image is divided into several patches first, and then PLIN is used to infer the labels of these patches, finally the inferred labels are fed into CDN for yielding the final pavement label. Clearly, WSPLIN only has two core modules, namely PLIN and CDN, whose corresponding mapping functions are $\Gamma_\zeta(\cdot)$ and $\Theta_\xi(\cdot)$ respectively.

\vspace{-0.2cm}
\subsection{Patches Collection}
\label{sec:pc}

We adopt three different patch collection strategies for producing patches. They are Slide Window (SW), Image Pyramid (IP) and Sparse Sampling (SS).  The first strategy is also adopted by IOPLIN. WSPLIN uses the second strategy to fully exploit image information from different scales.  The third strategy is newly designed by us based on the IP strategy for speeding up WSPLIN via reducing the patch amount for training.

\textbf{Slide Window}:
The pavement image is simply divided into a series of uniform scale patches following non-overlapping strategy. We adopt $300\times300$ as the sliding window size with 300 sliding stride. The patch collection can be denoted as $P_i= \tau(x_i) = \{p^i_1, ..., p^i_m\}, m = 12$ where $\tau(\cdot)$ is our patch extraction operation.

\textbf{Image Pyramid}:
The slide window strategy does not consider the scale information. So we resize the pavement image into three resolutions, $300 \times 300$, $600\times 600$ and $1200 \times 900$ (the original size), to construct a three-layer image pyramid from top to down, and then employ sliding window method for dividing the image into patches. The patch collection can be denoted as $P_i= \{\tau(x_i^l)\}_{l\in\{0,1,2\}} = \{p^i_1, ..., p^i_m\}, m=\sum_l m_l = 17$ where $x_i^0=x_i$ and $l$ indicates the layer ID. Similar to the slide window strategy, we also apply $300\times300$ as the sliding window size with 300 sliding strides in the image pyramid. So, $m_0=12$, $m_1=4$, and $m_2=1$.

\textbf{Sparse Sampling}:
The patch number determines the scale of training data, and the patches in the same image pyramid also contain some redundant information in the scale space. Therefore, we can sample some patches for each image for reducing the training burden, and thereby speeding up the model.  More specifically, let $\alpha$ be the sparse sample ratio to control
the number of sampled patches for each layer, $n_l=\left \lceil m_{l}\times \left ( 1-\alpha  \right ) \right \rceil$ where $\left \lceil  \cdot \right \rceil$ returns the smallest integer that is greater than or equal to the input. We design a simple strategy for sampling patches in each layer. In this strategy, the sampled patches of all three layers should cover all scales while maximizing the spatial coverage. The optimal patch sparse sampling strategy is mathematically denoted as follows,
\begin{equation}
\setlength\abovedisplayskip{1pt
\setlength\belowdisplayskip{1pt}
   \{\hat{\mathcal{C}}_l\}_{l=0,1,2}\leftarrow\arg\underset{\{\mathcal{C}_l\}_{l=0,1,2}}\max { \text{Vol}(\bigcup_{l=0}^2\bigcup_{t\in \mathcal{C}_l} p_t)}, \quad \text{s.t.} |\mathcal{C}_l|=n_l,
\end{equation}
where $\text{Vol}(\cdot)$ returns the volume of the given set and $\mathcal{C}_l$ denotes an index subset to patches in $l$-th layer. Since the solutions of the above problem are limited, we can use the enumeration method to address this issue efficiently when $n_l$ is fixed. In this paper, we empirically set $\alpha = 0.25$. In such a manner, $n_0=1$, $n_1=1$ and $n_2=3$.

\emph{For distinguishing different versions of WSPLIN, WSPLIN-SW, WSPLIN-IP and WSPLIN-SS indicate the versions that use sliding window, image pyramid and sparse sampling patch collection strategies respectively. The default version of WSPLIN is WSPLIN-IP.}

\vspace{-0.2cm}
\subsection{Patch Label Inference Network}
\label{sec:plin}
Similar to IOPLIN~\cite{c16}, we adopt EfficientNet-B3~\cite{c21} as our Patch Label Inference Network (PLIN) due to its good trade-off between performance and efficiency. We denote $\Gamma_\zeta(\cdot)$ as the mapping function of PLIN. The patch label inference procedure is denoted as,
\begin{equation}
S_i = \Gamma_\zeta(\tau(x_i)),
\end{equation}
where $S_i$ is an $m\times C$-dimensional matrix in which every column encodes the label inference confidences of patches. Such confidences are expected to be zero if the patch does not exist the distress and reflect the possibility of the certain distress that the corresponding patch has. Note, there is no supervised information of patches, so all these labels are randomly produced just via forward propagation. We need to leverage the follow-up comprehensive decision network to guide the PLIN to generate reasonable patch labels with image-level labels in a weakly supervised manner. We will introduce such a procedure later.

\noindent\textbf{Patch Label Sparsity Constraint (PLSC):} Since the distressed area is often the small part of the pavement, and it is seldom that a pavement image has many different distress. In such a manner, the label confidence matrix $S_i$ should only have very limited nonzero elements. In other words, $S_i$ should be sparse. Thus, we introduce an $L_1$-norm constraint to the label confidence matrices of the distressed training samples,
\begin{equation}
    \mathcal{L}_s=\sum_{i \in \{i|y_i\neq y_{\text{normal}}\}}||S_i||_1,
\end{equation}
where $y_{normal}$ is the label of the normal pavement image. We introduce this constraint only to the distressed samples, since there should be no nonzero element in the label confidence matrices of the normal samples, $||S_i||_1=0$

\vspace{-0.2cm}
\subsection{Comprehensive Decision Network}
\label{sec:cdn}
We establish a Comprehensive Decision Network (CDN) for accomplishing the final pavement image classification based on the aforementioned patch label results. CDN consists of four layers where the first two layers are all the $m\times C$ fully connection layers followed by a ReLU, and Dropout layer, the third layer is also an $m\times C$ fully connection layer and the size of output fully connection layer is $C$. Here, $C$ is the number of categories. Let $\Theta_\xi(\cdot)$ be the mapping function of CDN, then the predicted pavement distress label $\hat{y}_i$ can be obtained by,
\begin{equation}
    \hat{y}_i = \Theta_\xi(\Gamma_\zeta(\tau(x_i)))=\Theta_\xi(S_i).
\end{equation}
We use the cross-entropy to measure the discrepancy between the predicted label and ground-truth and denote it as the classification loss $\mathcal{L}_c$,
\begin{equation}
    \label{eq:cel}
    \mathcal{L}_c = -\frac{1}{n}\sum_{i=1}^n\sum_{j=1}^C y_{ij}\log{\hat{y}_{ij}}.
\end{equation}
Finally, the optimal WSPLIN model is learned by minimizing the following loss,
\begin{equation}
    (\xi,\zeta)  \leftarrow\arg\underset{\hat{\xi},\hat{\zeta}}\min~~\mathcal{L}_{total}:=\mathcal{L}_c+\lambda\mathcal{L}_s,
\end{equation}
where $\lambda$ is a positive parameter for reconciling the classification loss and the sparsity constraint.

WSPLIN is an end-to-end deep learning framework that uses the back-propagation to compute the loss deviation and update the parameters layer by layer. In WSPLIN, CDN requires the patch label results produced by PLIN that should be useful for the final classification and the patch label sparsity constraint forces WSPLIN to highlight only a few of the most crucial patches for participating in the final decision. Clearly, these highlighted patches should be distressed ones and their inferred patch label results should be nonzero since only the distressed patches can provide helpful information for the final detection and recognition. In such a manner, CDN essentially guides the training of PLIN in a weakly supervised manner.

\vspace{-0.2cm}
\subsection{Pavement Distress Detection and Recognition}
\label{sec:pavement DR}

\textbf{Detection and One-Stage Recognition:}
The pavement detection and one-stage pavement distress detection can be deemed as a one-stage pavement image classification problem. To tackle these tasks, we can train our model as a pavement image classifier. Once the model is trained, the pavement image $x$ can be divided into patches with different patch collection strategies, which are fed into WSPLIN for yielding the final classification,
\begin{equation}
    y = F(x) = \Theta_\xi(\Gamma_\zeta(\tau(x))),
\end{equation}
where $F \in \{F_{det}, F_{reg}\}$ and the predicted category should be corresponding to the maximum element of $y$.

\textbf{Two-Stage Recognition:}
The two-stage recognition has two stages to accomplish the pavement distress recognition. The first stage is to train our model as a pavement distress detector $F_{det}$ for filtering out the normal samples and finding the distressed samples. The second stage is to train our model as a multi-class pavement image classifier $F_{reg}$ for completing the final distress recognition,
\begin{equation}
    y_i = F_{reg}(x_i) = \Theta_\xi(\Gamma_\zeta(\tau(x_i))),
\end{equation}
where $x_i \in \{x|F_{det}(x)\neq y_{\text{normal}}\}$ and the maximum element of $y_i$ reflects the specific pavement distress category of the distressed pavement image $x_i$.

\vspace{-0.2cm}
\section{Experiments and Results}
\vspace{-0.2cm}
\subsection{Dataset and Setup}
We test our method on pavement distress detection and recognition tasks under four application settings. The first one is the one-stage recognition (\textbf{I-REC}), which tackle the pavement distress detection and recognition tasks jointly. In this setting, all samples (including both the distressed and normal ones) and their fine-grained category label are available for training and testing model. Moreover, both the detection and recognition performances can be evaluated under this setting. The second one is the one-stage detection (\textbf{I-DET}), which is the conventional detection fashion. In this setting,  all samples (including both the distressed and normal ones) are involved, but only the binary coarse-grained category label (distressed or normal) is available. The other two settings are all from the two-stage recognition scenario.  One is the ideal second-stage recognition \textbf{II-REC(i)} which assumes all distressed samples are ideally detected via the first-stage detection. In this setting, the recognition models are only evaluated with distressed pavement images. The last setting is the normal second-stage recognition \textbf{II-REC(n)}. The training stage of \textbf{II-REC(i)} and \textbf{II-REC(n)} are identical. But their testing stages are different. In \textbf{II-REC(n)}, the recognition models are only evaluated on the images detected by the detection model trained in \textbf{I-DET}. In such a manner, the recognition error under this setting is the errors accumulated by both the first-stage detection and the second-stage recognition, since some distressed images may be incorrectly filtered out while some normal images may be incorrectly classified as distressed ones by the detector in the recognition testing stage under \textbf{II-REC(n)}. The results in \textbf{II-REC(n)} can reflect the comprehensive performances of two-stage recognition.

A large-scale bituminous pavement distress dataset named CQU-BPDD~\cite{c16} is used for evaluating the approaches under four application settings. This dataset involves seven different types of distress: alligator crack, crack pouring, longitudinal crack, massive crack, transverse crack, raveling, and mending. For settings of \textbf{I-DET}, \textbf{I-REC}, and \textbf{II-REC(n)}. We simply follow the data split strategy in \cite{c16}. With regard to the setting of \textbf{II-REC(i)}, 5140 distressed pavement images are randomly selected as the training set while the rest 11589 distressed pavement images are for testing. The detailed data split information of different settings are tabulated in Table~\ref{settings}.

Similar to IOPLIN, we adopt EfficientNet-B3 as the Patch Label Inference Network (PLIN). Since the comprehensive decision network (CDN) adopts a fully connected layer with fixed dimensions, WSPLIN requires the input size to be fixed at $300\times300$, and the optimizer uses RangerLars, which is just a combination of RAdam\cite{c24}, LookAhead\cite{c25} and LARS\cite{c26}. The learning rate is $8\times 10^{-4}$, and the cosine annealing strategy is adopted to adjust the learning rate: the learning rate remained unchanged in the first 25\% of the training process, and gradually decreased with the cosine function in the subsequent training process. Data augments such as rotation, flipping, and brightness balance are carried out for the raw images. The dropout rate of the classification layer is 0.5.

\begin{table}[tb]
    \small
    \caption{The detailed information of four different application settings on CQU-BPDD.} \label{settings}
    \small
    \begin{tabularx}{9cm}{p{1.4cm}p{1.8cm}<{\centering}p{0.9cm}<{\centering}p{0.6cm}<{\centering}p{0.9cm}<{\centering}p{0.6cm}<{\centering}}
        \toprule
        \multirow{2}{*}{Setting}& \multirow{2}{*}{Classifier}&\multicolumn{2}{c}{Train} & \multicolumn{2}{c}{Test} \\
        \cmidrule(r){3-4}\cmidrule(r){5-6}
        && \#Sample  & \#Class & \#Sample  & \#Class \\
        \midrule
        \textbf{I-DET} & Detector & 10137 & 2 & 49919 & 2\\[2pt]
        \hdashline[1.5pt/1.5pt]
        \textbf{I-REC} & I-Recognizer & 10137 & 8 & 49919 & 8 \Tstrut\\[2pt]
        \hdashline[1.5pt/1.5pt]
        \textbf{II-REC(i)} & II-Recognizer & 5140 & 7 & 11589 & 7  \Tstrut\\[2pt]
        \hdashline[1.5pt/1.5pt]
        \multirow{2}{*}{\textbf{II-REC(n)} } & Detector & 10137 & 2 & \multirow{2}{*}{49919} & \multirow{2}{*}{8} \Tstrut\\[1.5pt]
        &II-Recognizer& 5140 & 7 &  & \\[2pt]
        \bottomrule
        \end{tabularx}
    \vspace{-0.5cm}
\end{table}

\vspace{-0.2cm}
\subsection{Evaluation Metrics}
\subsubsection{Evaluation Metrics of Detection}
For pavement distress detection task, we adopt Area Under Curve (AUC) of Receiver Operating Characteristic (ROC)~\cite{auc}, which is common in binary classification tasks (this metric is not affected by classification threshold). It is mathematically defined as follows,
\begin{equation}\label{AUC}
    AUC=\frac{S_{p}-N_{p}\left ( N_{p}+1 \right )/2}{N_{p}N_{n}}
\end{equation}
where $S_{p}$ is the sum of the all positive samples ranked, while $N_{p}$ and $N_{n}$ denote the number of positive and negative samples.
Additionally, Binary $F1$ score, which is the harmonic mean of precision and recall, is used to measure the models more comprehensively. The binary $F1$ score is defined as:
\begin{equation}\label{f1_bin}\small
    F1_{binary}= \frac{2\times \textbf{P}\times\textbf{R}}{\textbf{P} + \textbf{R}},\quad \textbf{P}=\frac{TP}{TP+FP},\quad\textbf{R}=\frac{TP}{TP+FN},
\end{equation}
where $\textbf{P}$ is the precision while \textbf{R} is the recall.  $TP$, $FP$, and $FN$ are the numbers of true positives, false positives and false negatives respectively. The precision measures how many true positive samples are among the samples that are predicted as the positive samples. Similarly, recall measures how many true positive samples are correctly detected among all positive samples. Moreover, in the medical or pavement image analysis tasks, it is more meaningful to discuss the precision under the high recall, since the miss of the positive samples (the distressed sample) may lead to a more serious impact than the miss of the negative ones.
\subsubsection{Evaluation Metrics of Recognition}
For pavement distress recognition task, we mainly use the Top-1 accuracy and Marco $F1$ score to evaluate the performance of models. Top-1 accuracy mainly measures the overall accuracy of the models, while Marco $F1$ score evaluates the accuracy of the model across different categories. The Macro $F1$ score can be mathematically represented as follows,
\begin{equation}\label{f1_marco}
    F1_{marco}=\frac{1}{c}\sum_{i}^{c} F1_{binary}^{i},
\end{equation}
where $F1_{binary}^{i}$ indicates the binary $F1$ score of the $i$-th category, and $c$ is the total number of categories.

\textbf{Note:} The $F1$ represents $F1_{binary}$ and $F1_{marco}$ in pavement distress detection and recognition tasks respectively.

\vspace{-0.2cm}
\subsection{Baselines}
Histogram of Oriented Gradient (HOG)~\cite{c27}, Local Binary Pattern (LBP)~\cite{lbp}, Fisher Vector(FV)~\cite{fv}, Support Vector Machine (SVM)~\cite{c23},  ResNet-50~\cite{c19}, Inception-v3~\cite{c13}, VGG-19~\cite{c17}, ViT-S/16~\cite{vit}, ViT-B/16~\cite{vit}, EfficientNet-B3~\cite{c21}, and Iterative Optimized Patch Label Inference Network (IOPLIN)~\cite{c16} are selected as baselines. The first four approaches are the shallow learning-based approaches. ResNet-50, Inception-v3, VGG-19, and EfficientNet-B3 are the classical Convolutional Neural Network (CNN) models. ViT-S/16 and ViT-B/16 are the recently popular transformer models. IOPLIN is a well elaborated pavement distress detection approach.
\begin{table}[htb]
    \small
    \caption{The pavement distress detection performances of different approaches on the CQU-BPDD detection benchmark. P@R = $n$\% indicates the precision when the corresponding recall is equal to $n$\%. }
    \centering
    \begin{tabularx}{9cm}{l p{0.6cm}<{\centering} X<{\centering}X<{\centering}p{0.6cm}<{\centering}}
    \toprule

     Detectors(\textbf{I-DET}) & AUC & P@R=90\% & P@R=95\% & $F1$\\
  \midrule
    HOG+PCA+SVM~\cite{c27}& 77.7\% &31.2\%&28.4\%& -\\[1.5pt]
    LBP+PCA+SVM~\cite{lbp}&82.4\% &34.9\%&30.3\%& -\\[1.5pt]
    HOG+FV+SVM~\cite{fv}&88.8\% &43.9\%&35.4\%& -\\[1.5pt]
    ResNet-50~\cite{c19}  & 90.5\% & 45.0\% & 35.3\% & -\\[1.5pt]
    Inception-v3~\cite{c13} & 93.3\% & 56.0\% & 42.3\% & -\\[1.5pt]
    VGG-19~\cite{c17}  & 94.2\% & 60.0\% & 45.0\% & -\\[1.5pt]
    ViT-S/16~\cite{vit}  & 95.4\% & 67.7\% & 51.0\% & 81.1\%\\[1.5pt]
    ViT-B/16~\cite{vit}  & 96.1\% & 71.2\% & 56.1\% & 80.6\%\\[1.5pt]
    EfficientNet-B3~\cite{c21} & 95.4\% & 68.9\% & 51.1\% & 81.3\%\\[1.5pt]
    IOPLIN~\cite{c16} & 97.4\% & 81.7\% & 67.0\% & 85.3\%\\[1.5pt]
    {\bf WSPLIN-IP} & {\bf 97.5\%} & {\bf 83.2\%} & {\bf 69.5\%} & {\bf 86.4\%}\\[1.5pt]
     \midrule
    Recognizers(\textbf{I-REC}) & AUC & P@R=90\% & P@R=95\% & $F1$\\[2pt]
      \midrule
    EfficientNet-B3~\cite{c21}  & 96.0\% & 77.3\% & 59.9\% & 83.2\%\\[1.5pt]
    {\bf WSPLIN-IP} & {\bf 97.6\%} & {\bf 85.3\%} & {\bf 72.6\%} & {\bf87.4}\%\\[1.5pt]
    \bottomrule
    \end{tabularx}
    \label{tab:compare_binary}
    \vspace{-0.5cm}
\end{table}

\vspace{-0.4cm}
\subsection{Pavement Distress Detection}
Table~\ref{tab:compare_binary} reports pavement distress detection performances of different approaches. These approaches include the detectors trained under \textbf{I-DET} and the recognizers trained under \textbf{I-REC} where recognizers address the detection issue along with the recognition task. Based on observations, WSPLIN-IP outperforms all baselines in all evaluation metrics under both \textbf{I-DET} and \textbf{I-REC}. In \textbf{I-DET}, WSPLIN-IP improves the performances of IOPLIN by 0.1\%, 1.5\%, 2.5\%, and 1.1\% in AUC, P@R=90\%, P@R=95\%, and F1-score respectively. In \textbf{I-REC}, WSPLIN-IP achieves 1.6\%, 8.0\%, 12.7\%, and 4.2\% performance gains over EfficientNet-B3 in AUC, P@R=90\%, P@R=95\%, and F1-score respectively. Moreover, the methods under \textbf{I-REC} consistently perform much better than the ones under~\textbf{I-DET}. For example, the WSPLIN-IP trained under \textbf{I-REC} achieves 0.1\%, 2.1\%, 3.1\%, and 1.0\% performance gains than the WSPLIN-IP trained under \textbf{I-DET} in AUC, P@R=90\%, P@R=95\%, and F1-score respectively. Similarly, the gains of EfficientNet-B3 are 0.6\%, 8.4\%, 8.8\%, and 1.9\%.  We attribute this to the fact that recognizers trained under \textbf{I-REC} utilize fine-grained distress labels instead of binary distress labels for training the pavement image classification models. It reflects that the much finer-grained supervised information, such as the specific pavement distress information, can benefit the pavement distress detection.

\begin{table}[tb]
    \small
    \centering
    \caption{The pavement distress recognition performances of different methods under different settings, namely \textbf{I-REC}, \textbf{II-REC(n)} and \textbf{II-REC(i)}. "Para." indicates the parameter scale of the deep learning model. Top-1 indicates the top-1 accuracy.}
    \begin{tabularx}{9cm}{p{3.3cm} X<{\centering}X<{\centering}X<{\centering} }
    \toprule
    Recognizers(\textbf{I-REC})  &Para. & Top-1& $F1$\\\midrule
ResNet-50 \cite{c19}&23M & 88.3\%  & 60.2\%    \\[1.5pt]
VGG-16 \cite{c17}&134M & 87.7\%  & 58.4\%      \\[1.5pt]
ViT-S/16~\cite{vit}&22M & 86.8\%  & 59.0\%    \\[1.5pt]
ViT-B/16~\cite{vit}&86M & 88.1\%  & 61.2\%    \\[1.5pt]
Inception-v3 \cite{c13}&22M & 89.3\%  & 62.9\%\\[1.5pt]
EfficientNet-B3~\cite{c21}&11M & 88.1\% &63.2\%\\[1.5pt]
\textbf{WSPLIN-IP}&\textbf{11M} & \textbf{91.1\%} &\textbf{66.3\%}\\[1.5pt] \midrule

    Recognizers(\textbf{II-REC(n)})  &Para. & Top-1& $F1$\\
    \midrule
        ResNet-50 \cite{c19}&23M & 82.8\%  & 53.6\%    \\[1.5pt]
    VGG-16 \cite{c17}&134M & 86.5\%  & 55.0\%      \\[1.5pt]
    ViT-S/16~\cite{vit}&22M & 86.7\%  & 56.6\%    \\[1.5pt]
    ViT-B/16~\cite{vit}&86M & 87.5\%  & 59.6\%    \\[1.5pt]
    Inception-v3 \cite{c13} &22M& 88.3\%  & 59.8\%     \\[1.5pt]
    EfficientNet-B3 \cite{c21}&11M & 88.9\%  & 61.2\%    \\[1.5pt]
    \textbf{WSPLIN-IP}    & \textbf{11M}        & \textbf{90.0\%}  & \textbf{64.5\%}    \\[1.5pt]

    \midrule
     Recognizers(\textbf{II-REC(i)})  &Para. & Top-1& $F1$\\
    \midrule

    ResNet-50 \cite{c19}&23M & 71.2\%  & 61.5\%    \\[1.5pt]
    VGG-16 \cite{c17}&134M & 74.6\%  & 65.0\%      \\[1.5pt]

    ViT-S/16~\cite{vit}&22M & 75.0\%  & 64.9\%    \\[1.5pt]
    ViT-B/16~\cite{vit}&86M & 75.3\%  & 67.0\%    \\[1.5pt]
    Inception-v3 \cite{c13} &22M& 77.6\%  & 69.8\%     \\[1.5pt]
    EfficientNet-B3 \cite{c21}&11M & 78.6\%  & 70.3\%    \\[1.5pt]
    \textbf{WSPLIN-IP}    & \textbf{11M}        & \textbf{85.0\%}  & \textbf{77.2\%}    \\[1.5pt]
    \bottomrule
    \end{tabularx}
    \vspace{-0.5cm}
\label{rec}
\end{table}

\vspace{-0.5cm}
\subsection{Pavement Distress Recognition}
Table~\ref{rec} records the pavement distress recognition performances and parameter scales of different approaches under different application settings on the CQU-BPDD dataset. Similar to the pavement distress detection performances, WSPLIN-IP achieves better recognition performances than baselines under all settings but enjoys the smaller parameter scale. In \textbf{I-REC}, the performance gains of WSPLIN-IP over Inception-v3, which is the second-best method, are 1.8\% and 3.4\% in top-1 accuracy and F1-score, respectively. In~\textbf{II-REC(n)} and \textbf{II-REC(i)}, the EfficientNet-B3 achieves the second-best performances. The performance gains of WSPLIN-IP over it under \textbf{II-REC(n)} are 1.1\% and 3.3\% in top-1 accuracy and F1-score, respectively. Such gains under \textbf{II-REC(i)} are 6.4\% and 6.9\%. The distribution of different pavement image categories are imbalanced. Top-1 accuracy is sensitive to this data imbalance while F1-score is more stable to this imbalance. Therefore, F1-score can better reflect the comprehensive performances of recognizers. According to the observations, WSPLIN-IP shows more advantages compared with baselines in F1-score.

The test settings of \textbf{I-REC} and \textbf{II-REC(n)} are identical as seen in Table~\ref{settings}. However, the models trained under~\textbf{I-REC} outperform the ones of \textbf{II-REC(n)}. For example, Inception-v3, ViT-B/16, EfficientNet-B3, and WSPLIN-IP trained under \textbf{I-REC} achieve 3.1\%, 1.6\%, 2.0\%, and 1.8\% improvements over the ones under \textbf{II-REC(n)} in F1-score. This implies that the end-to-end pavement distress recognition solution, which addresses detection and recognition tasks jointly (\textbf{I-REC}), enjoys more advantages than the conventional two-stage implementation solution, which addresses detection and recognition tasks individually (\textbf{II-REC(n)}), in the real-world application. We attribute this to that the end-to-end solution exploits the complementarity of these two tasks and introduces global optimization.

An interesting phenomenon is observed from Table~\ref{rec} that the top-1 accuracies of \textbf{II-REC(i)} are lower than the ones of the rest two settings while its F1-scores are higher than the F1-scores of the rest settings. This is because the test setting of \textbf{II-REC(i)} is different from the settings of \textbf{I-REC} and \textbf{II-REC(n)}, which does not involve any normal pavement sample. Top-1 accuracy is measured sample-wise but F1-score is measured category-wise. The normal samples comprise a large proportion of the whole data in \textbf{I-REC} and \textbf{II-REC(n)}. So, the superabundant normal samples will make the recognizers trained under \textbf{I-REC} bias to the classification of the normal sample, which leads to the high top-1 accuracy but the low F1-score. With regard to \textbf{II-REC(n)}, the massive normal samples push up the top-1 accuracy. However, the measure of F1-score is independent of the sample amount and the classification error of \textbf{II-REC(n)} is accumulated from both the detection and recognition stages. Therefore, it achieves a lower F1-score in comparison with \textbf{II-REC(i)}.

\vspace{-0.2cm}
\subsection{Ablation Study}
In this section, we will systematically discuss the effects of different components and different hyperparameters on our model. Table~\ref{ablation} records the performances and efficiencies of our approaches under different settings in comparison with IOPLIN.
\begin{table}[b]
\vspace{-0.3cm}
    \small
    \caption{The ablation study results of the proposed approaches under the same memory in different settings and evaluation metrics. (w/o indicates without and PLSC is the patch label sparsity constraint.)}
 \begin{tabular}{p{3.15cm}p{1.3cm}<{\centering}p{1.2cm}<{\centering}c}
        \toprule
        \textbf{Method} & \tabincell{c}{\textbf{I-DET}\\(P@R=90\%)} & \tabincell{c}{\textbf{I-REC} \\($F1$)}& \tabincell{c}{\textbf{TrainTime}\\(Reduction)}\\
        \midrule
        IOPLIN &  81.7\% &  - &12.5h\\%[1.5pt]

        WSPLIN-SW &  80.9\% &  64.4\% &9.6h (\textcolor{blue}{-23\%})\\%[1.5pt]

        WSPLIN-IP w/o PLSC & 81.2\%&64.5\% &11.0h (\textcolor{blue}{-12\%}) \\%[1.5pt]
        WSPLIN-IP &  \textbf{83.2\%} &  \textbf{66.3\%} &11.1h (\textcolor{blue}{-11\%})\\%[1.5pt]

        WSPLIN-SS ($\alpha = 0.25$) &  81.1\% &  64.1\%&  \textbf{3.2h (\textcolor{blue}{-74\%})}\\%[1.5pt]

       WSPLIN-SS  ($\alpha = 0.50$) &  81.4\% & 64.9\% &5.7h (\textcolor{blue}{-54\%})\\%[1.5pt]
        WSPLIN-SS  ($\alpha = 0.75$) &  80.0\% &  63.7\% & 8.4h (\textcolor{blue}{-33\%})\\%[1.5pt]
        \bottomrule
        \end{tabular}
    \label{ablation}
\end{table}

\subsubsection{Discussion on Patch Collection Strategies}We adopt three strategies named Slide Window (SW), Image Pyramid (IP), and Sparse Sampling (SS) to collect the patches from pavement images. Their corresponding versions are WSPLIN-SW, WSPLIN-IP, and WSPLIN-SS respectively. In all three versions, WSPLIN-IP, which is also the default version of WSPLIN, achieve the best performances under two application settings with different evaluation metrics. WSPLIN-IP achieves 1.5\% performance gains in P@R=90\% in the pavement distress detection case. However, its training time is only 89\% of the training time of IOPLIN. In comparison with WSPLIN-SW, WSPLIN-IP not just exploits the local information but also exploits the scale information of pavement image. The results indicate that such scale information can further improve the performance of WSPLIN. Although WSPLIN-SW has not outperformed IOPLIN with a 0.8\% performance decrease in pavement distress detection, WSPLIN-SW is much faster than WSPLIN-IP and its training only takes around 3/4 of the training time of IOPLIN. We attribute this to the efficiency advantage of the end-to-end model optimization strategy. Moreover, compared with other versions, WSPLIN-SW has not suffered from the scale variation, so it can produce better patch label inference visualization results and thereby enjoys the better interpretability.

WSPLIN-SS also takes the scale information into consideration, and can be deemed as a speedy version of WSPLIN-IP. The best performed WSPLIN-SS ($\alpha = 0.5$) achieves the similar performance as IOPLIN where $\alpha$ is a hyperparameter to control the number of sampled patches in each layer of an image pyramid. However, WSPLIN-SS saves around half of the training time compared with IOPLIN, and it is only 76\% of the training time of WSPLIN-IP. Clearly, WSPLIN-SS highly speeds up WSPLIN only with an acceptable performance decrease. Another interesting phenomenon is observed that WSPLIN-SS with a higher $\alpha$ does not always enjoy better performance. Generally, a higher $\alpha$ implies collecting more patches which means more information can be preserved for classification. However, the results indicate that not all preserved information is necessary for classification. Moreover, the less amount of patches per image means more images can be taken into one batch for model optimization since the memory size is fixed in our case. The higher diversity of pavement images in each batch benefits the model optimization. A good sparse sampling strategy should optimize the trade-off between patch preservation and the diversity of samples in the same batch. We believe this is the reason why WSPLIN-SS ($\alpha = 0.50$) performs well in both tasks.

In summary, all WSPLIN approaches show prominent advantages in training efficiency with similar or even better performances. We recommend using WSPLIN-IP in the application scenarios, which pay more attention to the performance instead of the efficiency, while using WSPLIN-SS in the application scenarios, which needs to take both performance and efficiency into considerations. WSPLIN-SW is recommended in the application scenarios, which pay more attention to the visual analysis of distressed images.

\subsubsection{Discussion on Patch Label Sparsity Constraint}
The distressed area is often a small proportion over the whole image. So we introduce the Patch Label Sparsity Constraint (PLSC) to model and leverage this prior property for better addressing the pavement image classification issue. Table~\ref{ablation} reports the performances of WSPLIN-IP with and without PLSC. WSPLIN-IP with PLSC achieves 2.0\% more accuracies in P@R=90\% under \textbf{I-DET} and  1.8\% greater F1-scores under \textbf{I-REC} over WSPLIN without PLSC. This implies that PLSC can offer a considerable improvement of WSPLIN. We also leverage Grad-CAM~\cite{grad_cam} to plot the Class Activation Maps (CAM) of the features extracted by the WSPLIN-IP models before and after using PLSC in Figure~\ref{pic_heatmap}. The CAM visualization results also validate that PLSC benefits the distressed feature extraction.

$\lambda$ is a positive hyperparameter for reconciling the classification loss and the PLSC. Figure~\ref{pic_lambda} plots the relationships between the different values of $\lambda$ and the performances of WSPLIN-IP under \textbf{I-DET} and \textbf{I-REC}. From observations, we can find that the WSPLIN-IP is insensitive to the setting of $\lambda$. The optimal $\lambda$ is $10^{-3}$.

\begin{figure}[tb]

     \begin{center}
    \includegraphics[scale=0.35]{./PLSC_CAM.pdf}
    \caption{The Class Activation Map (CAM) Visualizations of the WSPLIN-IP models trained with or without PLSC. From the left column to the right column respectively indicate the ground truth, CAMs of baseline, CAMs of WSPLIN-IP without PLSC, and CAMs of WSPLIN-IP with PLSC.}
    \label{pic_heatmap}
    \vspace{-0.4cm}
    \end{center}
\end{figure}

\begin{figure}[tb]

    \begin{center}
    \includegraphics[scale=0.5]{./hyperpara.pdf}

    \vspace{-0.2cm}
        \caption{ The impacts of different $\lambda$ to the performance of WSPLIN-IP under \textbf{I-DET} and \textbf{I-REC}.}
           \label{pic_lambda}
    \end{center}
    \vspace{-0.6cm}
\end{figure}

\subsubsection{The Efficiency of WSPLIN}
According to observations in Table~\ref{ablation}, all WSPLIN approaches are more efficient than IOPLIN. Moreover, IOPLIN and WSPLIN have very similar network structure, so they have the same parameter scale.

\vspace{-0.2cm}
\subsection{User Scenarios}
WSPLIN has wider application scenarios in comparison to IOPLIN. IOPLIN can only address the pavement distress detection problem, which is a typical binary image classification issue and attempts to find the distressed samples only. WSPLIN can tackle both the pavement distress detection and the recognition tasks under various aforementioned application settings shown in Table~\ref{settings}. The pavement distress recognition task is a multi-class image classification task, which tries to classify the pavement image into the specific distress category. The usages of WSPLIN and IOPLIN are similar. Once the corresponding models are trained, we can input pavement images into the models for acquiring their detection or recognition labels. For more details about their usages, please refer to~\cite{c16}. Similar to IOPLIN, WSPLIN can also roughly localize the distressed area of a pavement images. The main difference between IOPLIN and WSPLIN in such process is that WSPLIN can further recognize the diseases in those distressed areas. Figure~\ref{fig:visualization} gives some examples of this scenario via visualizing the patch labels produced by the trained WSPLIN.
\begin{figure}[tb]
    \centering
    \includegraphics[width=9cm, keepaspectratio]{ROI_vertical.jpg}
    \caption{The visualizations of pavement images labeled by WSPLIN in the patch level.}
    \label{fig:visualization}
    \vspace{-0.4cm}
\end{figure}

\vspace{-0.2cm}
\section{Conclusions}
\label{sec:conclusions}
In this paper, we present a novel patch-based deep learning model named WSPLIN for automatic pavement distress detection and recognition in the wild. WSPLIN divides the pavement image into patches with different patch collection strategies and then learns the label of patches in a weakly supervised manner. Finally, these inferred patch labels are fed into a comprehensive decision network for yielding the final recognition results. Similar to IOPLIN, WSPLIN can sufficiently utilize the resolution and scale information, and can also provide interpretable information, such as the location of the distressed area. However, WSPLIN is more efficient than IOPLIN with similar or even better performance. The experiments on a large pavement distress dataset validate the effectiveness of our approach.

\footnotesize
\bibliographystyle{IEEEtran}