\section{Introduction} \label{Introduction}
Many biological and biomimetic structures use geometrically pronounced features to produce highly nonlinear behavior. These materials include seashells, hierarchical honeycombs, snail spiral, seahorse tail, fish scales, lobster exoskeleton, crab exoskeleton, butterfly wings, armadillo exoskeleton, sponge skeleton, etc. \cite{c07,c08,c09,c10,c11}. Among these structures, dermal scales have garnered special attention recently due to complex mechanical behavior in bending and twisting \cite{c12,c13,c14,c15,c16,c17,c18}. Scales in nature are naturally multifunctional, durable and lightweight \cite{c19,c20,c21,c22,c23,c24,c25,c26,c27,c28}, and protective for the underlying substrate, which has been an inspiration of armor designs \cite{c17,c18,c29,c30} where overlapping scales can resist penetration and provide additional stiffness \cite{c17,c18,c31,c32}. Fabrication methods such as synthetic mesh sewing and stretch-and-release have been recently developed to produce overlapping scale-covered structures in 2D and 1D configuration \cite{c33,c34}. These fabricated structures show almost ten times more puncture resistance than soft elastomers.

However, in addition to these localized loads, global deformation modes such as bending and twisting can be important for a host applications that require a structural mode of deformation such as soft robotics, prosthetics or morphing structures. It is here that characterizing bending and twisting play an important role in ascertaining the benefit of these structures. Prior research has shown that bending and twisting of a substrate show small strain reversible nonlinear stiffening and locking behavior due to the sliding kinematics of the scales in one-dimensional substrates \cite{c35,c36,c37,c38,c39,c40,c41,c42,c43,c44,c48}. The universality of these behavior across bending of uniformly distributed scales, functionally graded scales and uniformly distributed twisting is an important discovery. However, the role of friction and its possible universal role has not been established in literature. In other words, questions remain about the parallels of properties modification brought about by friction in bending with twisting.

For instance, Coulomb friction in bending regime advances the locking envelopes but at the same time, limits the range of operation \cite{c39}. In the dynamic regime, Coulomb friction can lead to damping behavior, which mimics viscous damping \cite{c44}. Clearly, friction between sliding scales can significantly alter the nature of nonlinearity. However, in spite of these studies, the role of friction in influencing the twisting behavior has never been investigated before.

In this paper we investigate the role of friction in affecting the twisting behavior of biomimetic scale-covered systems under pure torsion for the first time. To this end, we establish an analytical model aided by finite element (FE) computational investigations. We assume rigid scales, linear elastic behavior of the substrate and Coulomb model of friction between scales’ surfaces. We compare our results with FE model to verify the proposed analytical model.

\section{Materials and methods} \label{Materials and methods}

\subsection{Materials and geometry} \label{Materials and geometry}

We consider a rectangular deformable prismatic bar with a row of rigid rectangular plates embedded on substrate’s top surface. For the sake of illustration, we fabricate prototypes of 3D-printed PLA scales ($E_{PLA}\sim3$ $GPa$), embedded onto a silicone substrate and adhered with silicone glue to prefabricated grooves on the molded slender Vinylpolysiloxane (VPS) substrate ($E_{VPS}\sim1.5$ $MPa$) as shown in Figure \ref{Fig1a}. The prototype has been shown under twisting configuration in Figure \ref{Fig1b}. The rigidity assumption is valid in the limit of much higher stiffness of the scales, away from the locking state \cite{c18,c45}.

\begin{figure}[htbp]%
\centering
\subfigure[][]{%
\label{Fig1a}%
\includegraphics[width=3.3in]{Fig1a.pdf}}%
\hspace{8pt}%
\subfigure[][]{%
\label{Fig1b}%
\includegraphics[width=3.3in]{Fig1b.pdf}} \\
\caption[]{The fabricated prototype made of 3D-printed PLA scales and molded slender Vinylpolysiloxane (VPS) substrate:
\subref{Fig1a} untwisted configuration; and
\subref{Fig1b} twisted configuration.}%
\label{Fig1}%
\end{figure}

The pure twisting behavior allows us to assume periodicity, letting us isolate a fundamental representative volume element (RVE) for modeling the system, Figure \ref{Fig2a}. The scales are considered to be rectangular rigid plates with thickness $t_s$, width $2b$, and length $l_s$, and oriented at angles $\theta$ and $\alpha$ as shown in Figure \ref{Fig2b} with respect to the rectangular prismatic substrate. $\theta$ is the scale inclination angle defined as the dihedral angle between the substrate’s top surface and the scale’s bottom surface, and $\alpha$ is the angle between the substrate’s cross section and the scale’s width. The length of exposed section of scales is denoted as $l$, and the length of embedded section of the scales is $L$. Therefore, the total length of the scale is $l_s=L+l$. The spacing between the scales is constant and denoted by $d$, which is a geometrical parameter reciprocal to the density of scales. We assume that the scale’s thickness $t_s$ is negligible with respect to the length of the scales is $l_s$ ($t_s \ll l_s$), and the scale’s embedded length is also negligible with respect to the substrate’s thickness ($0 \ll L \ll 2t$). This thin-plate idealization for the biomimetic scales is appropriate for this case and typically used in literature for analogous systems \cite{c36,c39,c40,c41,c42,c43,c44}.

\begin{figure}[htbp]%
\centering
\subfigure[][]{%
\label{Fig2a}%
\includegraphics[width=3.3in]{Fig2a.pdf}}%
\hspace{8pt}%
\subfigure[][]{%
\label{Fig2b}%
\includegraphics[width=3.3in]{Fig2b.pdf}} \\
\caption[]{The schematic of three consecutive scales geometrical configuration:
\subref{Fig1a} top view of scales configuration; and
\subref{Fig1b} dimetric view to represent scales orientational angles of $\theta$ and $\alpha$, and the embedded part of the each scales. Angle $\theta$ and thickness $t_s$ are exaggerated here.}%
\label{Fig2}%
\end{figure}

\subsection{Kinematics} \label{Kinematics}

For global deformation modes such as pure bending and twisting the scale periodicity is a good approximation \cite{c36,c42}. Periodicity assumption allows us to consider just three consecutive scales configuration at the RVE level, We call these scales as ``zeroth scale", ``1\textsuperscript{st} scale", and ``2\textsuperscript{nd} scale" respectively from left to right. Without loss of generality, we consider 1\textsuperscript{st} scale is fixed locally with respect to other scales. A twisting deformation with twist rate $\Upphi$, is applied to the rectangular prismatic substrate about torsion axis, which passes through the beam cross section center. Due to this underlying deformation, the 2\textsuperscript{nd} scale rotates by twist angle of $\varphi=\Upphi d$, and the zeroth scale rotates in reverse direction about the torsion axis with $-\varphi=-\Upphi d$, because 1\textsuperscript{st} scale assumed locally fixed. The continual twisting of the substrate progresses the contact between each two consecutive scales simultaneously due to periodicity, by coincidence between lines $C_1B_1$ and $D_2C_2$, as well as lines $D_1C_1$ and $C_0B_0$.

To find a contact criterion between 1\textsuperscript{st} scale and 2\textsuperscript{nd} scale, the 3D-equations of lines $C_1B_1$ and $D_2C_2$ would be established. We place the coordinates $XYZ$ on the midpoint of 1\textsuperscript{st} scale's width as shown in Figure \ref{Fig2a}. Then we place coordinates $xyz$ on the torsion axis at point $O=(0,-t,0)$ measured form the coordinates $XYZ$. Hereafter, coordinates $xyz$ is our reference frame. We establish a local coordinates on each scales denoted as ``$i$\textsuperscript{th} scale" and its origin is located on the corner of the scale at point $D_i$. In these local coordinates, the unit vector of $x$-axis ($\bi{{n}_{Xi}}$) is on the edge $D_iC_i$, the unit vector of $y$-axis ($\bi{{n}_{Yi}}$) is on the edge $D_iA_i$, and the unit vector of $z$-axis ($\bi{{n}_{Zi}}$) is out of plane and perpendicular to $\bi{{n}_{Xi}}$ and $\bi{{n}_{Yi}}$, Figure \ref{Fig2b}. On each scales, edges $D_iC_i$ and $A_iB_i$ are parallel and in direction of $\bi{{n}_{Xi}}$, and edges $C_iB_i$ and $D_iA_i$ are parallel and in direction of $\bi{{n}_{Yi}}$. Point $M_i$ is located in the middle of edge $C_iB_i$. Using these established coordinates, symmetric equations of line $C_1B_1$ of 1\textsuperscript{st} scale is as follows \cite{c46}:
\vspace{0.7pc}
\begin{equation} \label{Eq1}
\frac{x-x_{M_1}}{x_{\bi{{n}_{Y1}}}}=\frac{y-y_{M_1}}{y_{\bi{{n}_{Y1}}}}=\frac{z-z_{M_1}}{z_{\bi{{n}_{Y1}}}},
\end{equation}
\vspace{0.7pc}
where $\bi{{n}_{Y1}}=(x_\bi{{n}_{Y1}},y_\bi{{n}_{Y1}},z_\bi{{n}_{Y1}})$. By putting (\ref{Eq1}) equal to $p$ and using geometrical parameters in Figure \ref{Fig2}, we will have parametric form of the equation of line $C_1B_1$ as follows, where $p$ can vary from $-b$ to $b$:
\vspace{0.7pc}
\numparts \begin{eqnarray}
x(p) = p\cos \alpha - l\sin \alpha \cos \theta, \label{Eq2a} \\
y(p) = t + l\sin \theta, \label{Eq2b}\\
z(p) = p\sin \alpha + l\cos \alpha \cos \theta. \label{Eq2c}
\end{eqnarray} \endnumparts
\vspace{-0.7pc}

Point $D_i$ is located at one end of the edge $D_iC_i$. Symmetric equations of line $D_2C_2$ of 2\textsuperscript{nd} scale is as follows:
\vspace{0.7pc}
\begin{equation} \label{Eq3}
\frac{x-x_{D_2}}{x_{\bi{{n}_{X2}}}}=\frac{y-y_{D_2}}{y_{\bi{{n}_{X2}}}}=\frac{z-z_{D_2}}{z_{\bi{{n}_{X2}}}},
\end{equation}
\vspace{0.7pc}
where $\bi{{n}_{X2}}=(x_\bi{{n}_{X2}},y_\bi{{n}_{X2}},z_\bi{{n}_{X2}})$. To find parametric equation of the line $D_2C_2$, which is on the 2\textsuperscript{nd} scale rotating with angle $\varphi$ about torsion axis, first we locate the corners of 2\textsuperscript{nd} as shown in Figure \ref{Fig2a}, and then their locations are found after rotation, using rotation matrix. Therefore, rotated local coordinates on this scale and the unit vector in direction $D_2C_2$ ($\bi{{n}_{X2}}$) can be established. By using these geometrical parameters and putting (\ref{Eq3}) equal to $q$, we have parametric form of the equation of line $D_2C_2$ as follows, where $q$ can vary from $0$ to $l$:
\vspace{0.7pc}
\numparts \begin{eqnarray}
x(q) = ( {\tan \theta \tan \varphi - \sin \alpha } )q + ( {t\sin \varphi - b\cos \alpha \cos \varphi }), \label{Eq4a} \\
y(q) = ( {\tan \theta + \sin \alpha \tan \varphi } )q + ( {t\cos \varphi + b\cos \alpha \sin \varphi }), \label{Eq4b} \\ z(q) = ( {\frac{{\cos \alpha }}{{\cos \varphi }}} )q + ( {d - b\sin \alpha } ). \label{Eq4c}
\end{eqnarray} \endnumparts
\vspace{-0.2pc}

To find a contact point between these two lines, (2) and (4) must be identical at $x$, $y$ and $z$ coordinate simultaneously. By putting (\ref{Eq2a}) equal to (\ref{Eq4a}) and also (\ref{Eq2b}) equal to (\ref{Eq4b}) simultaneously, we arrive at the following systems of equations:
\vspace{0.7pc}
\begin{equation} \label{Eq5}
\left[\begin{array}{cccc} x_{\bi{{n}_{Y1}}} & -x_{\bi{{n}_{X2}}} \\ y_{\bi{{n}_{Y1}}}& -y_{\bi{{n}_{X2}}}\end{array} \right]  \left[\begin{array}{cccc} p \\ q\end{array} \right] = \left[\begin{array}{cccc} x_{C_2}-x_{M_1} \cr y_{C_2}-y_{M_1}\end{array} \right].
\end{equation}
\vspace{0.7pc}

Solving (\ref{Eq5}) will lead us to equations for $p$ and $q$, and by putting derived equation of $p$ or $q$, into the (\ref{Eq2c}) or (\ref{Eq4c}), yields to an analytical relationship between $\varphi$ and $\theta$. To represent a general form for this relationship, we define dimensionless geometric parameters including $\eta=l/d$, $\beta=b/d$, and $\lambda=t/d$ as the overlap ratio, dimensionless scale width, and dimensionless substrate thickness, respectively. The governing nonlinear relationship between the substrate twist angle $\varphi$ and the scale inclination angle $\theta$ can be written as:
\vspace{0.7pc}
\begin{eqnarray} \label{Eq6}
\fl (\cos \varphi - 1) {\Big( \beta \sin 2\alpha \sin \theta + \eta {{\cos }^2}\alpha \sin 2\theta + 2\lambda \cos 2\alpha \cos \theta \Big)} - 2\cos \alpha \cos \varphi \sin \theta + \\ 2\sin \alpha \sin \varphi ( {\eta + \lambda \sin \theta } ) + 2\cos \alpha \sin \varphi \cos \theta ( {\beta - \sin \alpha } ) = 0. \nonumber
\end{eqnarray}
\vspace{-0.7pc}

From the beginning of scales engagement, the relationship (\ref{Eq6}) is established between the substrate twist angle $\varphi$ and the scales inclination angle $\theta$. After engaging, scales slide over each other and $\theta$ starts to increase from its initial value $\theta_0$ according to the nonlinear relationship (\ref{Eq6}). Scales engagement start at relatively small twist angle, therefore to find an explicit relationship for the engagement twist angle $\varphi_e$, we linearize (\ref{Eq6}) by considering small twist regime ($\varphi \ll 1$, $\theta \ll 1$) which leads to $\varphi_e={\theta_0}/(\eta \tan \alpha+ \beta- \sin \alpha)$.

Using the kinematic relationship (\ref{Eq6}), we probe the existence of a singular point where locking can take place. This would be the envelope defined by $\partial \varphi /\partial \theta=0$, and beyond which no more sliding is possible without significant deformation of the scales. This point is called the ``kinematic locking" of the system \cite{c42}.

By putting derived equation of $p$ or $q$ into the (2) or (4), we will have the location of point $P_{12}$ as the intersection between lines $D_2C_2$ and $C_1B_1$. We can use the same procedure to establish the locations of zeroth scale's corners and its local coordinates after rotating with angle $-\varphi$ about torsion axis. We find the same nonlinear relationship between $\varphi$ and $\theta$ due to the periodicity of the system, then we can find the location of point $P_{10}$ as the intersection between lines $D_1C_1$ and $C_0B_0$, using the same method.

\subsection{Mechanics} \label{Mechanics}

To investigate the role of friction in twisting behavior of biomimetic scale-covered substrate, we investigate the  free body diagram of the RVE (here 1\textsuperscript{st} scale) during engagement as shown in Figure \ref{Fig3}. The forces on the 1\textsuperscript{st} scale are as follows. At contact point between zeroth scale and 1\textsuperscript{st} scale $P_{10}$, there are two reaction forces including friction force $\bi{f_{10}}$ acting in the plane of 1\textsuperscript{st} scale by angle $\chi_{10}$ with respect to the unit vector $\bi{{n}_{X1}}$, and normal force $\bi{N_{10}}$ acting perpendicular to this plane in direction $-\bi{{n}_{Z1}}$ as shown in Figure \ref{Fig3}. Also, at contact point between 1\textsuperscript{st} scale and 2\textsuperscript{nd} scale $P_{12}$, two reaction forces are acting as friction force $\bi{f_{12}}$ in the plane of 2\textsuperscript{nd} scale by angle $\chi_{12}$ with respect to the unit vector $\bi{{n}_{X2}}$, and normal force $\bi{N_{12}}$ perpendicular to the plane of 2\textsuperscript{nd} scale in direction $\bi{{n}_{Z2}}$ as shown in Figure \ref{Fig3}.

\begin{figure}[htbp]
\centering
\includegraphics[width=3.3in]{Fig3.pdf}
\caption{Free body diagram of each pair of scales representing their contact points, applied normal force $\bi{N}$, and friction force $\bi{f}_{fr}$ at the contact points.}
\label{Fig3}
\end{figure}

Note that the direction of friction forces are dependent on the direction of relative motion between each scale pairs. Due to the periodicity, the value of friction forces are equal $f_{fr}=f_{10}=f_{12}$, and also the value of normal forces are equal $N=N_{10}=N_{12}$. According to the described  free body diagram, the balance of moments at the base of 1\textsuperscript{st} scale can be described in the vectorial format as follows:
\vspace{0.7pc}
\begin{eqnarray} \label{Eq7}
\fl K_ \theta (\theta - \theta_0)=\bigg( \bi{O_{1}P_{10}} \times \Big( -({f_{fr}} \cos \chi_{10})\bi{{n}_{X1}} - ({f_{fr}} \sin \chi_{10})\bi{{n}_{Y1}} - (N)\bi{{n}_{Z1}} \Big)+ \\
\bi{O_{1}P_{12}} \times \Big( ({f_{fr}} \cos \chi_{12})\bi{{n}_{X2}} + ({f_{fr}} \sin \chi_{12})\bi{{n}_{Y2}} + (N)\bi{{n}_{Z2}} \Big) \bigg).\bi{{n}_{Y1}}, \nonumber
\end{eqnarray}
\vspace{-0.5pc}

\noindent where $\bi{O_{1}P_{10}}$ and $\bi{O_{1}P_{12}}$ are the position vector of contact points $P_{10}$ and $P_{12}$ with respect to the base of the 1\textsuperscript{st} scale, respectively as shown in Figure \ref{Fig3}. $K_{\theta}$ is the ``rotational spring constant" or the ``rigid scale--elastic substrate joint stiffness". As the scales engage, they tend to push each other and increase their inclination angle $\theta$, but the elastic substrate resists against scales rotation. This resistance is modeled as linear torsional spring \cite{c35,c36}, and the absorbed energy due to the rotation of each scale is $U_{scale}=\frac{1}{2} {K}_\theta (\theta-\theta_0 )^2$, thus the local reaction moment would be $M_{scale}=K_\theta (\theta-\theta_0)$. According to developed scaling expression in \cite{c42}, $K_{\theta}=3.62{E_B}{t_s}^2b( {{L}/{t_s}})^{1.55}$, where $E_B$ is the elastic modulus of substrate.

To describe the relative motion between zeroth scale and 1\textsuperscript{st} scale, we would need the relative motion of contact point $P_{10}$ on the edge $D_1C_1$ and edge $C_0B_0$. Motion of point $P_{10}$ on the edge $D_1C_1$ can be described as the change in the length of vector $\bi{P_{10}C_1}$, which is always in direction of $\bi{{n}_{X1}}$, and the change in the length of vector $\bi{P_{10}C_0}$, which is always in direction of $\bi{{n}_{Y0}}$. By using the superposition principle, the total differential displacement of point $P_{10}$ can be described in vectorial format as $\mathrm{d}\bi{R_{10}}=\big(\mathrm{d}|\bi{P_{10}C_1}|\big)\bi{{n}_{X1}}+\big(\mathrm{d}|\bi{P_{10}C_0}|\big)\bi{{n}_{Y0}}$, Figure \ref{Fig3}. The unit vector $\bi{{n}_{Y0}}$ can be described in the local coordinate established on 1\textsuperscript{st} scale as follows:
\vspace{0.7pc}
\begin{equation} \label{Eq8}
\bi{{n}_{Y0}}=(\bi{{n}_{Y0}}.\bi{{n}_{X1}})\bi{{n}_{X1}}+(\bi{{n}_{Y0}}.\bi{{n}_{Y1}})\bi{{n}_{Y1}}+(\bi{{n}_{Y0}}.\bi{{n}_{Z1}})\bi{{n}_{Z1}}.
\end{equation}
\vspace{0.7pc}

By projecting $\bi{{n}_{Y0}}$ on the 1\textsuperscript{st} scale plane, we can describe relative motion of zeroth scale with respect to 1\textsuperscript{st} scale as the planar relative displacement, as follows:
\vspace{0.7pc}
\begin{equation} \label{Eq9}
\fl \mathrm{d}\bi{r}=\Big(\mathrm{d}|\bi{P_{10}C_1}|+\mathrm{d}|\bi{P_{10}C_0}|(\bi{{n}_{Y0}}.\bi{{n}_{X1}})\Big)\bi{{n}_{X1}}+ \Big(\mathrm{d}|\bi{P_{10}C_0}|(\bi{{n}_{Y0}}.\bi{{n}_{Y1}})\Big)\bi{{n}_{Y1}}.
\end{equation}
\vspace{0.7pc}

The length of (\ref{Eq9}) can be described as the relative differential displacement value:
\vspace{0.7pc}
\begin{equation} \label{Eq10}
\fl \mathrm{d}r=|\mathrm{d}\bi{r}|=\sqrt{\Big(\mathrm{d}|\bi{P_{10}C_1}|+\mathrm{d}|\bi{P_{10}C_0}|(\bi{{n}}_{Y0}.\bi{{n}_{X1}})\Big)^2+ \Big(\mathrm{d}|\bi{P_{10}C_0}|(\bi{{n}_{Y0}}.\bi{{n}_{Y1}})\Big)^2}.
\end{equation}
\vspace{0.7pc}

To find the angle between the friction force $\bi{{f}_{fr}}$ acting in the plane of 1\textsuperscript{st} scale and the unit vector $\bi{{n}_{X1}}$, we can use (\ref{Eq9}) and (\ref{Eq10}) as the relative displacement vector and its value, then angle $\chi_{10}$ is derived as:
\vspace{0.7pc}
\begin{equation} \label{Eq11}
\chi_{10} = \arccos \Big( \frac{1}{\mathrm{d}r} \big( \mathrm{d}|\bi{P_{10}C_1}|+\mathrm{d}|\bi{P_{10}C_0}|(\bi{{n}_{Y0}}.\bi{{n}_{X1}}) \big) \Big).
\end{equation}
\vspace{0.7pc}

If we repeat similar steps for the relative motion between 1\textsuperscript{st} scale and 2\textsuperscript{nd} scale, it will lead to the similar relationship for the angle between the friction force $\bi{{f}_{fr}}$ acting in the plane of 2\textsuperscript{nd} scale and the unit vector $\bi{{n}_{X2}}$. Finally by computing the values of these relationships, we find that $\chi_{10} = \chi_{12}$, and can be shown as ${\chi}$. This finding also conform the periodicity in the system.

According to the Coulomb's Law of Friction, scales do not slide while $f_{fr} \leq \mu N$, where $\mu$ and $N$ are coefficient of friction and normal force, respectively, while sliding regime is marked by the equality. Note that we use the same value for static coefficient of friction as well as the kinetic coefficient of friction in this study, although typically static coefficient of friction is slightly higher. Using these considerations, we can derive the following expression as the non-dimensionalized friction force $\overline{f}_0$, with respect to the free body diagram shown in Figure \ref{Fig3}:
\vspace{0.7pc}
\begin{eqnarray} \label{Eq12}
\fl \overline{f}_0 = \frac{f_{fr} l}{K_\theta} \leq \\
\fl \frac{(\theta - \theta_0)l}{
 \Big( \hspace{-2pt} \bi{O_{1}P_{12}} \hspace{-3pt} \times \hspace{-3pt} \big( \hspace{-2pt} \cos \chi \bi{{n}_{X2}} \hspace{-2pt} + \hspace{-2pt} \sin \chi \bi{{n}_{Y2}} \hspace{-2pt} + \hspace{-2pt} \frac{\bi{{n}_{Z2}}}{\mu} \hspace{-2pt} \big) \hspace{-2pt} - \hspace{-2pt}
 \bi{O_{1}P_{10}} \hspace{-3pt} \times \hspace{-3pt} \big( \hspace{-1pt} \cos \chi \bi{{n}_{X1}} \hspace{-2pt} + \hspace{-2pt} \sin\chi \bi{{n}_{Y1}} \hspace{-2pt} + \hspace{-2pt} \frac{\bi{{n}_{Z1}}}{\mu} \hspace{-1pt} \big) \hspace{-1pt} \Big).\bi{{n}_{Y1}}}. \nonumber
\end{eqnarray}
\vspace{0.1pc}

Due to the nature and the geometrical configuration of the system, the magnitude of the friction force derived in (\ref{Eq12}), may exhibit singularity at a certain twist rate. This rise in friction force may lead to a ``frictional locking" mechanism, observed in the bending case \cite{c39}. If predicted, the frictional locking should happen at the lower twist rate rather than the kinematic locking, because of the limiting nature of friction force. We call the twist rate in which locking happens as $\Upphi_{lock}$, and the twist angle and the scale inclination angle would be as $\varphi_{lock}=\Upphi_{lock}d$ and $\theta_{lock}$, respectively.

The friction force computed above will lead to dissipative work in the system during sliding. The non-dissipative component of the deformation is absorbed as the elastic energy of the biomimetic beam. This elastic energy is composed of elastic energy of the beam and the scales rotation. To calculate this elastic energy of the beam, we consider a linear elastic behavior for the beam with a warping coefficient $C_w$ for a non-circular beam \cite{c42,c47}. Furthermore, due to the finite embedding of the scales, there will be an intrinsic stiffening of the structure even before scales engagement. This stiffening can be accurately captured by using an inclusion correction factor $C_f$ \cite{c42}. $C_f$ is function of the volume fraction of the rigid inclusion into the elastic substrate, and postulated as $C_f=1+1.33({\zeta \beta}/{\lambda})$, where $\zeta=L/d$ for an analogous system \cite{c42}. With these considerations, modified torque-twist relationship of the beam is $T=C_f C_w G_B I\Upphi$, and the elastic energy of the beam can be considered as $U_B=\frac{1}{2} C_f C_w G_B I\Upphi^2$. As mentioned earlier, the energy absorbed by the scales can be obtained by assuming the scale’s resistance as linear torsional spring and the absorbed energy due to the rotation of each scale will be $U_{scale}=\frac{1}{2} K_\theta (\theta-\theta_0)^2$. Similarly the dissipation can be given as the product of the sliding friction and distance travelled by the point of application per scale. Then we use the work–energy balance to arrive at:
\vspace{0.7pc}
\begin{equation} \label{Eq13}
\fl \int_{0}^{\Upphi} T( {\Upphi }')\mathrm{d}{\Upphi}' = {\frac{1}{2}}{C_f}{C_w}{G_B}I{{\Upphi}^2} + \bigg( {\frac{1}{2}} {\frac{1}{d}}{K_\theta}{( {\theta - {\theta _0}})^2} + {\frac{1}{d}} \int_{\Upphi_e}^{\Upphi} \hspace{-3pt} f_{fr} \mathrm{d}r \bigg) H(\mathrm{\Upphi} - {{\Upphi}_e} ),
\end{equation}
\vspace{0.7pc}
where $\Upphi$, $\Upphi_e=\varphi_e/d$, $G_B$, and $I$ are the current twist rate, the engagement twist rate, the shear modulus of elasticity, and the cross section's moment of inertia of the beam. $H(\Upphi-\Upphi_e)$ is the Heaviside step function to track scales engagement. Also, $C_f$, $C_w$, and $K_\theta$ are inclusion correction factor, warping coefficient, and rotational spring constant of scale–substrate joint stiffness, respectively. In (\ref{Eq13}), $f_{fr}$ is representing the friction force between scales, and $\mathrm{d}r$ is the relative differential displacement described in (\ref{Eq10}).

The torque--twist rate relationship for the substrate's unit length could be obtained by taking the derivative of (\ref{Eq13}) with respect to the twist rate $\Upphi$, while considering $\varphi=\Upphi d$, as follows:
\vspace{0.7pc}
\begin{equation} \label{Eq14}
 T( \Upphi)={C_f}{C_w}{G_B}I{\Upphi} + \bigg({K_\theta}(\theta - \theta _0)\frac{\partial\theta}{\partial\varphi} + f_{fr} \frac{\mathrm{d}r}{\mathrm{d}\varphi} \bigg) H(\mathrm{\Upphi} - {{\Upphi}_e}).
\end{equation}
\vspace{0.7pc}

We also compute the maximum possible dissipation of the system by computing the frictional work done till locking ($W_{fr}$) and compare it with the total work done ($W_{sys}=U_{el}+W_{fr}$, where $U_{el}$ is the elastic energy of the system). These energies can be computed per unit length of the beam as:
\vspace{0.7pc}
\numparts \begin{eqnarray}
U_{el}={\frac{1}{2}}\bigg({C_f}{C_w}{G_B}I{(\Upphi_{lock})^2} +{\frac{1}{d}}{K_\theta}{( {\theta_{lock} - {\theta _0}})^2 \bigg)}, \label{Eq15a} \\
W_{fr}={\frac{1}{d}} \int_{\Upphi_e}^{\Upphi_{lock}} \hspace{-3pt} f_{fr} \mathrm{d}r. \label{Eq15b}
\end{eqnarray} \endnumparts
\vspace{-0.2pc}

We define the relative energy dissipation ($RED$) factor as the ratio of the frictional work per unit length $W_{fr}$, to the total work done on the system per unit length $W_{sys}$:
\vspace{0.7pc}
\begin{equation} \label{Eq16}
RED=\frac{W_{fr}}{W_{sys}}.
\end{equation}
\vspace{0.7pc}

Generally, $RED$ is dependant on the coefficient of friction $\mu$, dimensionless geometric parameters of the system $\eta$, $\beta$, and $\lambda$, scale spacing $d$, scales initial orientation angles $\alpha$ and $\theta_0$, substrate elastic properties $G_B$, $I$, and $C_w$, and scale--substrate joint parameters $K_{\theta}$ and $C_f$, but the most important parameters are $\mu$, $\eta$, and $\alpha$.

\section{Finite element simulations} \label{Finite element simulations}

We developed an FE model for verification of the analytical model of the biomimetic scale-covered system under twisting deformation. The FE simulations are carried out using commercially available software ABAQUS/CAE 2017 (Dassault Syst\`emes). We considered 3D deformable solids for scale and substrate. However, for the scales, rigid body constraint was imposed. A sufficient substrate length is considered for rectangular prismatic substrate to satisfy the periodicity. Then an assembly of substrate with a row of 25 scales embedded on its top surface is created. The scales are oriented at angles of $\theta_0$ and $\alpha$ as defined in the analytical model. Linear elastic material properties including $E_B$ and $\nu$ are applied to the substrate part which leads to the shear modulus of $G_B={\frac{E_B}{2(1+\nu)}}$.

The simulation considered as a static step with nonlinear geometry option. The left side of the beam is fixed and the twisting load was applied on the other side of the beam. A frictional contact criteria is applied to the scales surfaces with coefficient of friction $\mu$ for a twisting simulation. The top layer of substrate is meshed with tetrahedral quadratic elements C3D10 due to the geometrical complexity around scales inclusion. Quadratic hexahedral elements C3D20 are used for other regions of the model. A mesh convergence study is carried out to find sufficient mesh density for different regions of the model. A total of almost 70,000 elements are employed in the FE model.

\section{Results and discussion} \label{Results and discussion}

To study the frictional force behavior in this system, we use (\ref{Eq12}) to plot non-dimensionalized friction force $\overline{f}_0$ for different $\mu$ values at various non-dimensionalized twist rate $\Upphi/\Upphi_e$. This is shown in Figure \ref{Fig4} for a system with $\eta=3$, $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, and $\lambda=0.45$. From this figure, it is clear that increasing twist leads to a rapid increase in the friction force for any coefficient of friction. There is a singular characteristic to this load as shown with dashed lines for each $\mu$ in Figure \ref{Fig4}, which indicates a friction based locking mechanism. This is in addition to the purely kinematic locking mechanism reported earlier in literature for frictionless counterparts \cite{c42}. We call the value of twist rate at the locking point, as the locking twist rate $\Upphi_{lock}$.

\begin{figure}[htbp]
\centering
\includegraphics[width=3in]{Fig4.pdf}
\caption{Non-dimensionalized friction force vs Non-dimensionalized twist rate ($\Upphi_e$ is the engagement twist rate) for various coefficients of friction with the given values of $\eta=3$, $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, and $\lambda=0.45$. This figure shows that the friction forces approach singularity near a certain twist rate as the frictional locking configuration for each $\mu$.}
\label{Fig4}
\end{figure}

Next, we investigate the scale rotation in response to applied twist. This is achieved by plotting the scale angle rotation $\theta$ versus twist angle $\varphi$. Using nonlinear relationship (\ref{Eq6}), two plots are established spanned by $(\theta-\theta_0)/\pi$ and $\varphi/\pi$ as shown in Figure \ref{Fig5} for different $\eta$ and $\alpha$, respectively.

In Figure \ref{Fig5a}, the given geometrical parameters are as follows $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, and $\lambda=0.45$. For $\mu = 0$, which indicates frictionless case, we obtain purely kinematic locking points for each $\eta$ by using $\partial \varphi / \partial \theta=0$ to obtain rigidity envelope \cite{c42}. We juxtapose this with plots the rough interfaces ($\mu>0$), where the locking limits are found via the singularity point of friction force described in (\ref{Eq12}). Clearly, friction advances the locking configuration. However, the locking line does not merely translate downwards as observed in the bending case \cite{c39}. This is an important distinction from the pure bending of rough biomimetic beams reported earlier \cite{c39}. As coefficient of friction increases, the frictional locking envelope can intersect the horizontal axis. This is the instantaneous locking or the ``static friction" lock case.

In Figure \ref{Fig5b}, the effect of scales orientation with angle $\alpha$ is investigated. This angle serves as an important geometric tailorability parameter of the system \cite{c42}. In this plot, $\eta=3$, $\theta_0=10^\circ$, $\beta =1.25$, and $\lambda=0.45$. For higher angles $\alpha$, a quicker engagement occurs with steeper nonlinear gains and earlier locking. Interestingly, by decreasing $\alpha$ sufficiently, the system would not reach to the kinematic locking. However, frictional locking is universal and will thus determine the locking behavior. In this aspect, this system again differs from bending case since friction can cause locking even when no-kinematic locking is possible. This figure also shows the possibility of static friction locking for increasing $\mu$. However note that as $\alpha$ increases, such static friction lock becomes more difficult to achieve requiring much higher frictional coefficients. Overall the frictional locking envelope is a highly nonlinear function admitting no closed form solution unlike the pure bending case \cite{c39}.

\begin{figure}[htbp]%
\centering
\subfigure[][]{%
\label{Fig5a}%
\includegraphics[width=3in]{Fig5a.pdf}}%
\hspace{8pt}%
\subfigure[][]{%
\label{Fig5b}%
\includegraphics[width=3in]{Fig5b.pdf}} \\
\caption[]{The plot representation
of the biomimetic scale-covered beam under twisting differentiated to three distinct regimes of
performance including: linear (before scales engagement), kinematically determined nonlinear (during scales engagement), and
a frictional locking boundary for various coefficients of friction:
\subref{Fig5a} plot of the system for different $\eta$ with the given values of $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, and $\lambda=0.45$; and
\subref{Fig5b} plot of the system for different $\alpha$ with the given values of $\eta=3$, $\theta_0=10^\circ$, $\beta=1.25$, and $\lambda=0.45$.}%
\label{Fig5}%
\end{figure}

In order to understand the effect of friction force on the mechanics of the system, we use (\ref{Eq14}) to plot the non-dimensionalized post-engagement torque–-twisting rate plot for various coefficients of friction, Figure \ref{Fig6a}. Dimensionless geometrical parameters for this case are $\eta=3$, $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, $\lambda=0.45$, $\zeta=0.35$, and $L/t_s=35$. To verify the analytical model, we have developed an FE model as described in section \ref{Finite element simulations}. Then we have done FE simulations for different $\eta$ and $\mu$ values and extracted torsional response of the structure $T(\Upphi)/G_B I$, versus twist rate from the beginning of the simulation as shown in Figure \ref{Fig6b}. The following dimensionless parameters are used for this model: $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=0.6$, $\lambda=0.32$, $\zeta=0.18$, and $L/t_s=45$. Also the following elastic properties are considered for substrate: $E_B=25$ $GPa$, $\nu=0.25$, with a cross section dimension of $32 \times 16$ $mm$. In this figure, the dotted lines are representing FE results. The plot highlights remarkable agreement between analytical and FE results for two different overlap ratios along different coefficients of friction. The small deviation between results could be caused by edge effects and numerical issues.

As shown in Figure \ref{Fig6a}, higher coefficient of friction significantly increases the torsional stiffness of the structure. Therefore, the friction force has a dual contribution to the mechanical response of biomimetic scale-covered system –- while advancing locking, thereby limiting range of motion but also increasing the torsional stiffness of the system.

\begin{figure}[htbp]%
\centering
\subfigure[][]{%
\label{Fig6a}%
\includegraphics[width=3in]{Fig6a.pdf}}%
\hspace{8pt}%
\subfigure[][]{%
\label{Fig6b}%
\includegraphics[width=3in]{Fig6b.pdf}} \\
\caption[]{Torque--twisting rate curve derived from (\ref{Eq14}) for different cases:
\subref{Fig6a} non-dimensionalized post-engagement torque--twisting rate curves for various coefficients of friction with the given values of $\eta=3$, $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=1.25$, $\lambda=0.45$, $\zeta=0.35$, and $L/t_s=35$, showing the perceptible effect of friction in the effective torsional stiffness of the biomimetic scale-covered structure; and
\subref{Fig6b} verification of analytical model using numerical results through the plot of ${T(\Upphi )}/{G_B}I$ versus twist rate ($\Upphi$) for various coefficients of friction and two different $\eta$ with the given values of $\theta_0=10^\circ$, $\alpha=45^\circ$, $\beta=0.6$, $\lambda=0.32$, $\zeta=0.18$, and $L/t_s=45$. Black dotted lines represent FE results.}%
\label{Fig6}%
\end{figure}

In order to quantify the dual contribution of friction, we investigate the frictional work during twisting by using the relative energy dissipation ($RED$), described in (\ref{Eq16}). Fixing all parameters involved in $RED$, except $\mu$, $\eta$, and $\alpha$ for the current simulation leads to contour plots shown in Figure \ref{Fig7}. In these contour plots, we have considered $\theta_0=10^\circ$, $\beta=1.25$, $\lambda=0.45$, $\zeta=0.35$, $L/t_s=35$, and the substrate’s properties as follows $E_B=25$ $GPa$, $\nu=0.25$, and the cross section dimension of $32 \times 16$ $mm$.

In Figure \ref{Fig7a}, we fix $\alpha=45^\circ$ to obtain an energy dissipation contour plot spanned by $\eta$ and $\mu$. This plot indicates that $RED$ increases for higher $\mu$, and also increases very slightly with $\eta$. This contour plot shows that $\eta$ does not have as strong effect as coefficient of friction on frictional energy dissipation of the system. However, the frictional work quickly saturates with higher coefficient of friction for all $\eta$.

To obtain Figure \ref{Fig7b}, we fix $\eta=3$ and the $RED$ contour plot spanned by $\alpha$ and $\mu$. This plot shows that, despite that locking twist rate $\Upphi_{lock}$ increases by decreasing $\alpha$ according to Figure \ref{Fig7b}, the effect of the friction is higher at the range of $40^\circ<\alpha<60^\circ$, and the $RED$ passes through its maximum by increasing $\mu$ around this range of $\alpha$. Also at lower $\alpha$, unilaterally increasing $\mu$ does not necessarily increase the frictional dissipation. The white region in this contour plot is related to the instantaneous post-engagement frictional locking, which happens at lower $\alpha$ and higher $\mu$. At this condition, the system lock statically at the engagement point and the friction force does not work on the system.

\begin{figure}[htbp]%
\centering
\subfigure[][]{%
\label{Fig7a}%
\includegraphics[width=3in]{Fig7a.pdf}}%
\hspace{8pt}%
\subfigure[][]{%
\label{Fig7b}%
\includegraphics[width=3in]{Fig7b.pdf}} \\
\caption[]{Non-dimensional relative energy dissipation ($RED$) factor contour plot with given values of $\theta_0=10^\circ$, $\beta=1.25$, $\lambda=0.45$, $\zeta=0.35$, $L/t_s=35$, $E_B=25$ $GPa$, $\nu=0.25$, and the substrate's cross section of $32 \times 16$ $mm$ for two different cases:
\subref{Fig5a} spanned by $\mu$ and $\eta$ for fixing $\alpha=45^\circ$; and
\subref{Fig5b} spanned by $\mu$ and $\alpha$ for fixing $\eta=3$.}%
\label{Fig7}%
\end{figure}

\section{Conclusion} \label{Conclusion}

We investigate for the first time, the effect of Coulomb friction on the twisting response of a biomimetic beam using a combination of analytical and FE model. We established the extent and limits of universality of frictional behavior across bending and twisting regimes. The analytical model which have been developed, would help in obviating the need for full-scale FE simulations, which are complicated for large number of scales and for large deflection. We find that several aspects of the mechanical behavior show similarity to rough bending case investigated earlier. At the same time, critical differences in response were observed, most notably the effect of the additional dihedral angle. This work shows the dual contribution of frictional forces on the biomimetic scale-covered system, which includes advancing the locking envelope and at the same time adding to the torsional stiffens. Interestingly, if the coefficient of friction is large enough for a given configuration, it can lead to the instantaneous post-engagement frictional locking known as the static friction locking. This investigation demonstrates that engineering scale surfaces to produce wide range of coefficients of friction can play an important role on tailoring the deformation response of biomimetic scales systems under a variety of applications.

\section*{References}


\section{Introduction}
In the last few decades, the increasing amount of information to which users are exposed on a daily basis determined the rise of recommendation systems. These models are able to identify and exploit patterns of users interest, with the goal of providing personalized recommendations and improving the final user experience. As a result, recommendation systems are now integrated in a large number of commercial applications, with some prominent examples being Amazon, Netflix, Spotify, Uber or Airbnb.

One of the most popular approaches used in the development of recommendation systems is \textit{Collaborative Filtering} (CF). In CF, past user behaviours are used to derive relationships between users and inter-dependencies among items, in order to identify new user-item associations \cite{Koren09matrixfactorization}. The main advantage of CF lies in its intrinsic domain-free nature and in its ability to model user-item interactions without requiring the creation of explicit profiles

In this work, we explore a wide spectrum of state-of-the-art approaches to Collaborative Filtering.
In particular, we focus on both \textit{memory-based} approaches, where inference is based on calculations on past users' preference records (e.g. similarity models) and \textit{model-based} approaches, where the same past users' preferences are used to train a model which is then used at inference time to predict new recommendations (e.g. factorization, and neural models) \cite{taxonomy}.
Other than exploring existing models, we propose a novel stochastic extension of a similarity-based algorithm, SCSR.
Finally, we empirically verify that blending factorization-based and similarity-based approaches can yield more accurate results, decreasing the validation RMSE by 9.4\% with respect to the difference between the baseline and the best-performing single model.

The performances of these model are compared on the 2022 CIL CF dataset.
This dataset consists of a sparse matrix (only $11.77\%$ of the entries are observed) that defines the interactions (integer ratings between 1 and 5) between $10000$ users and $1000$ items.

In the following sections,
we theoretically introduce the investigated models (§\ref{sect:methods}) and explain in detail our experiments (§\ref{sect:exp}). Finally, we conclude with some final remarks on our work (§\ref{sect:conclusion}).

\section{Methods}
\label{sect:methods}
\subsection{Matrix Factorization}
As demonstrated by the Netflix Prize competition \cite{bennett2007netflix}, matrix factorization techniques are very effective in the context of CF.
The main idea behind these algorithms consists in mapping both users and items to a joint (lower-dimensional) latent factor space, such that the original interactions between users and items can be reconstructed as inner products in that latent space.

A direct implementation of this concept can be obtained applying Singular Value Decomposition (SVD) \cite{clsvd}, a well-established technique for identifying latent semantic factors in information retrieval.
SVD decomposes the original matrix in the product
\begin{equation}
    A = U \Sigma V^T
\end{equation}
where $U\in \mathbb{R}^{m\,\times \,m}$ and $V\in \mathbb{R}^{n\,\times \, n}$ are orthogonal matrices and $\Sigma \in \mathbb{R}^{m\,\times \, n}$ is the diagonal matrix of singular values.
In our context, matrices $U$ and $V$ could be interpreted as the latent factors associated to users and items respectively, while matrix $\Sigma$ expresses the weight associated to each latent feature.
Since the main goal is to generalize well to unobserved entries at inference time and not obtain a null reconstruction error, the singular value diagonal matrix $\Sigma$ is approximated using only the largest $k$ singular values. Hence, the latent features for users and items are embedded in a lower $k$-dimensional space, with $k<\!\!<n,m$.

However, SVD needs a large storage space and has a large computational complexity. In order to overcome these limitations, a simple and yet remarkably powerful method, FunkSVD \cite{funk}, was proposed as part of the Netflix Prize competition.
The algorithm initializes the latent factor matrices $U$ and $V$ randomly, and then optimize them using Stochastic Gradient Descent (SGD). The objective function optimized with SGD is:

\begin{equation}
    \argmin_{U,V} \left\lVert \, A - \widetilde{A} \,\right\rVert_F + \alpha  \left\lVert U \right\rVert + \beta \left\lVert V \right\rVert
\end{equation}

where $\left\lVert \, \cdot \, \right\rVert_F$ is the Frobenius norm, $A$ is the original matrix, and $\widetilde{A} = UV$ is the reconstructed matrix. The last two terms of the objective are regularization terms.

Another well-known matrix-factorization technique based on iterative optimization is Alternating Least Square (ALS). The objective function which is optimized by the method is

\begin{equation}
\label{eq:als}
    \argmin_{U,V} \left\lVert \, A - \widetilde{A} \,\right\rVert_F^2 + \lambda \left( ||U||_F^2 + ||V||_F^2 \right)
\end{equation}

Since both $U$ and $V$ are unknown, (\ref{eq:als}) is not convex. However, if we fix one of the unknowns, the optimization problem becomes quadratic and can be solved in closed form.
Thus, ALS techniques iteratively improves $U$ and $V$ by solving a least-square problem, with a loss that always decreases and, eventually, converges.

The last investigated matrix factorization approaches are Factorization Machines (FM) \cite{Rendle2010FactorizationM}, which allow for predictions over a sparse dataset. FMs have the same flexibility as Support Vector Machines (SVM) in that they accept any feature vector, yet work well over sparse data by factorizing the parameters modeling the relevant interactions. The model equation of a FM of degree 2 for $n$ features is as follows:

\footnotesize
\begin{equation}
\hat{y}(\boldsymbol{x}) := w_0 + \sum_{i=1}^n{w_i x_i} + \sum_{i=1}^n\sum_{j=i+1}^n{\langle \boldsymbol{v_i}, \boldsymbol{v_j} \rangle x_i x_j}
\end{equation}
\normalsize

\noindent
with learned parameters $w_0 \in \mathbb{R}, w \in \mathbb{R}^n$, and $V \in \mathbb{R}^{n \times k}$. $w_0$ is the global bias, $w$ models the weights of the individual features, $V$ a factorization of the weights of the interactions between pairs of features, and $v_i \in \mathbb{R}^k$ is the $i$-th row of $V$.

Adding Bayesian inference, as in Bayesian Factorization Machines (BFM) \cite{Freudenthaler2011BayesianFM}, is a considerable improvement: it allows to avoid the manual tuning of multiple hyper-parameters, and performs better than SGD and ALS optimization.
BFM uses Gibbs sampling, a Markov Chain Monte Carlo (MCMC) algorithm for posterior inference.
with a complexity of $\mathcal{O}(k N_z)$ for each sampling step ($N_z$ being the number of non-zero elements in the feature matrix).

One of the main advantages of FM, compared to the other matrix factorization models, is that we can easily choose how to engineer the feature vectors.
The standard way is to simply concatenate a one-hot encoding of the user with a one-hot encoding of the item: $(u, i)$, to which a list of ratings is associated.
These user-movie-rating combinations are also referred to as \textit{explicit feedback} \cite{Oard1998ImplicitFF}, i.e.  information directly provided by the users themselves. However, \textit{implicit feedback} \cite{Oard1998ImplicitFF}, which is information not directly provided by the users but that collected through their usage of the service \cite{Oard1998ImplicitFF}, is also useful to the model. For example, the date of the rating (in days since the first rating of the dataset or since the movie release) has been used successfully for this task \cite{Koren2009CollaborativeFW}.

Even if our data does not include any implicit feedback, we can recreate some about the users by highlighting all the movies that each user has rated, resulting in the feature vector $(u, i, iu)$, as in SVD++ model \cite{Koren2008FactorizationMT}, $iu$ standing for implicit user (feedback).
Similarly, we can also go in the other direction by showing which users have rated each movie, such as in the Bayesian timeSVD++ flipped model \cite{onthediffic}, only without the time information: $(u, i, iu, ii)$, $ii$ standing for implicit item (feedback).

\subsection{Similarity Based}
Similarity based methods are designed to find neighborhoods of similar users/items using a similarity function on common ratings.
Within these neighborhoods, it is possible to compute weighted averages of the observed ratings (with weights being proportional to the similarity degree), and use them to infer missing entries.
In our work, we investigate 3 different similarity measures: cosine similarity, PCC and SiGra \cite{wu2017sigra}. The similarity between two users or items $u$ and $v$ are defined, respectively, as:

\footnotesize
\begin{equation*}
    \textrm{\normalsize cosine(u, v)} = \frac{\sum_{k \in I_u \cap I_v} r_{uk} \cdot r_{vk}}{\sqrt{\sum_{k \in I_u \cap I_v} r_{uk}^{2}} \cdot \sqrt{\sum_{k \in I_u \cap I_v} r_{vk}^{2}}}
\end{equation*}
\begin{equation*}
    \textrm{\normalsize PCC(u, v)} = \frac{\sum_{k \in I_u \cap I_v} (r_{uk}-\overline{\mu_u}) \cdot (r_{vk}-\overline{\mu_v})}{\sqrt{\sum_{k \in I_u \cap I_v} (r_{uk}-\overline{\mu_u})^{2}} \cdot \sqrt{\sum_{k \in I_u \cap I_v} (r_{vk}-\overline{\mu_v})^{2}}}
\end{equation*}
\begin{equation*}
    \textrm{\normalsize SiGra(u, v)} = \left(1+\exp{\left(-\frac{|I_u|+|I_v|}{2\cdot |I_u \cap I_v|}\right)}\right)^{-1}
    \cdot
    \frac{\sum_{k \in I_u \cap I_v}\frac{\min(r_{uk}, r_{vk})}{\max(r_{uk}, r_{vk})}}{|I_u \cap I_v|}
\end{equation*}

\vspace*{0.2cm}
\normalsize
\noindent
where $k \in I_u \cap I_v$ represents the indexes of all commonly rated items, respectively users and $r_{uk}$ is the rating given by user $u$ for item $k$.
These 3 functions could be either applied between all items or between all users, and we experiment with both.

It can be observed that PCC is closely related to cosine similarity, except that it first centers the ratings by removing the average and hence try to overcome the bias which occur when some users give overall better ratings than others.
Both PCC and cosine similarity tend to overestimate the similarity between users or items which have only few commonly rated items.
To overcome that, some weighting functions can be added on top of the previous functions to penalize more the similarity of users or items having only few common ratings.
We experiment with normal, significance and sigmoid weighting, defined as follows:

\footnotesize
\begin{equation*}
    {\text{\normalsize w\textsubscript{normal}(u, v)}} = \frac{2 \cdot |I_u \cap I_v|}{|I_u| + |I_v|}  \qquad {\text{\normalsize w\textsubscript{significance}(u, v)}} = \frac{\min(|I_u \cap I_v|, \beta)}{\beta}
\end{equation*}
\begin{equation*}
   {\text{\normalsize w\textsubscript{sigmoid}(u, v)}} = \left(1+\exp{-\frac{|I_u \cap I_v|}{2}}\right)^{-1}
\end{equation*}

\vspace*{0.1cm}
\normalsize
\noindent
SiGra already have such weighting incorporated into its formula, hence we do not add any of them to it.
Note that the significance weighting only penalize the similarity of items or users which have fewer common ratings than $\beta$ which is an hyper-parameter.

To find the neighborhoods and compute the predictions for the missing values once the similarity matrix has been computed, different methods exists. In our work, we use a weighted average of the $k$ nearest neighbors. The prediction is therefore defined as follows:

\small
\begin{equation*}
    \hat{r}_{ui} = \overline{\mu_u} + \frac{\sum_{v\in N_k^{u,i}} \textrm{sim}(u, v) \cdot (r_{vi} - \overline{\mu_v})}{\sum_{v\in N_k^{u,i}} |\textrm{sim}(u, v)|}
\end{equation*}
\normalsize
where $N_k^{u,i}$ represents the set of neighbors i.e. the set of the $k$ most similar users which have a rating for item $i$.
We also experiment with a combination of both user and item similarity for the final prediction, taking into account the confidence of the rating

Finally, we re-implement the Comprehensive Similarity Reinforcement (CSR) algorithm \cite{hu2017mitigating}, which alternatively optimize the user similarity matrix using the item similarity matrix and vice versa.
Note however that this method is extremely slow, as it runs in $\mathcal{O}(|I|^2 \cdot |U|^2 \cdot \textrm{max\_iter})$, and hence not applicable to our problem.

For this reason, we propose a novel implementation of the algorithm, Stochastic CSR (Algorithm \ref{alg:scsr}) which only uses a random sample of items and users at each iteration to update the other matrix.
This implementation of the algorithm runs in  $\mathcal{O}((|I|^2 + |U|^2) \cdot \textrm{sample\_size}^2 \cdot \textrm{max\_iter})$, which can be significantly smaller if sample\_size is small enough.

\subsection{Neural Based}
Traditional matrix factorization techniques are often interpreted as a form of dimensionality reduction, therefore some authors investigated the use of autoencoders to tackle CF.
In this case, the input $x$ is a sparse vector and the AE's output $y = f(x)$ is its dense correspondence, containing all the rating predictions of items in the corpus. First notable AE CF model was I-AutoRec (item-based) and U-AutoRec (user-based) \cite{autorec}.

DeepRec \cite{deeprec} is a fine-tuned version of the AutoRec model. It employs deeper networks, embraces SELU as activations \cite{selu} and apply heavy regularization techniques to avoid overfitting.
It also proposes a variation in training algorithm to address the fixed point constraint, intrinsic to the AE CF problem objective \cite{deeprec}.
DeepRec augments every optimization step with an iterative dense re-feeding step, i.e. it performs a classic weight update on the sparse-to-dense loss $L(x, f(x))$ and then treat $y = f(x)$ as a new artificial input, performing another weight update on the dense-to-dense loss $L(y,f(y))$.
DeepRec requires no prior training of the layers.
The model is optimized using a masked MSE loss.
During inference, model prediction is defined as a single forward pass $\hat{y} = f(x)$.

In the alternative Graph Convolutional Networks (GCN) \cite{gcn} paradigm, LightGCN (LGC) \cite{lightgcn} explores a simplified and optimized version of NGCF \cite{ngcf}, the former state-of-the-art model in GCN for CF. LGC gets rid of the feature transformation matrices and the non-linear activation function, two legacy layer propagation operations that NGCF inherited from GCN, which were argued to be not helpful for CF purposes \cite{lightgcn}. Hence, LGC propagation rule is simply defined as:

\footnotesize
\begin{align*}
e_u^{(k+1)} = \sum_{i \in \mathcal{N}_u} \frac{1}{\sqrt{|\mathcal{N}_u||\mathcal{N}_i}|} e_i^{(k)}
\qquad\quad
e_i^{(k+1)} = \sum_{u \in \mathcal{N}_i}  \frac{1}{\sqrt{|\mathcal{N}_i||\mathcal{N}_u}|} e_u^{(k)}
\end{align*}
\normalsize

where $e_u^{(k)}$ and $e_i^{(k)}$ respectively denote the refined embedding of user $u$ and item $i$ after $k$ layers propagation, and $\mathcal{N}_u$ denotes the set of items that are interacted by user $u$ (respectively for $\mathcal{N}_i$).
The embeddings are then combined with a weighted linear combination, setting an importance to each of the $k$-th layer embeddings.
The only other trainable parameters of LGC are the embeddings of the $0$-th layer.
Originally, the model is optimized using the Bayesian Personalized Ranking (BPR) loss, but we adapt it to use a standard RMSE loss.
During inference, model prediction is defined as the inner product of user and item final representations $\hat{
y}_{ui} = e_u^T e_i$.

\subsection{Blending}
Ensemble methods are designed to boost predictive accuracy by blending the predictions of multiple machine learning models.
These methods have been proven effective in many fields, and CF is no exception.
For example, both the first two classified solutions in the Netflix Prize challenge exploited blending \cite{blending1, blending2}.

We propose an evaluation of different blending techniques, based on a wide range of regression approaches (linear, neural, and boosting).
The underlying idea is to learn a transformation $\varphi:\mathbb{R}^{n\times m}\times ... \times \mathbb{R}^{n\times m} \rightarrow \mathbb{R}^{n\times m}$
such that the final prediction $\hat{A} = \varphi(\hat{A}^1,\, \dots \,, \hat{A}^k)$ is obtained in function of the predictions of $k$ different models.

\section{Experiments}
\label{sect:exp}
\subsection{Matrix factorization}
The first matrix factorization model to be implemented was ALS.
Matrices $U$ and $V$ were initialized with the SVD decomposition of the normalized initial matrix (column-level) imputed with 0s.
The results for this model are included in Table \ref{table:results}.
A non-greedy tuning was performed, but none of the alternative parameters (rank, epoch, initialization, and imputation) improved the baseline score.

The SVD-based factorization model was also implemented from scratch. The normalization and imputation steps were the same as for ALS. The optimal rank ($k=5$) was obtained using validation.
A wide range of different parameters was tested, but with no improvement on the validation score.
This model did not perform well, as shown by the validation score in in Table \ref{table:results}.

FunkSVD was on the other hand not developed from scratch. Instead, we adapted to our framework an existing Python implementation \cite{bolmier}.
FunkSVD achieved better results compared to SVD, but did not improve on the baseline score

We used myFM Python library \cite{myFM} to implement the BFM
Firstly, we tried the most basic version the BFM: only the one-hot encoded users and movies relevant to each rating are provided in the sparse feature matrix $(u, i)$.
The only hyper-parameters to tune are the embedding dimension, i.e. the latent feature dimension $k$, and the number of iterations. Increasing the former decreases the validation error, reaching a plateau at around 50. Increasing the latter
generally improves the model, but the validation RMSE start decreasing beyond 500 iterations.  Therefore, for the rest of our experimentation we chose to set them at 50 and 500, respectively.

\begin{figure}[!b]
    \centering
    \includegraphics[width=.9\columnwidth]{Images/rank_analysis.pdf}
    \caption{Rank analysis for matrix factorization techniques.}
    \label{fig:rankanalysis}
\end{figure}

We decided to experiment with adding implicit features \cite{Oard1998ImplicitFF}, as shown in the myFM \cite{myFM} documentation.
Each row has the format $(u, i, iu)$, where $iu$, a sparse vector of size the number of movies overall, represents all the movies rated by that row's user. This is done by setting the values at the relevant indices of $iu$ to a normalized value $1/\sqrt{\left|N_u\right|}$, with $N_u$ the number of movies rated by user $u$, or to $1$ if $N_u = 0$ \cite{Koren2008FactorizationMT}. We also added implicit movie information in a similar fashion, showing all the users that have watched a movie for every movie, the complete feature matrix now being of the form $(u, i, iu, ii)$ \cite{onthediffic}. As can be observed in Table \ref{table:results}, for regression BFM the biggest change came through the addition of $iu$, bringing our validation score from 0.98034 to 0.97492. Adding $ii$ to that further improves our RMSE to 0.97268.

Furthermore, myFM allows to use ordered probit, a type of ordinal classification
This means that the model also learns four cutpoints, separating the real domain into five sections, one for each rating.
We are then able to obtain the probabilities of a rating being in each of these categories, and we get the final rating prediction by calculating the expected value of the category. Conceptually, this way of framing the problem seems more natural, since ratings are always given as round numbers. Here, the Gibbs sampler for ordered probit on the BFM uses a Metropolis-within-Gibbs scheme \cite{myFM}, since on its own, Gibbs sampling does not work for classification problems beyond binary classification.
The ordered probit implementation consistently outperformed the standard regression, decreasing the best RMSE from 0.97268 to 0.97032 (see Table \ref{table:results}).
This may be explained by the fact that ratings are fundamentally discrete values and not necessarily linear, and hence ordered probit might be able to better model them.

The most interesting analysis that can be performed to compare different matrix-factorization models is on their performances when varying the number of latent features $k$.
We show the result of this investigation in Figure \ref{fig:rankanalysis}.
It can be observed that, while all the other factorization techniques achieve a lowest point in the RMSE for lower ranks ($<10$),
BFM continue to improve their validation score even for larger $k$.
This behaviour was observed also in \cite{onthediffic}.

\subsection{Similarity Based}

For what concerns similarity-based models, we investigated different similarity functions, weighting functions, and a few different hyper-parameters.
It was not possible to conduct a complete grid-search over all the possible combinations of parameters due to time constraints.
Therefore, we fixed some of them and optimized over only the most promising.
In our experiments, we combined the 3 possible uses of similarity (item-item, user-user, and both together) with the 3 similarity functions (PCC, cosine and SiGra), the 4 weightings (normal, significance, sigmoid, and no weighting) and different number of neighbors (1, 3, 6, 10, 30 and 10000), for a total of 162 trained models.
The fixed parameters were the significance threshold $\beta$ (set to 7 for user similarity, 70 for item similarity, and 20 when we used both) and the prediction weights used when both similarity methods were used (set to 0.5).

Finally, we also tested Stochastic CSR algorithm using the original similarity model (PCC similarity with normal weighting and all the neighbors). The model was trained for 15 iterations and a sample size of 15 items.
Table \ref{table:results} shows some of the best scoring models for each categories we obtained during this experiment.

In general, the models that used item-item similarity managed to achieve lower RMSE than the one using the user-user similarity.
Since the loss for the user-based methods was in average higher, the ensemble using both methods also scored worse than the item-based methods with the initial user weight (0.5). However, by reducing this weight to 0.06, we managed to improve the item-based methods as we expected, since more information is available to the model.

The similarity function which achieved the best results was PCC with normal weighting. SiGra performed worse than we expected and the significance weighting or the sigmoid weighting were not as effective as the normal one. Furthermore, the best number of neighbors happened to be 60 for our best scoring model.

Finally, our novel extension of CSR algorithm, SCSR, greatly reduced the computational cost of the original algorithm, but also decreased it's effectiveness, performing worse than the model used for its initialization. We speculate that this might be due to an excessively small sample size, which might not be able to retain enough information to improve the initial solution.
However, this algorithm was not sufficiently explored due to time constraints, and a more in-depth analysis of its performance could be carried out in future works.

\subsection{Neural Based}
Even conducting an extensive hyper-parameters search, we were not able to bring neural-based methods to stand-alone competitive results. DeepRec still showed decent performances (Table \ref{table:results}) with an unconstrained 3 layer AE architecture (512,128,1024,128,512) that could be blended in our ensemble techniques.
LightGCN showed its strongest (yet poor) results on 4 layers with embedding size of 64.
Overall, DeepRec is fast to train (according to layers architecture) and has demonstrated decent results, while LightGCN is much longer to train (1-60) and performs systematically worse.

\subsection{Blending}
Different blending techniques were investigated in our work. Our approach consisted in fitting different CF algorithms on a 80 split of the data and predicting the remaining 20, which became the training dataset for blending models.
We experimented with different regression models, namely (regularized) linear regression, random forest, MLP, and XGBoost.
Inspired by \cite{Freudenthaler_bayesianfactorization}, we included in our ensemble all the BFM models, and then added other explored CF algorithms based on repeated k-fold cross-validation results.

Table \ref{table:results} shows the results obtained by the different combinations of CF and regression models.
Linear regression proved to be the most effective regression model (for the others we report only the RMSE on the best feature set), and our final blending model was hence of a linear combination of the different input predictions.

The most effective combination of CF models consisted of a  blending of 8 BFM and 5 similarity based models on PCC.
An interesting phenomenon observed during this experiment was that none of the other factorization models was able to improve the ensemble score. We speculate that this happens because BFM already extract the maximum information among factorization techniques.

On the other hand, similarity-based approaches, despite their higher RMSE, had a positive impact on the ensemble.
Blending factorization-based and similarity-based predictions allowed us to greatly decrease the validation error by an additional 9.4\% (with respect to the difference between the best single model and the baseline score), proving once again the effectiveness of blending methods in CF and showing that BFM can benefit from being combined with similarity-based approaches.

\section{Conclusion}
\label{sect:conclusion}
In this work, we presented the task of Collaborative Filtering and explored a wide range of models that can be used to tackle it.
We showed that older matrix-factorization methods, in particular BFM, greatly outperform all the other investigated techniques, and that blending their predictions with similarity-based techniques can further improve their performances.
Moreover, we proposed a stochastic variation of a similarity-based approach, SCSR, which consistently reduce the asymptotic complexity of the original algorithm.
Our final submission to the Kaggle competition, predicted with the ensemble of BFM and similarity models, achieved a public RMSE of 0.96374, placing our team first on the public leaderboard among the 16 teams which joined the 2022 Collaborative Filtering competition.
\clearpage

\onecolumn
\vspace*{\fill}
\renewcommand{\arraystretch}{1.2}
\begin{table*}[h]
    \centering
  \begin{tabular}{|c | c | c | l |  c  c|}
    \hline
    \textbf{Category} & \textbf{Method} & \textbf{Model} & \textbf{Parameters} & \textbf{Validation} & \textbf{Public Score} \\
    \hline
    \multirow{10}{*}{Memory-based}  & \multirow{9}{*}{Similarity} & Item (1) & PCC, normal, 30 nn. & 0.99012 & 0.98265\\
    &&Item & PCC, normal, 60 nn. &  0.98858 & 0.98174\\
    &&Item (2)& PCC, normal, all nn. & 0.98944 & 0.98388\\
    &&Item (3)& PCC, None, 30 nn. & 0.99069 & 0.98279\\
    &&Item (4)& PCC, None, all nn. & 0.99105 & 0.96454\\
    &&Item (6) & SiGra, all nn. & 1.00258 & -\\
    &&User & PCC, normal, all nn. & 1.00025 & -\\
    &&Both (7) & Cosine , normal, 30 nn., 0.5 w. & 0.99568 & 0.99052\\
    &&Both & PCC, normal, all nn., 0.5 w. & 0.99941 & - \\
    &&Both (5) & PCC, normal, 30 nn., 0.06 w. & 0.98767 & 0.98009\\
    &&Both & PCC, normal, 60 nn., 0.06 w. & 0.98755 & 0.98069\\
    \cline{2-6}
    &\multirow{2}{*}{Iterative similarity}  &  \multirow{2}{*}{Stochastic CSR} & PCC, normal, all nn., 0.5 w. & \multirow{2}{*}{1.00578}&\multirow{2}{*}{-}\\
    &&& 15 samples, 15 iter, $\alpha=0.5$ && \\
    \hline
    \multirow{22}{*}{Model-based}  & \multirow{10}{*}{Matrix factorization} & ALS & rank 3, $\lambda=0.1$, 20 iter  & 0.98865 & 0.98747 \\
    && SVD & rank 5 & 1.01240 & -  \\
    && FunkSVD & rank 3, $\eta$ = 1e-3, $\lambda$=5e-3 & 0.99880 & 0.99892 \\
    \cline{3-6}
    && BFM regression (8)& k = 50, 500 iters & 0.98034 & - \\
    && BFM r. [u, i, iu] (9)& k = 50, 500 iters & 0.97492 & - \\
    && BFM r. [u, i, ii] (10) & k = 50, 500 iters & 0.97773 & - \\
    && BFM r. [u, i, iu, ii] (11)& k = 50, 500 iters & 0.97268 & - \\
    && BFM ordered probit (12) & k = 50, 500 iters & 0.97668 & - \\
    && BFM o.p. [u, i, iu] (13)& k = 50, 500 iters & 0.97191 & - \\
    && BFM o.p. [u, i, ii] (14)& k = 50, 500 iters & 0.97527 & - \\
    && BFM o.p. [u, i, iu, ii] (15) & k = 50, 500 iters & 0.97032 & 0.96543 \\
    \cline{2-6}
    & \multirow{2}{*}{Neural} & DeepRec (16) & $\eta=1e-3$, batch 128, 300 epochs  &  0.98777 & 0.98559\\
    & & LightGCN & $\eta=1e-4$, batch 2048, 250 epochs &  0.99987 & 0.99929\\
    \cline{2-6}
    &\multirow{12}{*}{Blending} &  \multirow{7}{*}{Linear Regression} & (5) + (15) &  0.96968 & 0.96472\\
    &&& (5) + (6) + (7) + (15) & 0.96956 & -\\
    &&& (15) + (1)-(5) & 0.96939 & 0.96454\\
    &&& (5) + (15) + (16) & 0.969691 & -\\
    &&& (15) + (1)-(7) & 0.96940 & -\\
     &&& (8)-(15) & 0.96981 & -\\
     &&& (8)-(15) + (1)-(5) &\textbf{ 0.96870} & \textbf{0.96374}\\
    \cline{3-6}
    & &  Lasso & best, $\alpha=0.001$ &  0.96930	 & - \\
    & &  Ridge & best, $\alpha=0.01$ &  0.96870	 & - \\
    & &  MLP & best, hidden\_size=100, iter=1000 &   0.97937 & -\\
    & &  XGBoost &best,  n\_est=100, depth=7 &  0.97032 & -\\
    & &  Random Forest &best,  n\_est=100, depth=2 & 0.98367 & -\\
    \hline
  \end{tabular}
  \vspace*{0.2cm}
  \caption{Validation and submissions results.}
  \label{table:results}
\end{table*}
\vspace*{\fill}
\clearpage
\twocolumn

\bibliographystyle{IEEEtran}