\section{Introduction}
\hspace{1.0cm}As recently pointed out by Gamboa et al \cite{1} the cornstone of cosmology is the cosmological principle which is based on the fact that
the spacetime is Lorentz invariant. Nevertheless, physical problems such as that primordial magnetic fields (PMF)
\cite{2}, matter antimatter asymmetry and dark energy (matter) lead us to rethink Lorentz invariant Einsteinian relativity dogma. One of the ways to address this issue
has been recently discovered by Kostelecky et al \cite{3}, which considers that LV could be associated to another alternative gravity theory
gravity theory called torsion theory \cite{4}. An important issue at this point is to stress that here as in most string inspired
Kalb Rammond theory, torsion propagates in vacuum instead of being considered as a contact interaction as in Einstein-Cartan gravitation
\cite{5}.
 In this paper we shall be concerned with some examples where LV not only is present in spacetime but it can be used to place
limits in spacetime torsion by its manifestations on dynamo effects and GBRs and cosmic microwave background (CMB)
with an interesting analogy of electromagnetic (EM) waves in magnetised plasmas \cite{6}. Faraday rotation, which is so
important in measuring magnetic fields \cite{7} is used to place constraints on LV torsion vector. An important issue is that here one addopts perturbative approach to quantum electrodynamics
(QED) effects and not the non-perturbative ones used by Enqvist et al \cite{8}. The idea of using this EW analogy in GRB
ultraviolet LV has been used by Kahniatishvilly et al \cite{9} in Minkowski spacetime without torsion. One of the main diffrences
between their results and ours, is that to EM waves frequency a torsion wave frequency is summed up, and only in the very low frequency
torsion waves, they approach. In this first paper one takes in account this low frequency limit and left the high frequency torsion wave limit
to a future work. One also consider here a linearised approach to dispersive and rotations measures. The paper is
organised as follows: In section II the de Sabbata-Gasperini formulation of the Riemann-Cartan (RC) Maxwells
vacuum electrodynamics, with photon-torsion semi-minimal coupling is reviewed. In section III the EM waves analogy in magnetised plasma is considered and
the CMB torsion limit is placed in the low frequency torsion wave limit. In section IV galactic magnetic dynamo seeds are used to place limits on LV through
torsion modes. In section V conclusions and discussions are presented.
\section{Perturbative QED in Minkowski torsioned spacetime}
Throughout the paper second order effects on torsion
shall be neglected in the electrodynamics including LV terms due to the three-dimensionl torsion vector $\textbf{Q}$. In this section we consider a
simple cosmological application concerning the electrodynamics in vacuum QED
spacetime background. Perturbative approach to electrodynamics leads to the following set of equations
\begin{equation}
{\partial}_{i}F^{ji}=4{\pi}J^{j}+\frac{2{\alpha}}{3{\pi}}{\epsilon}^{jklm}Q_{l}{F_{kj}}
\label{1}
\end{equation}
\begin{equation}
{\partial}_{[i}F_{jk]}=0
\label{2}
\end{equation}
where ${\partial}_{i}$ is the partial derivative, and $\alpha$ is the e.m fine structure constant, while $F^{ij}={\partial}^{i}A^{j}-{\partial}^{j}A^{i}$
is the electromagnetic field tensor non-minimally coupled to torsion gravity. Here $A^{i}$ is the electromagnetic
vector potential and $(i,k=0,1,2,3)$ and $Q_{l}$ represents the torsion four-vector. In three-dimensional notation the above Maxwells generalised equations read
\begin{equation}
{\nabla}.{\textbf{E}}=4{\pi}{\rho}+\frac{4{\alpha}}{3{\pi}}\textbf{Q}.\textbf{B}
\label{3}
\end{equation}
\begin{equation}
{\nabla}.\textbf{B}=0
\label{4}
\end{equation}
\begin{equation}
{\nabla}{\times}\textbf{E}= -\frac{{\partial}{\textbf{B}}}{{\partial}t}
\label{5}
\end{equation}
\begin{equation}
{\nabla}\times{\textbf{B}}=\frac{4{\alpha}}{3{\pi}}\textbf{E}{\times}\textbf{Q}+\frac{{\partial}\textbf{E}}{{\partial}t}
\label{6}
\end{equation}
After some algebraic manipulation on these generalised Maxwell equations one obtains the EM wave equations
\begin{equation}
{\nabla}^{2}\textbf{E} -\frac{{\partial}^{2}{\textbf{E}}}{{\partial}t^{2}}+
\frac{4{\alpha}}{3{\pi}}[\textbf{Q}{\times}\frac{{\partial}\textbf{E}}{{\partial}t}-\textbf{E}{\times}\frac{{\partial}\textbf{Q}}{{\partial}t}]
=0\label{7}
\end{equation}
\begin{equation}
{\nabla}^{2}\textbf{B} -\frac{{\partial}^{2}{\textbf{B}}}{{\partial}t^{2}}-
\frac{16{\alpha}}{3}{\rho}\textbf{Q}-(\frac{4{\alpha}}{3{\pi}})^{2}\textbf{Q}(\textbf{Q}.\textbf{B})=0
\label{8}
\end{equation}
By Fourier analyzing the first expression or substituting ${\partial}_{t}\rightarrow{i{\omega}}$ and
${\nabla}\rightarrow{-ik}$ one obtains from expression (\ref{7}) the following expression
\begin{equation}
[({\omega}^{2}-k^{2}){\delta}_{ab}-\frac{4{\alpha}}{3{\pi}}i({\omega}_{1}+{\omega}){\epsilon}_{acb}Q^{c}]E^{b}=0
\label{9}
\end{equation}
\begin{equation}
ik_{a}E_{a}=\frac{4{\alpha}}{3{\pi}}Q_{c}E_{c}
\label{10}
\end{equation}
where $(a,b=1,2,3)$, ${\omega}$ is the EM wave frequency while ${\omega}_{1}$ is the torsion wave frequency. Here we also chose the charge density
${\rho}=0$ since we are addopting vacuum QED. The dispersion relation is given by
\begin{equation}
{\omega}^{2}\mp({\omega}_{1}+{\omega})kQ-k^{2}[1\mp{\gamma}]=0
\label{11}
\end{equation}
where ${\gamma}(k)$ is the photon-spin-sign-dependent term on the LHS of equation (\ref{11}), to account for the
phenomenological LV of an energy-dependent photon speed. Now by considering the analogy to EM waves in a magnetised plasma with an index of refraction
of refraction of $n=\frac{k}{\omega}$, one obtains
\begin{equation}
n_{L,R}=({\epsilon}_{1}\pm{\epsilon}_{2})^{\frac{1}{2}}
\label{12}
\end{equation}
where ${\epsilon}$ is the electric permittivity. From the dispersion relation above one obtains
the permittivities
\begin{equation}
{\epsilon}_{1}=\frac{1}{(1\pm{\gamma}(k))}
\label{13}
\end{equation}
\begin{equation}
{\epsilon}_{2}=-\frac{({\omega}_{1}+{\omega})Q}{(1\pm{\gamma}(k))}\approx{({\omega}+{\omega}_{1})}Q
\label{14}
\end{equation}
From the approximation of low torsion frequency ${\omega}_{1}<<<{\omega}$, this can be dropped in the last expression and torsion
vector reduces to the $\textbf{g}$ LV vector used by Khniatishivilly et al. Thus these expressions one obtains the refraction index
\begin{equation}
{n}_{L,R}=(1\pm{\omega}Q\pm{\gamma}(k))^{\frac{1}{2}}
\label{15}
\end{equation}
By making the approximation ${\gamma}<<Q{\omega}$ the refractive index reduces to
\begin{equation}
{n}_{L,R}\approx{(1\pm{\omega}Q)^{\frac{1}{2}}}=\frac{k}{\omega}
\label{16}
\end{equation}
Now the dispersion measure and rotation measure (RM) of the GRBs depend on the photon travel distance ${\Delta}l$ and are expressed as
\begin{equation}
{\Delta}t_{L,R}={\Delta}l(1-\frac{{\partial}k_{L,R}}{{\partial}{\omega}})
\label{17}
\end{equation}
\begin{equation}
{\Delta}{\phi}=\frac{1}{2}(k_{L}-k_{R}){\Delta}l
\label{18}
\end{equation}
where ${\phi}$ is the polarization plane rotation of the electric field describing the Faraday rotation.
These expressions can be written in terms of torsion by
\begin{equation}
{\Delta}t_{L,R}=\pm{\Delta}l{\omega}Q
\label{19}
\end{equation}
\begin{equation}
{\Delta}{\phi}\approx{\frac{1}{2}{\omega}^{2}Q{\Delta}l}
\label{20}
\end{equation}
Therefore, when the photon-spin is damped by the torsion wave, the Faraday rotation of
${\Delta}{\phi}\approx{10^{-2}rad}$, allows one to estimate torsion as
\begin{equation}
Q_{CMB}\approx{{10}^{-18}GeV}
\label{21}
\end{equation}
thus stablishing new limits for LV from torsin distinct from those of Kostelecky et al.
\section{Galactic dynamo seeds constraints to LV in spacetime with torsion}
Recently a more complicated approach to place constraints on LV from galactic dynamo magnetic seed fields appeared in
the literature \cite{3,4}. Here folllowing the perturbative methd bove nd the magnetic field equation one obtains a much simpler
and straightforward method of placing limits on LV from torsion and galactic dynamo seeds. Performing the Fourier spectrum of the
magnetic field equation yields
\begin{equation}
[({\omega}^{2}-k^{2}){\delta}_{ab}+\frac{16{\alpha}^{2}}{9{\pi}}Q_{a}Q_{b}]B^{b}=0
\label{22}
\end{equation}
which yields the following dispersion relation
\begin{equation}
{\omega}^{2}= k^{2}+\frac{16{\alpha}^{2}}{9{\pi}^{2}}Q^{2}
\label{23}
\end{equation}
Actually ${\omega}$ coincides with dynamo growth rate ${\gamma}_{0}$ with the ansatz
\begin{equation}
B(t)= B_{0}e^{{\gamma}t}
\label{24}
\end{equation}
From the dispersion relation one may conclude that in order that torsion may contribute to dynamo action it must be
be comparable with the large scale coherence which is the inverse of the wave vector k, under the law
\begin{equation}
k^{2}\approx{\frac{16{\alpha}^{2}}{9{\pi}^{2}}Q^{2}}
\label{25}
\end{equation}
As for today coherence scales torsion would be extremely weak and of the order of $Q\approx{10^{-21}cm^{-1}}$.
This is exactly the estimate obtained by Laemmerzahl \cite{11} on the basis of Earth laboratory Hughes-Drever experiment. It is interesting to note that Kostelecky et al
have also obtained LV with table top experiments on Earth lab.
\section{Discussion and conclusions}
The investigation of Faraday rotation has been proved very important in high-energy astronomy of magnetic fields in the
universe. Here one uses Faraday rotation to stablish limits of the LV in terms of torsion as stblished by Kostelecky
et al with torsion being a constant vector. Actually here torsion vector is not constant though LV is attainble. Methods used here were previously
investigated by Kahniatshivilly et al in the context of GRBs in torsionless Minkowski spacetime.
Dynamo plasma is obtained from the dispersion relation where torsion can be express3ed in terms of the coherence
scale of magnetic fields. Quantum effects may be obtained here from perturbative QED instead of non-perturbative primordial magnetic fields obtained by Enqvist.
\section*{Acknowledgements}
 I would like to express my gratitude to Tina Kahniashivily and Rodion Stepanov for helpful discussions on the
subject of this paper. I also thank Tanmay Vachaspati for his kind invitation to the Primordial magnetic fields 2011 held in rizona State
University last april. Thanks are due to F Hehl for some useful  correspondence on torsion waves. I also benifit from discussions with some participants like Bharat Ratra. Financial support from CNPq. and University of State of Rio de Janeiro (UERJ) are grateful acknowledged.


\section{Introduction}
\label{sec:intro}
The current success of deep learning models is showing how modern artificial intelligent systems can manage supervised machine learning tasks with growing accuracy. However, when the level of supervision decreases, all the limitations of the existing data-hungry approaches become evident. For many applications, large amount of supervised data are not readily available, moreover collecting and manually annotating such data may be difficult or very costly. Different sub-fields of computer vision, such as \emph{domain adaptation} \cite{csurka} and \emph{self-supervised learning} \cite{doersch2015unsupervised}, aim at designing new learning solutions to compensate for this lack of supervision. Domain adaptation focuses on leveraging a fully supervised data-rich source domain to learn a classification model that performs well on a different but related unlabeled target domain.
Traditional domain adaptation methods assume that the target contains exactly the same set of labels of the source (\emph{closed-set} scenario). In recent years, this constraint has been relaxed in favor of the more realistic \emph{open-set} scenario where the target also contains samples drawn from unknown classes. In this case, it becomes important to identify and isolate the unknown class samples before reducing the domain shift to avoid negative transfer. Self-supervised learning focuses on training models on pretext tasks, such as image colorization or rotation prediction, using unlabeled data to then transfer the acquired high-level knowledge to new tasks with scarce supervision.
Recent literature has highlighted how self-supervision can be used for domain adaptation: jointly solving a pretext self-supervised task together with the main supervised problem leads to learning robust cross-domain features and supports generalization \cite{xu2019self-supervised,Carlucci_2019_CVPR}. Other works have also shown that the output of self-supervised models can be used in anomaly detection to discriminate normal and anomalous data  \cite{golan2018deep,Bergman2020Classification-Based}. However, these works only tackle binary problems (normal and anomalous class) and deal with a single domain.

In this paper, we propose for the first time to use the inherent properties of self-supervision both for cross-domain robustness and for novelty detection to solve \emph{Open-Set Domain Adaptation} (OSDA). To this purpose, we propose a two-stage method called \emph{Rotation-based Open Set} (ROS) that is illustrated in Figure \ref{fig:scheme}. In the first stage, we separate the known and unknown target samples by training the model on a modified version of the rotation task that consists in predicting the relative rotation between a reference image and the rotated counterpart. In the second stage, we reduce the domain shift between the source domain and the known target domain using, once again, the rotation task. Finally we obtain a classifier that predicts each target sample as either belonging to one of the known classes or rejects it as unknown.
While evaluating ROS on the two popular benchmarks \emph{Office-31}~\cite{saenko2010adapting} and \emph{Office-Home}~\cite{venkateswara2017deep}, we expose the reproducibility problem of existing OSDA approaches and assess them with a new evaluation metric that better represents the performance of open set methods.
\noindent \textbf{We can summarize the contributions of our work as following}:
\begin{enumerate}
    \item we introduce a novel OSDA method that exploits rotation recognition to tackle both known/unknown target separation and domain alignment;
    \item we define a new OSDA metric that properly accounts for both known class recognition and unknown rejection;
    \item we present an extensive experimental benchmark against existing OSDA methods with two conclusions: (a) we put under the spotlight the urgent need of a rigorous experimental validation to guarantee result reproducibility; (b) our ROS defines the new state-of-the-art on two benchmark datasets.
\end{enumerate}

\noindent
A Pytorch implementation of our method, together with instructions to replicate our experiments, is available at \url{https://github.com/silvia1993/ROS} .

\begin{figure*}[t!]
    \centering
    \includegraphics[width=\linewidth]{img/teaser.pdf}
    \caption{Schematic illustration of our Rotation-based Open Set (\textbf{ROS}). Stage I: the source dataset $\mathcal{D}_s$ is used to train the encoder $E$, the semantic classifier $C_1$, and the multi-rotation classifier $R_1$ to perform known/unknown separation. $C_1$ is trained using the features of the original image, while $R_1$ is trained using the concatenated features of the original and rotated image. After convergence, the prediction of $R_1$ on the target dataset $\mathcal{D}_t$ is used to generate a normality score that defines how the target samples are split into a known target dataset $\mathcal{D}_t^{knw}$ and an unknown target dataset $\mathcal{D}_t^{unk}$. Stage II: $E$, the semantic+unknown classifier $C_2$ and the rotation classifier $R_2$ are trained to align the source and target distributions and to recognize the known classes while rejecting the unknowns. $C_2$ is trained using the original images from $\mathcal{D}_s$ and $\mathcal{D}_t^{unk}$, while $R_2$ is trained using the concatenated features of the original and rotated known target samples.}
    \label{fig:scheme}
\end{figure*}

\section{Related Work}
\label{sec:related}
\textbf{Self-supervised learning} applies the techniques of supervised learning on problems where external supervision is not available. The idea is to manipulate the data to generate the supervision for an artificial task that is helpful to learn useful feature representations. Examples of self-supervised tasks in computer vision include predicting the relative position of image patches~\cite{doersch2015unsupervised,noroozi2016}, colorizing a gray-scale image~\cite{zhang2016colorful,Larsson_2017_CVPR}, and inpainting a removed patch~\cite{pathakCVPR16context}. Arguably, one of the most effective self-supervised tasks is rotation recognition~\cite{gidaris2018unsupervised} that consists in rotating the input images by multiples of $90^{\circ}$ and training the network to predict the rotation angle of each image. This pretext task has been successfully used in a variety of applications including anomaly detection~\cite{golan2018deep} and closed-set domain adaptation~\cite{xu2019self-supervised}.

\textbf{Anomaly detection}, also known as outlier or novelty detection, aims at learning a model from a set of \emph{normal} samples to be able to detect out-of-distribution (\emph{anomalous}) instances. The research literature in this area is wide with three main kind of approaches. \emph{Distribution-based} methods~\cite{zimek2012survey,rKDE,deepanomaly,zong2018deep} model the distribution of the available normal data so that the anomalous samples can be recognized as those with a low likelihood under the learned probability function. \emph{Reconstruction-based} methods~\cite{Eskin2002,rPCA,Xia_2015_ICCV,autoenc,schlegl2017unsupervised} learn to reconstruct the normal samples from an embedding or a set of basis functions. Anomalous data are then recognized by having a larger reconstruction error with respect to normal samples. \emph{Discriminative} methods~\cite{OSVM,ruff18a,HendrycksG17,liang2018enhancing} train a classifier on the normal data and use its predictions to distinguish between normal and anomalous samples.

\textbf{Closed-set domain adaptation} (CSDA) accounts for the difference between source and target data by considering them as drawn from two different marginal distributions. The literature of DA can be divided into three groups based on the strategy used to reduce the domain shift. \emph{Discrepancy-based} methods~\cite{Long:2015,sun2016return,Xu_2019_ICCV} define a metric to measure the distance between source and target data in feature space. This metric is minimized while training the network to reduce the domain shift. \emph{Adversarial} methods~\cite{ganin2016domain,tzeng2017adversarial,russo2018from} aim at training a domain discriminator and a generator network in an adversarial fashion so that the generator converges to a solution that makes the source and target data indistinguishable for the domain discriminator. \emph{Self-supervised} methods~\cite{ghifary2016deep,bousmalis2016domain,Carlucci_2019_CVPR} train a network to solve an auxiliary self-supervised task on the target (and source) data, in addition to the main task, to learn robust cross-domain representations.

\textbf{Open Set Domain Adaptation} (OSDA) is a more realistic version of CSDA, where the source and target distribution do not contain the same categories. The term ``OSDA" was first introduced by Busto and Gall~\cite{Busto_2017_ICCV} that considered the setting where each domain contains, in addition to the shared categories, a set of private categories. The currently accepted definition of OSDA was introduced by Saito~\emph{et al}\onedot~\cite{saito2018open} that considered the target as containing all the source categories and additional set of private categories that should be considered \emph{unknown}. To date, only a handful of papers tackled this problem. \emph{Open Set Back-Propagation}~(OSBP)~\cite{saito2018open} is an adversarial method that consists in training a classifier to obtain a large boundary between source and target samples whereas the feature generator is trained to make the target samples far from the boundary. \emph{Separate To Adapt}~(STA)~\cite{liu2019separate} is an approach based on two stages. First, a multi-binary classifier trained on the source is used to estimate the similarity of target samples to the source. Then, target data with extreme high and low similarity are re-used to separate known and unknown classes while the features across domains are aligned through adversarial adaptation. \emph{Attract or Distract}~(AoD)~\cite{feng2019attract} starts with a mild alignment with a procedure similar to~\cite{saito2018open} and refines the decision by using metric learning to reduce the intra-class distance in known classes and push the unknown class away from the known classes. \emph{Universal Adaptation Network}~(UAN)\footnote{UAN is originally proposed for the universal domain adaptation setting that is a superset of OSDA, so it can also be used in the context of this paper.}~\cite{you2019universal} uses a pair of domain discriminators to both generate a sample-level transferability weight and to promote
the adaptation in the automatically discovered common label set. Differently from all existing OSDA methods, \textbf{our approach abandons adversarial training in favor of self-supervision. Indeed, we show that rotation recognition can be used, with tailored adjustments, both to separate known and unknown target samples and to align the known source and target distributions}\footnote{See Appendix E for a discussion on the use of other self-supervised tasks.}.

\section{Method}
\label{sec:method}
\subsection{Problem formulation}
\label{subsec:formulation}
Let us denote with $\mathcal{D}_s=\{(\boldsymbol{x}_j^s,y_j^s)\}_{j=1}^{N_s} \sim p_s$ the labeled source dataset drawn from distribution $p_s$ and $\mathcal{D}_t=\{\boldsymbol{x}^t_j\}_{j=1}^{N_t} \sim p_t$ the unlabeled target dataset drawn from distribution $p_t$. In OSDA, the source domain is associated with a set of \emph{known} classes $y^s \in \{1,\ldots, |\mathcal{C}_s |\}$ that are shared with the target domain $\mathcal{C}_s\subset\mathcal{C}_t$, but the target covers also a set $\mathcal{C}_{t \setminus s}$ of additional classes, which are considered \emph{unknown}. As in CSDA, it holds that $p_s\neq p_t$ and we further have that $p_s\neq p_t^{\mathcal{C}_s}$, where $p_t^{\mathcal{C}_s}$ denotes the distribution of the target domain belonging to the shared label space $\mathcal{C}_s$. Therefore, in OSDA we face both a domain gap ($p_s\neq p_t^{\mathcal{C}_s}$) and a category gap ($\mathcal{C}_s \neq \mathcal{C}_t$). OSDA approaches aim at assigning the target samples to either one of the $|\mathcal{C}_s |$ shared classes or to reject them as \emph{unknown} using only annotated source samples, with the unlabeled target samples available transductively. An important measure characterizing a given OSDA problem is the \emph{openness} that relates the size of the source and target class set. For a dataset pair $(\mathcal{D}_s, \mathcal{D}_t )$, following the definition of~\cite{bendale2016towards}, the openness $\mathbb{O}$ is measured as $\mathbb{O}=1-\frac{|\mathcal{C}_s |}{|\mathcal{C}_t |}$. In CSDA $\mathbb{O} = 0$, while in OSDA $\mathbb{O} > 0$.

\subsection{Overview}
\label{subsec:overview}
When designing a method for OSDA, we face two main challenges: \emph{negative transfer} and \emph{known/unknown separation}. Negative transfer occurs when the whole source and target distribution are forcefully matched, thus also the unknown target samples are mistakenly aligned with source data. To avoid this issue, cross-domain adaptation should focus only on the shared $\mathcal{C}_s$ classes, closing the gap between $p^{\mathcal{C}_s}_t$ and $p_s$.
This leads to the challenge of known/unknown separation: recognizing each target sample as either belonging to one of the shared classes $\mathcal{C}_s$ (known) or to one of the target private classes $\mathcal{C}_{t \setminus s}$ (unknown). Following these observations, we structure our approach in two stages: (i) we separate the target samples into known and unknown, and (ii) we align the target samples predicted as known with the source samples (see Figure \ref{fig:scheme}). The first stage is formulated as an anomaly detection problem where the unknown samples are considered as anomalies. The second stage is formulated as a CSDA problem between source and the known target distribution. Inspired by recent advances in anomaly detection and CSDA~\cite{xu2019self-supervised,golan2018deep}, we solve both stages using the power of self-supervision. More specifically, we use two variations of the rotation classification task to compute a normality score for the known/unknown separation of the target samples and to reduce the domain gap.

\subsection{Rotation recognition for open set domain adaptation}
Let us denote with $rot90(\boldsymbol{x},i)$ the function that rotates clockwise a 2D image $\boldsymbol{x}$ by $i\times90^{\circ}$. Rotation recognition is a self-supervised task that consists in rotating a given image $x$ by a random $i \in [1,4]$ and using a CNN to predict $i$ from the rotated image $\tilde{\boldsymbol{x}}=rot90(\boldsymbol{x},i)$.
We indicate with $|r|=4$ the cardinality of the label space for this classification task.
In order to effectively apply rotation recognition to OSDA, we introduce the following variations.

\textit{Relative rotation:}
Consider the images in Figure~\ref{fig:relative_rotation}. Inferring by how much each image has been rotated without looking at its original (non-rotated) version is an ill-posed problem since the pens, as all the other object classes, are not presented with a coherent orientation in the dataset. On the other hand, looking at both original and rotated image to infer the relative rotation between them is well-defined. Following this logic, we modify the standard rotation classification task~\cite{gidaris2018unsupervised} by introducing the original image as an anchor. Finally, we train the rotation classifier to predict the rotation angle given the concatenated features of both original (anchor) and rotated image. As indicated by Figure  \ref{fig:relative_rotation2}, the proposed relative rotation has the further effect of boosting the discriminative power of the learned features. It guides the network to focus more on specific shape details rather than on confusing texture information across different object classes.

\begin{figure}[t!]
\begin{minipage}[b]{0.42\textwidth}
\centering
\includegraphics[width=\textwidth]{img/relativerot.pdf}
\caption{\label{fig:relative_rotation} Are you able to infer the rotation degree of the rotated images without looking at the respective original one?}
\end{minipage}
\hfill
\begin{minipage}[b]{0.52\textwidth}
\centering
\includegraphics[width=\textwidth]{img/discriminativerot3.pdf}
\caption{\label{fig:relative_rotation2} The objects on the left may be confused. The relative rotation guides the network to focus on discriminative shape information}
\end{minipage}\vspace{-2mm}
\end{figure}

\textit{Multi-rotation classification:} The standard setting of anomaly detection considers samples from one semantic category as the normal class and samples from other semantic categories as anomalies. Rotation recognition has been successfully applied to this setting, but it suffers when including multiple semantic categories in the normal class~\cite{golan2018deep}. This is the case when coping with the known/unknown separation of OSDA, where we have all the $|\mathcal{C}_s|$ semantic categories as known data.
To overcome this problem, we propose a simple solution: we extend rotation recognition from a $4$-class problem to a $(4\times |\mathcal{C}_s|)$-class problem, where the set of classes represents the combination of semantic and rotation labels. For example, if we rotate an image of category $y^s=2$ by $i=3$, its label for the multi-rotation classification task is $z^s=(y^s\times 4)+i=11$.
In Appendix E, we discuss the specific merits of the multi-rotation classification task with further experimental evidences.
In the following, we indicate with $\boldsymbol{y},\boldsymbol{z}$ the one-hot vectors respectively for the class and multi-rotation labels.

\subsection{Stage I: known/unknown separation}
\label{subsec:stage1}
To distinguish between the known and unknown samples of $\mathcal{D}_t$, we train a CNN on the multi-rotation classification task using $\tilde{\mathcal{D}}_s=\{(\boldsymbol{x}^s_j, \tilde{\boldsymbol{x}}^s_j, z^s_j)\}^{4\times N_s}_{j=1}$. The network is composed of an encoder $E$ and two heads: a multi-rotation classifier $R_1$ and a semantic label classifier $C_1$.
The rotation prediction is computed on the stacked features of the original and rotated image produced by the encoder $\hat{\boldsymbol{z}}^s=\text{softmax}\big(R_1([E(\boldsymbol{x}^s),E(\tilde{\boldsymbol{x}}^s)])\big)$, while the semantic prediction is computed only from the original image features as $\hat{\boldsymbol{y}}^s=\text{softmax}\big(C_1(E(\boldsymbol{x}^s)\big)$. The network is trained to minimize the objective function $\mathcal{L}_1 = \mathcal{L}_{C_1} + \mathcal{L}_{R_1}$, where the semantic loss $\mathcal{L}_{C_1}$ is defined as a cross-entropy and the multi-rotation loss $\mathcal{L}_{R_1}$ combines cross-entropy and center loss \cite{centerWenZL016}. More precisely,\vspace{-1mm}
\begin{align}
\mathcal{L}_{C_1} &= -\sum_{j \in \mathcal{D}_s} \boldsymbol{y}^s_j \cdot \log(\hat{\boldsymbol{y}}^s_j),\\
    \mathcal{L}_{R_1} &= \sum_{j \in \tilde{\mathcal{D}}_s} -\lambda_{1,1} \boldsymbol{z}^s_j \cdot \log(\hat{\boldsymbol{z}}^s_j) + \lambda_{1,2} ||\boldsymbol{v}^s_j - \gamma(\boldsymbol{z}^s_j)||^2_2, \label{eq:centerloss}\vspace{-6mm}
\end{align}
where $||.||_2$ indicates the $l_2$-norm operator, $\boldsymbol{v}_j$ indicates the output of the penultimate layer of $R_1$ and $\gamma(\boldsymbol{z}_j)$ indicates the corresponding centroid of the class associated with $\boldsymbol{v}_j$. By using the center loss we further encourage the network to minimize the intra-class variations while keeping far the features of different classes. This supports the following use of the rotation classifier output as a metric to detect unknown category samples.

Once the training is complete, we use $E$ and $R_1$ to compute the \emph{normality score} $\mathcal{N} \in [0,1]$ for each target sample, with large $\mathcal{N}$ values indicating normal (known) samples and vice-versa.
We start from the network prediction on all the relative rotation variants of a target sample $\hat{\boldsymbol{z}_i}^t=\text{softmax}\big(R_1([E(\boldsymbol{x}^t),E(\tilde{\boldsymbol{x}}_i^t)])\big)_i$ and their related entropy $H(\hat{\boldsymbol{z}}_i^t)= \big(\hat{\boldsymbol{z}}_i^t \cdot \log(\hat{\boldsymbol{z}}_i^t)/\log|\mathcal{C}_s|\big)_i$ with $i=1,\ldots,|r|$. We indicate with $[\hat{\boldsymbol{z}}^t]_m$ the $m$-th component of the $\hat{\boldsymbol{z}}^t$ vector.
The full expression of the normality score is:
\begin{equation}
    \mathcal{N}(\boldsymbol{x}^t) = \max \Bigg\{ \max_{k=1,\ldots,|\mathcal{C}_s|}\bigg(\sum_{i=1}^{|r|}[\hat{\boldsymbol{z}}_i^t]_{k\times|r|+i}\bigg) , \bigg(1-\frac{1}{|r|}\sum_{i=1}^{|r|}H(\hat{\boldsymbol{z}}_i^t)\bigg)\Bigg\}~.
    \label{eq:normalityscore
\end{equation}
In words, this formula is a function of the ability of the network to correctly predict the semantic class and orientation of a target sample (first term in the braces, \emph{Rotation Score}) as well as of its confidence evaluated on the basis of the prediction entropy (second term, \emph{Entropy Score}). We maximize over these two components with the aim of taking the most reliable metric in each case.
Finally, the normality score is used to separate the target dataset into a known target dataset $\mathcal{D}_t^{knw}$ and an unknown target dataset $\mathcal{D}_t^{unk}$. The distinction is made directly through the data statistics using the average of the normality score over the whole target $\bar{\mathcal{N}}=\frac{1}{N_t}\sum_{j=1}^{N_t}\mathcal{N}_j$,
without the need to introduce any further parameter:
\begin{equation}
\begin{cases}
    \boldsymbol{x}^t \in \mathcal{D}_t^{knw} & \quad \text{if} \quad \mathcal{N}(\boldsymbol{x}^t) > \bar{\mathcal{N}} \\
    \boldsymbol{x}^t \in \mathcal{D}_t^{unk} & \quad  \text{if} \quad \mathcal{N}(\boldsymbol{x}^t) < \bar{\mathcal{N}}~.
\end{cases}
\end{equation}
It is worth mentioning that only $R_1$ is directly involved in computing the normality score, while $C_1$ is only trained for regularization purposes and as a warm up for the following stage. For a detailed pseudo-code on how to compute $\mathcal{N}$ and generate $\mathcal{D}_t^{knw}$ and $\mathcal{D}_t^{unk}$, please refer to Appendix G.

\subsection{Stage II: domain alignment}
\label{subsec:stage2}
Once the target unknown samples have been identified, the scenario gets closer to that of standard CSDA. On the one hand, we can use $\mathcal{D}_t^{knw}$ to close the domain gap without the risk of negative transfer and, on the other hand, we can exploit $\mathcal{D}_t^{unk}$ to extend the original semantic classifier, making it able to recognize the unknown category.
Similarly to Stage I, the network is composed of an encoder $E$ and two heads: a rotation classifier $R_2$ and a semantic label classifier $C_2$. The encoder is inherited from the previous stage. The heads also leverage on the previous training phase but have two key differences with respect to Stage I:
(1) $C_1$ has a $|\mathcal{C}_s|$-dimensional output, while $C_2$ has a $(|\mathcal{C}_s|+1)$-dimensional output because of the addition of the unknown class; (2) $R_1$ is a multi-rotation classifier with a $(4\times|\mathcal{C}_s|)$-dimensional output, $R_2$ is a rotation classifier with a $4$-dimensional output.
The rotation prediction is computed as $\hat{\boldsymbol{q}}=\text{softmax}\big(R_2([E(\boldsymbol{x}),E(\tilde{\boldsymbol{x}})])\big)$  while the semantic prediction is  $\hat{\boldsymbol{g}}=\text{softmax}\big(C_2(E(\boldsymbol{x})\big)$.
The network is trained to minimize the objective function $\mathcal{L}_2 = \mathcal{L}_{C_{2}} + \mathcal{L}_{R_2}$, where $\mathcal{L}_{C_{2}}$ combines the supervised cross-entropy and the unsupervised entropy loss for the classification task, while $\mathcal{L}_{R_2}$ is defined as a cross-entropy for the rotation task.
The unsupervised entropy loss is used to involve in the semantic classification process also the unlabeled target samples recognized as known. This loss enforces the decision boundary to pass through low-density areas.
More precisely,
\begin{align}
\mathcal{L}_{C_{2}} &= -\sum_{j \in \{\mathcal{D}_{s}\cup \mathcal{D}_t^{unk}\}} \boldsymbol{g}_j \cdot \log(\hat{\boldsymbol{g}}_j)
-\lambda_{2,1}\sum_{j \in \mathcal{D}_t^{knw}} \hat{\boldsymbol{g}}_j \cdot \log(\hat{\boldsymbol{g}}_j),\\
\mathcal{L}_{R_2} &=  -\lambda_{2,2}\sum_{j \in {\mathcal{D}}_t^{knw}} \boldsymbol{q}_j \cdot \log(\hat{\boldsymbol{q}}_j)~.
\end{align}
Once the training is complete, $R_2$ is discarded and the target labels are simply predicted as $c_j^t=C_2(E(\boldsymbol{x}_j^t))$ for all $j=1, \ldots, N_t$.

\section{On reproducibility and open set metrics}
\label{sec:metrics}
OSDA is a young field of research first introduced in 2017. As it is gaining momentum, it is crucial to guarantee the \emph{reproducibility} of the proposed methods and have a valid \emph{metric} to properly evaluate them.

\emph{Reproducibility:} In recent years, the machine learning community has become painfully aware of a reproducibility crisis~\cite{henderson2018deep,dodge2019show,lucic2018gans}. Replicating the results of state-of-the-art deep learning models is seldom straightforward due to a combination of non-deterministic factors in standard benchmark environments and poor reports from the authors. Although the problem is far from being solved, several efforts have been made to promote reproducibility through checklists~\cite{checklist}, challenges~\cite{challenge} and by encouraging authors to submit their code. On our side, we contribute by re-running the state-of-the-art methods for OSDA and compare them with the results reported in the papers (see Section~\ref{sec:expers}). Our results are produced using the original public implementation together with the parameters reported in the paper and, in some cases, repeated communications with the authors. We believe that this practice, as opposed to simply copying the results reported in the papers, can be of great value to the community.

\emph{Open set metrics:} The usual metrics adopted to evaluate OSDA are the average class accuracy over the known classes \emph{OS$^*$}, and the accuracy of the unknown class \emph{UNK}. They are generally combined in \emph{OS}$=\frac{|\mathcal{C}_s |}{|\mathcal{C}_s | + 1} \times$\emph{OS$^*$}$+ \frac{1}{|\mathcal{C}_s | + 1} \times$\emph{UNK} as a measure of the overall performance.
However, we argue (and we already demonstrated in \cite{LOGHMANI2020198}) that treating the unknown as an additional class does not provide an appropriate metric.
As an example, let us consider an algorithm that is not designed to deal with unknown classes (\emph{UNK}=$0.0\%$) but has perfect accuracy over $10$ known classes (\emph{OS$^*$}=$100.0\%$). Although this algorithm is not suitable for open set scenarios because it completely disregards false positives,
it presents a high score of \emph{OS}=$90.9\%$. With increasing number of known classes, this effect on OS becomes even more acute, making the role of \emph{UNK} negligible. For this reason, we propose a new metric defined as the harmonic mean of \emph{OS$^*$} and \emph{UNK}, \emph{HOS}~$= 2 \frac{\text{\emph{OS}$^*$} \times \text{\emph{UNK}}}{\text{\emph{OS$^*$}} + \text{\emph{UNK}}}$. Differently from \emph{OS}, \emph{HOS} provides a high score only if the algorithm performs well both on known and on unknown samples, independently of $|\mathcal{C}_s |$. Using a harmonic mean instead of a simple average penalizes large gaps between \emph{OS$^*$} and \emph{UNK}.

\section{Experiments}
\label{sec:expers}
\subsection{Setup: Baselines, Datasets}
We validate ROS with a thorough experimental analysis on two widely used benchmark datasets, Office-31 and Office-Home.
\emph{Office-31}~\cite{saenko2010adapting} consists of three domains, Webcam (W), Amazon (A) and Dslr (D), each containing $31$ object categories. We follow the setting proposed in~\cite{saito2018open}, where the first $10$ classes in alphabetic order are considered known and the last $11$ classes are considered unknown.
\emph{Office-Home}~\cite{venkateswara2017deep} consists of four domains, Product (Pr), Art (Ar), Real World (Rw) and  Clipart (Cl), each containing $65$ object categories.
Unless otherwise specified, we follow the setting proposed in~\cite{liu2019separate}, where the first $25$ classes in alphabetic order are considered known classes and the remaining $40$ classes are considered unknown. Both the number of categories and the large domain gaps make this dataset much more challenging than Office-31.

We compare ROS against the state-of-the-art methods STA~\cite{liu2019separate}, OSBP~\cite{saito2018open}, UAN~\cite{you2019universal}, AoD~\cite{feng2019attract},  that we already described in Section~\ref{sec:related}. For each of them, we run experiments using the official code provided by the authors, with the exact parameters declared in the relative paper. The only exception was made for AoD for which the authors have not released the code at the time of writing, thus we report the values presented in their original work.
We also highlight that STA presents a practical issue related to the similarity score used to separate known and unknown categories. Its formulation is based on the \emph{max} operator according to the Equation (2) in \cite{liu2019separate},
but appears instead based on \emph{sum} in the implementation code. In our analysis we considered both the two variants (STA\textsubscript{sum}, STA\textsubscript{max}) for the sake of completeness. All the results presented in this section, both for ROS and for the baseline methods, are the average over three independent experimental runs. We do not cherry pick the best out of several trials, but only run the three experiments we report.

\subsection{Implementation Details}
By following standard practice, we evaluate the performances of ROS on Office-31 using two different backbones {ResNet-50} \cite{he2016deep} and {VGGNet} \cite{simonyan2014very}, both pre-trained on ImageNet \cite{deng2009imagenet},  and we focus on ResNet-50 for Office-Home. The hyper-parameters values are the same regardless of the backbone and the dataset used. In particular, in both Stage I and Stage II of ROS the batch size is set to $32$ with a learning rate of $0.0003$ which decreases during the training following an inverse decay scheduling. For all layers trained from scratch, we set the learning rate $10$ times higher than the pre-trained ones.  We use SGD, setting the weight decay as $0.0005$ and momentum as $0.9$. In both stages, the weight of the self-supervised task is set three times the one of the semantic classification task, thus $\lambda_{1,1}= \lambda_{2,2}=3$. In Stage I, the weight of the center loss is  $\lambda_{1,2}=0.1$ and in Stage II the weight of the entropy loss is  $\lambda_{2,1}=0.1$. The network trained in Stage I is used as starting point for Stage II.
To take into consideration the extra category, in Stage II we set the learning rate of the new unknown class to twice that of the known classes (already learned in Stage I).
More implementation details and a sensitivity analysis of the hyper-parameters are provided in Appendix A and D.

\subsection{Results}
\label{subsec:results}

\paragraph{How does our method compare to the state-of-the-art?}
Table~\ref{tab:office31Resnet50} and \ref{tab:officehomeResnet50} show the average results over three runs on each of the domain shifts, respectively of Office-31 and Office-Home.
To discuss the results, we focus on the HOS metric since it is a synthesis of OS* and UNK, as discussed in Section \ref{sec:metrics}. Overall, ROS
outperforms the state-of-the-art on a total of $13$ out of $18$ domain shifts and presents the highest average performance on both Office-31 and Office-Home.
The HOS improvement gets up to $2.2\%$ compared to the second best method OSBP. Specifically, ROS has a large gain over STA, regardless of its specific max or sum implementation, while UAN is not a challenging competitor due to its low performance on the unknown class. We can compare against AoD only when using VGG for Office-31: we report the  original results in gray in Table \ref{tab:officehomeResnet50}, with the HOS value confirming our advantage.

A more in-depth analysis indicates that the advantage of ROS is largely related in its ability in separating known and unknown samples.
Indeed, while our average OS* is similar to that of the competing methods, our average UNK is significantly higher. This characteristic is also visible qualitatively by looking at the t-SNE visualizations in Figure~\ref{fig:tsne} where we focus on the comparison against the second best method OSBP. Here the features for the known (red) and unknown (blue) target data appear more confused than for ROS.

\begin{table}[t!]
\centering
\caption{Accuracy (\%) averaged over three runs of each method on Office-31 dataset using ResNet-50 and VGGNet as backbones}
\resizebox{\textwidth}{!}{
\begin{tabular}{l@{~~~~~}cc ccc ccc ccc ccc ccc ccc ccc}
\hline

       &       \multicolumn{21}{c}{\textbf{Office-31}}        \\

\hline

&       \multicolumn{21}{c}{\textbf{ResNet-50}}        \\
\hline
 & &\multicolumn{3}{c|}{A $\rightarrow$ W } & \multicolumn{3}{c|}{A $\rightarrow$ D } & \multicolumn{3}{c|}{D $\rightarrow$ W }  & \multicolumn{3}{c|}{W $\rightarrow$ D} & \multicolumn{3}{c|}{D $\rightarrow$ A } & \multicolumn{3}{c|}{W $\rightarrow$ A } & \multicolumn{3}{c}{\textbf{Avg.} }
 \\
        & & OS* & UNK & \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}   & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c}{\textbf{\underline{HOS}}} \\
\hline
\multicolumn{1}{c}{STA\textsubscript{sum}}& \multirow{2}{*}{\cite{liu2019separate}}   & 92.1 & 58.0 & \multicolumn{1}{c|}{71.0}  & 95.4 & 45.5 & \multicolumn{1}{c|}{61.6}  & 97.1 & 49.7 & \multicolumn{1}{c|}{65.5}  & 96.6 & 48.5 & \multicolumn{1}{c|}{64.4} &  94.1 & 55.0 & \multicolumn{1}{c|}{69.4}  & 92.1 & 46.2 & \multicolumn{1}{c|}{60.9}  & 94.6 & 50.5 & \multicolumn{1}{c}{65.5$\pm$0.3}  \\

\multicolumn{1}{c}{STA\textsubscript{max}}&  & 86.7 & 67.6 & \multicolumn{1}{c|}{75.9}  & 91.0 & 63.9 & \multicolumn{1}{c|}{75.0}  & 94.1 & 55.5 & \multicolumn{1}{c|}{69.8}  & 84.9 & 67.8 & \multicolumn{1}{c|}{75.2}  & 83.1 & 65.9 & \multicolumn{1}{c|}{73.2} & 66.2 & 68.0 & \multicolumn{1}{c|}{66.1} & 84.3 & 64.8 & \multicolumn{1}{c}{72.5$\pm$0.8}\\

\multicolumn{1}{c}{OSBP}  \cite{saito2018open} & & 86.8 & 79.2 & \multicolumn{1}{c|}{\textbf{82.7}}  & 90.5 & 75.5 & \multicolumn{1}{c|}{\textbf{82.4}}  & 97.7 & 96.7 & \multicolumn{1}{c|}{\textbf{97.2}} & 99.1 & 84.2 & \multicolumn{1}{c|}{91.1} & 76.1 & 72.3 & \multicolumn{1}{c|}{75.1} &  73.0 & 74.4 & \multicolumn{1}{c|}{73.7}  & 87.2 & 80.4 & \multicolumn{1}{c}{83.7$\pm$0.4} \\

\multicolumn{1}{c}{UAN}   \cite{you2019universal}&   & 95.5 & 31.0 &  \multicolumn{1}{c|}{46.8}   & 95.6  & 24.4 & \multicolumn{1}{c|}{38.9}   & 99.8 & 52.5 & \multicolumn{1}{c|}{68.8}   & 81.5 & 41.4 & \multicolumn{1}{c|}{53.0}   & 93.5 & 53.4 & \multicolumn{1}{c|}{68.0}   & 94.1 & 38.8 & \multicolumn{1}{c|}{54.9}   & 93.4 & 40.3 & \multicolumn{1}{c}{55.1$\pm$1.4} \\

\multicolumn{1}{c}{\textbf{ROS}} &   & 88.4 & 76.7 & \multicolumn{1}{c|}{82.1}  & 87.5 & 77.8 & \multicolumn{1}{c|}{\textbf{82.4}}  & 99.3 & 93.0 & \multicolumn{1}{c|}{96.0}  & 100.0 & 99.4 & \multicolumn{1}{c|}{\textbf{99.7}}  & 74.8 & 81.2 & \multicolumn{1}{c|}{\textbf{77.9}}  & 69.7 & 86.6 & \multicolumn{1}{c|}{\textbf{77.2}}  & 86.6 & 85.8 & \multicolumn{1}{c}{\textbf{85.9$\pm$0.2}}   \\

\hline\hline

&       \multicolumn{21}{c}{\textbf{VGGNet}}        \\
\hline

&  &\multicolumn{3}{c|}{A $\rightarrow$ W } & \multicolumn{3}{c|}{A $\rightarrow$ D } & \multicolumn{3}{c|}{D $\rightarrow$ W }  & \multicolumn{3}{c|}{W $\rightarrow$ D} & \multicolumn{3}{c|}{D $\rightarrow$ A } & \multicolumn{3}{c|}{W $\rightarrow$ A } & \multicolumn{3}{c}{\textbf{Avg.} }
 \\
        & & OS* & UNK & \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c}{\textbf{\underline{HOS}}} \\
\hline
\multicolumn{1}{c}{OSBP}  \cite{saito2018open}& & 79.4 & 75.8 & \multicolumn{1}{c|}{77.5}  & 87.9 & 75.2 & \multicolumn{1}{c|}{81.0}  & 96.8 & 93.4 & \multicolumn{1}{c|}{\textbf{95.0}}  & 98.9 & 84.2 & \multicolumn{1}{c|}{91.0} & 74.4 & 82.4 & \multicolumn{1}{c|}{\textbf{78.2}} & 69.7 & 76.4 & \multicolumn{1}{c|}{72.9}  & 84.5 & 81.2 & \multicolumn{1}{c}{82.6$\pm$0.8}   \\

\multicolumn{1}{c}{\textbf{ROS}} &  & 80.3 & 81.7 & \multicolumn{1}{c|}{\textbf{81.0}}  & 81.8 & 76.5 & \multicolumn{1}{c|}{79.0}  & 99.5 & 89.9 & \multicolumn{1}{c|}{94.4}  & 99.3 & 100.0 & \multicolumn{1}{c|}{\textbf{99.7}}  & 76.7  & 79.6 & \multicolumn{1}{c|}{78.1}  & 62.2 & 91.6 & \multicolumn{1}{c|}{\textbf{74.1}} & 83.3 & 86.5 & \multicolumn{1}{c}{\textbf{84.4$\pm$0.2}}    \\

\hline

\multicolumn{1}{c}{\textcolor{gray}{AoD \cite{feng2019attract}}}& & \textcolor{gray}{87.7} & \textcolor{gray}{73.4} & \multicolumn{1}{c|}{\textcolor{gray}{79.9}} &  \textcolor{gray}{92.0} & \textcolor{gray}{71.1} & \multicolumn{1}{c|}{\textbf{\textcolor{gray}{79.3}}}  & \textcolor{gray}{99.8} & \textcolor{gray}{78.9} & \multicolumn{1}{c|}{\textcolor{gray}{88.1}} & \textcolor{gray}{99.3} & \textcolor{gray}{87.2} & \multicolumn{1}{c|}{\textcolor{gray}{92.9}} &  \textcolor{gray}{88.4} & \textcolor{gray}{13.6} & \multicolumn{1}{c|}{\textcolor{gray}{23.6}}  & \textcolor{gray}{82.6} & \textcolor{gray}{57.3} & \multicolumn{1}{c|}{\textcolor{gray}{67.7}}   & \textcolor{gray}{91.6} & \textcolor{gray}{63.6} & \multicolumn{1}{c}{\textcolor{gray}{71.9}}  \\

\hline

\end{tabular}
}
    \label{tab:office31Resnet50}
\end{table}
\begin{table}[t]
\centering
\caption{Accuracy (\%) averaged over three runs of each method on Office-Home dataset using ResNet-50 as backbone}
\resizebox{\textwidth}{!}{
\begin{tabular}{l@{~~~~~~~}c ccc ccc ccc ccc ccc ccc ccc}
\hline

       &       \multicolumn{18}{c}{~~~~~~~~~~~\textbf{Office-Home}}        \\

\hline
 & & & \multicolumn{3}{c|}{Pr $\rightarrow$ Rw } & \multicolumn{3}{c|}{Pr $\rightarrow$ Cl } & \multicolumn{3}{c|}{Pr $\rightarrow$ Ar }  &

 \multicolumn{3}{c|}{Ar $\rightarrow$ Pr } & \multicolumn{3}{c|}{Ar $\rightarrow$ Rw } & \multicolumn{3}{c}{Ar $\rightarrow$ Cl }  \\
       & &  & OS* & UNK & \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c}{\textbf{\underline{HOS}}} \\
\hline

\multicolumn{1}{c}{STA\textsubscript{sum}} & \multirow{2}{*}{\cite{liu2019separate}}  & &  78.1 & 63.3 & \multicolumn{1}{c|}{69.7}  & 44.7 & 71.5 & \multicolumn{1}{c|}{55.0} & 55.4 & 73.7 & \multicolumn{1}{c|}{63.1} & 68.7 & 59.7 & \multicolumn{1}{c|}{63.7}  & 81.1 & 50.5 & \multicolumn{1}{c|}{62.1}  & 50.8 & 63.4 & \multicolumn{1}{c}{56.3}  \\
\multicolumn{1}{c}{STA\textsubscript{max}}& &  & 76.2 & 64.3 & \multicolumn{1}{c|}{69.5}  & 44.2 & 67.1 & \multicolumn{1}{c|}{53.2}  & 54.2 & 72.4 & \multicolumn{1}{c|}{61.9}  & 68.0 & 48.4 & \multicolumn{1}{c|}{54.0}   & 78.6 & 60.4 & \multicolumn{1}{c|}{68.3}  & 46.0 & 72.3 & \multicolumn{1}{c}{55.8}\\
\multicolumn{1}{c}{OSBP} \cite{saito2018open}& &   & 76.2 & 71.7 & \multicolumn{1}{c|}{73.9}  & 44.5 & 66.3 & \multicolumn{1}{c|}{53.2}  & 59.1 & 68.1 & \multicolumn{1}{c|}{\textbf{63.2}}  & 71.8 & 59.8 & \multicolumn{1}{c|}{65.2}  & 79.3 & 67.5 & \multicolumn{1}{c|}{72.9}   & 50.2 & 61.1 & \multicolumn{1}{c}{55.1}\\
\multicolumn{1}{c}{UAN} \cite{you2019universal}&  &  & 84.0 & 0.1 & \multicolumn{1}{c|}{0.2}  & 59.1 & 0.0 & \multicolumn{1}{c|}{0.0}  & 73.7 & 0.0 & \multicolumn{1}{c|}{0.0}  & 81.1 & 0.0 & \multicolumn{1}{c|}{0.0}  & 88.2 & 0.1 & \multicolumn{1}{c|}{0.2}  & 62.4 & 0.0 & \multicolumn{1}{c}{0.0} \\
\multicolumn{1}{c}{\textbf{ROS}} & & & 70.8 & 78.4 &  \multicolumn{1}{c|}{\textbf{74.4}} & 46.5 & 71.2 & \multicolumn{1}{c|}{\textbf{56.3}}  & 57.3 & 64.3 & \multicolumn{1}{c|}{60.6}  & 68.4 & 70.3 & \multicolumn{1}{c|}{\textbf{69.3}} & 75.8 & 77.2 & \multicolumn{1}{c|}{\textbf{76.5}} & 50.6 & 74.1 & \multicolumn{1}{c}{\textbf{60.1}} \\
\hline\hline
 & \multicolumn{3}{c|}{Rw $\rightarrow$ Ar } & \multicolumn{3}{c|}{Rw $\rightarrow$ Pr } & \multicolumn{3}{c|}{Rw $\rightarrow$ Cl }

  & \multicolumn{3}{c|}{Cl $\rightarrow$ Rw } & \multicolumn{3}{c|}{Cl $\rightarrow$ Ar } & \multicolumn{3}{c}{Cl $\rightarrow$ Pr }  & \multicolumn{3}{c}{\textbf{Avg.}} \\

          & OS* & UNK & \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} &  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}&  OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}}  & OS* & UNK &  \multicolumn{1}{c|}{\textbf{\underline{HOS}}} & OS* & UNK &  \multicolumn{1}{c}{\textbf{\underline{HOS}}} \\

\hline
\multicolumn{1}{c}{STA\textsubscript{sum}}  & 67.9 & 62.3 & \multicolumn{1}{c|}{65.0}  & 77.9 & 58.0 & \multicolumn{1}{c|}{66.4}  & 51.4 & 57.9 & \multicolumn{1}{c|}{54.2}  & 69.8 & 63.2 & \multicolumn{1}{c|}{66.3}   & 53.0 & 63.9 & \multicolumn{1}{c|}{57.9}  & 61.4 & 63.5 & \multicolumn{1}{c|}{62.5}  & 63.4 & 62.6 & \multicolumn{1}{c}{61.9$\pm$2.1}  \\
\multicolumn{1}{c}{STA\textsubscript{max}} & 67.5 & 66.7 & \multicolumn{1}{c|}{67.1}  & 77.1 & 55.4 & \multicolumn{1}{c|}{64.5} & 49.9 & 61.1 & \multicolumn{1}{c|}{54.5} & 67.0 & 66.7 & \multicolumn{1}{c|}{66.8}   & 51.4 & 65.0 & \multicolumn{1}{c|}{57.4}   & 61.8 & 59.1 & \multicolumn{1}{c|}{60.4}  & 61.8 & 63.3 & \multicolumn{1}{c}{61.1$\pm$0.3} \\
\multicolumn{1}{c}{OSBP}  & 66.1 & 67.3 & \multicolumn{1}{c|}{66.7} &  76.3 & 68.6 & \multicolumn{1}{c|}{72.3}  & 48.0 & 63.0 & \multicolumn{1}{c|}{54.5}  & 72.0 & 69.2 & \multicolumn{1}{c|}{\textbf{70.6}}   & 59.4 & 70.3 & \multicolumn{1}{c|}{\textbf{64.3}}   & 67.0 & 62.7 & \multicolumn{1}{c|}{64.7}    & 64.1 & 66.3 & \multicolumn{1}{c}{64.7$\pm$0.2} \\
\multicolumn{1}{c}{UAN}   & 77.5 & 0.1 & \multicolumn{1}{c|}{0.2}  & 85.0 & 0.1 & \multicolumn{1}{c|}{0.1}  & 66.2 & 0.0 & \multicolumn{1}{c|}{0.0}  & 80.6 & 0.1 & \multicolumn{1}{c|}{0.2}  & 70.5 & 0.0 & \multicolumn{1}{c|}{0.0}  & 74.0 & 0.1 & \multicolumn{1}{c|}{0.2}   & 75.2 & 0.0 & \multicolumn{1}{c}{0.1$\pm$0.0} \\
\multicolumn{1}{c}{\textbf{ROS}}  & 67.0 & 70.8 & \multicolumn{1}{c|}{\textbf{68.8}} & 72.0 & 80.0 & \multicolumn{1}{c|}{\textbf{75.7}}  & 51.5 & 73.0 & \multicolumn{1}{c|}{\textbf{60.4}}  & 65.3 & 72.2 & \multicolumn{1}{c|}{68.6}  & 53.6 & 65.5 & \multicolumn{1}{c|}{58.9} & 59.8 & 71.6 & \multicolumn{1}{c|}{\textbf{65.2}} & 61.6 & 72.4 & \multicolumn{1}{c}{\textbf{66.2$\pm$ 0.3}}\\

\hline

\end{tabular}
}
    \label{tab:officehomeResnet50}
\end{table}
\begin{figure}
\centering
\includegraphics[width=0.85\linewidth]{img/tsne.pdf}
\caption{t-SNE visualization of the target features for the
W$\rightarrow${A} domain shift from Office-31. Red and blue points are respectively features of known and unknown classes}
\label{fig:tsne}
\end{figure}

\paragraph{Is it possible to reproduce the reported results of the state-of-the-art?}
By analyzing the published OSDA papers, we noticed some incoherence in the reported results. For example, some of the results from OSBP are different between the pre-print~\cite{saito2018open-arxiv} and the published~\cite{saito2018open} version, although they present the same description for method and hyper-parameters. Also, AoD~\cite{feng2019attract} compares against the pre-print results of OSBP, while omitting the results of STA. To dissipate these ambiguities and gain a better perspective on the current state-of-the-art methods, in Table~\ref{tab:reproducibility} we compare the results on Office-31 reported in previous works with the results obtained by running their code. For this analysis we focus on OS since it is the only metric reported for some of the methods. The comparison shows that, despite using the original implementation and the information provided by the authors, the OS obtained by re-running the experiments is between $1.3\%$ and $4.9\%$ lower than the originally published results. The significance of this gap calls for greater attention in providing all the relevant information for reproducing the experimental results. A more extensive reproducibility study is provided in Appendix B.
\begin{table}[t]
\centering
\caption{Reported vs reproduced OS accuracy (\%) averaged over three runs}
\resizebox{0.8\textwidth}{!}{
\begin{tabular}{l@{~~~} cccccc ccc}
\hline
 \multicolumn{9}{c}{\textbf{Reproducibility Study}} \\
 \hline
 \multicolumn{6}{c|}{Office-31 (ResNet-50)} & \multicolumn{3}{c}{Office-31 (VGGNet)}\\
 \hline
  \multicolumn{3}{c|}{STA\textsubscript{sum}} & \multicolumn{3}{c|}{UAN}  & \multicolumn{3}{c}{OSBP}\\
OS\textsubscript{reported} & OS\textsubscript{ours} & \multicolumn{1}{c|}{gap} & OS\textsubscript{reported} & OS\textsubscript{ours}  & \multicolumn{1}{c|}{gap} & OS\textsubscript{reported} & OS\textsubscript{ours}  & \multicolumn{1}{c}{gap} \\
 \hline
 \multicolumn{1}{c}{92.9} &  \multicolumn{1}{c}{90.6$\pm$1.8} & \multicolumn{1}{c|}{\textbf{2.3}} &  \multicolumn{1}{c}{89.2} &   \multicolumn{1}{c}{87.9$\pm$0.03} & \multicolumn{1}{c|}{\textbf{1.3}} &  \multicolumn{1}{c}{89.1} &  \multicolumn{1}{c}{84.2 $\pm$0.4} & \multicolumn{1}{c}{\textbf{4.9}}\\
\hline
\end{tabular}
}
    \label{tab:reproducibility}
\end{table}

\paragraph{Why is it important to use the HOS metric?}
The most glaring example of why OS is not an appropriate metric for OSDA is provided by the results of UAN. In fact, when computing OS from the average (OS*,UNK) in Table~\ref{tab:office31Resnet50} and~\ref{tab:officehomeResnet50}, we can see that UAN has OS=$72.5\%$ for Office-Home and OS=$91.4\%$ for Office-31. This is mostly reflective of the ability of UAN in recognizing the known classes (OS*), but it completely disregards its (in)ability to identify the unknown samples (UNK). For example, for most domain shifts in Office-Home, UAN does not assign (almost) any samples to the unknown class, resulting in UNK=$0.0\%$. On the other hand, HOS better reflects the open set scenario and assumes a high value only when OS* and UNK are both high.

\paragraph{Is rotation recognition effective for known/unknown separation in OSDA?}
To better understand the effectiveness of rotation recognition for known/unknown separation, we measure the performance of our Stage I and compare it to the Stage I of STA. Indeed, also STA has a similar two-stage structure, but uses a multi-binary classifier instead of a multi-rotation classifier to separate known and unknown target samples. To assess the performance, we compute the \textit{area under receiver operating characteristic curve} (AUC-ROC) over the normality scores $\mathcal{N}$ on Office-31. Table~\ref{tab:ablationstep1} shows that the AUC-ROC of ROS ($91.5$) is significantly higher than that of the multi-binary used by STA ($79.9$). Table~\ref{tab:ablationstep1} also shows the performance of Stage I when alternatively removing the center loss (No Center Loss) from Equation (\ref{eq:centerloss}) ($\lambda_{1,2}=0$) and the anchor image (No Anchor) when training $R_1$, thus passing from relative rotation to the more standard absolute rotation. In both cases, the performance significantly drops compared to our full method, but still outperforms the multi-binary classifier of STA.

\begin{table}[t!]
\centering
\caption{Ablation Analysis on Stage I and Stage II}
\resizebox{0.95\textwidth}{!}{
\begin{tabular}{l@{~} c c c c c c c}

\hline
\multicolumn{8}{c}{\textbf{Ablation Study}}\\
\hline

\multicolumn{1}{c}{\multirow{1}{*}{\textbf{STAGE I} (AUC-ROC)}} & \multicolumn{1}{c|}{A $\rightarrow$ W }
 & \multicolumn{1}{c|}{A $\rightarrow$ D }
 & \multicolumn{1}{c|}{D $\rightarrow$ W }
 & \multicolumn{1}{c|}{W $\rightarrow$ D}
 & \multicolumn{1}{c|}{D $\rightarrow$ A }
 & \multicolumn{1}{c|}{W $\rightarrow$ A }
 & \textbf{Avg.} \\

\hline

\textbf{ROS} & \multicolumn{1}{c|}{90.1} & \multicolumn{1}{c|}{88.1} & \multicolumn{1}{c|}{99.4} & \multicolumn{1}{c|}{99.9} & \multicolumn{1}{c|}{87.5} & \multicolumn{1}{c|}{83.8} &\textbf{91.5}\\
Multi-Binary (from STA \cite{liu2019separate}) & \multicolumn{1}{c|}{83.2} & \multicolumn{1}{c|}{84.1} & \multicolumn{1}{c|}{86.8} & \multicolumn{1}{c|}{72.0} & \multicolumn{1}{c|}{75.7} & \multicolumn{1}{c|}{78.3} & 79.9\\
ROS - No Center loss & \multicolumn{1}{c|}{88.8} & \multicolumn{1}{c|}{83.2} & \multicolumn{1}{c|}{98.8} & \multicolumn{1}{c|}{99.8} & \multicolumn{1}{c|}{84.7} & \multicolumn{1}{c|}{84.5} & 89.9\\
ROS - No Anchor & \multicolumn{1}{c|}{84.5} & \multicolumn{1}{c|}{84.9} & \multicolumn{1}{c|}{99.1} & \multicolumn{1}{c|}{99.9} & \multicolumn{1}{c|}{87.6} & \multicolumn{1}{c|}{86.2} & 90.4\\
ROS - No Rot. Score & \multicolumn{1}{c|}{86.3} & \multicolumn{1}{c|}{82.7} & \multicolumn{1}{c|}{99.5} & \multicolumn{1}{c|}{99.9} & \multicolumn{1}{c|}{86.3} & \multicolumn{1}{c|}{82.9} & 89.6\\
ROS - No Ent. Score & \multicolumn{1}{c|}{80.7} & \multicolumn{1}{c|}{78.7} & \multicolumn{1}{c|}{99.7} & \multicolumn{1}{c|}{99.9} & \multicolumn{1}{c|}{86.6} & \multicolumn{1}{c|}{84.4} & 88.3\\
ROS - No Center loss, No Anchor & \multicolumn{1}{c|}{76.5} & \multicolumn{1}{c|}{79.1} & \multicolumn{1}{c|}{98.3} & \multicolumn{1}{c|}{99.7} & \multicolumn{1}{c|}{85.2} & \multicolumn{1}{c|}{83.5} & 87.1\\
ROS - No Rot. Score, No Anchor & \multicolumn{1}{c|}{83.9} & \multicolumn{1}{c|}{84.6} & \multicolumn{1}{c|}{99.4} & \multicolumn{1}{c|}{99.9} & \multicolumn{1}{c|}{84.7} & \multicolumn{1}{c|}{84.9} & 89.6\\
ROS - No Ent. Score, No Anchor & \multicolumn{1}{c|}{80.1} & \multicolumn{1}{c|}{81.0} & \multicolumn{1}{c|}{99.5} & \multicolumn{1}{c|}{99.7} & \multicolumn{1}{c|}{84.3} & \multicolumn{1}{c|}{83.3} & 87.9\\
ROS - No Rot. Score, No Center loss & \multicolumn{1}{c|}{80.9} & \multicolumn{1}{c|}{81.6} & \multicolumn{1}{c|}{98.9} & \multicolumn{1}{c|}{99.8} & \multicolumn{1}{c|}{85.6} & \multicolumn{1}{c|}{83.3} & 88.3\\
ROS - No Ent. Score, No Center loss & \multicolumn{1}{c|}{76.4} & \multicolumn{1}{c|}{79.8} & \multicolumn{1}{c|}{99.0} & \multicolumn{1}{c|}{98.3} & \multicolumn{1}{c|}{84.4} & \multicolumn{1}{c|}{84.3} & 87.0\\
ROS - No Ent. Score, No Center loss, No Anchor & \multicolumn{1}{c|}{78.6} & \multicolumn{1}{c|}{80.4} & \multicolumn{1}{c|}{99.0} & \multicolumn{1}{c|}{98.9} & \multicolumn{1}{c|}{86.2} & \multicolumn{1}{c|}{83.2} & 87.7\\
ROS - No Rot. Score, No Center loss, No Anchor & \multicolumn{1}{c|}{78.7} & \multicolumn{1}{c|}{82.2} & \multicolumn{1}{c|}{98.3} & \multicolumn{1}{c|}{99.8} & \multicolumn{1}{c|}{85.0} & \multicolumn{1}{c|}{82.6} & 87.8\\

\hline\hline

\multicolumn{1}{c}{\multirow{1}{*}{\textbf{STAGE II} (HOS)}}  & \multicolumn{1}{c|}{A $\rightarrow$ W } & \multicolumn{1}{c|}{A $\rightarrow$ D } & \multicolumn{1}{c|}{D $\rightarrow$ W }  & \multicolumn{1}{c|}{W $\rightarrow$ D} & \multicolumn{1}{c|}{D $\rightarrow$ A } & \multicolumn{1}{c|}{W $\rightarrow$ A } & \multicolumn{1}{c}{\textbf{Avg.}}
 \\
\hline
\textbf{ROS}  & \multicolumn{1}{c|}{82.1}  & \multicolumn{1}{c|}{82.4}  & \multicolumn{1}{c|}{96.0}  & \multicolumn{1}{c|}{99.7}   & \multicolumn{1}{c|}{77.9}  & \multicolumn{1}{c|}{77.2}  & \textbf{85.9}   \\

ROS Stage I - GRL \cite{ganin2016domain} Stage II  & \multicolumn{1}{c|}{83.5}  & \multicolumn{1}{c|}{80.9}   & \multicolumn{1}{c|}{97.1}  & \multicolumn{1}{c|}{99.4} & \multicolumn{1}{c|}{77.3} & \multicolumn{1}{c|}{72.6} &  85.1\\

ROS Stage I - No Anchor in Stage II  & \multicolumn{1}{c|}{80.0}   & \multicolumn{1}{c|}{82.3}  & \multicolumn{1}{c|}{94.5}   & \multicolumn{1}{c|}{99.2}   & \multicolumn{1}{c|}{76.9}  & \multicolumn{1}{c|}{76.6}  & 84.9\\

ROS Stage I - No Anchor, No Entropy in Stage II  & \multicolumn{1}{c|}{80.1}   & \multicolumn{1}{c|}{84.4}  & \multicolumn{1}{c|}{97.0}   & \multicolumn{1}{c|}{99.2}   & \multicolumn{1}{c|}{76.5}  & \multicolumn{1}{c|}{72.9}  & 85.0\\

\hline
\end{tabular}
}
    \label{tab:ablationstep1}
\end{table}

\paragraph{Why is the normality score defined the way it is?}
As defined in Equation (\ref{eq:normalityscore}), our normality score is a function of the rotation score and entropy score. The rotation score is based on the ability of $R_1$ to predict the rotation of the target samples, while the entropy score is based on the confidence of such predictions. Table~\ref{tab:ablationstep1} shows the results of Stage I when alternatively discarding either the rotation score (No Rot. Score) or the information of the entropy score (No Ent. Score). In both cases the AUC-ROC significantly decreases  compared to the full version, justifying our choice.

\paragraph{Is rotation recognition effective for domain alignment in OSDA?}
While rotation classification has already been used for CSDA~\cite{xu2019self-supervised}, its application in OSDA, where the shared target distribution could be noisy (\emph{i.e}\onedot} \def\Ie{\emph{I.e}\onedot contain unknown samples) has not been studied. On the other hand, GRL~\cite{ganin2016domain} is used, under different forms, by all existing OSDA methods. We compare rotation recognition and GRL in this context by evaluating the performance of our Stage II when replacing the $R_2$ with a domain discriminator. Table~\ref{tab:ablationstep1} shows that rotation recognition performs on par with GRL, if not slightly better. Moreover we also evaluate the role of the relative rotation in the Stage II: the results in the last row of Table~\ref{tab:ablationstep1} confirm that it improves over the standard absolute rotation (No Anchor in Stage II) even when the rotation classifier is used as cross-domain adaptation strategy. Finally, the cosine distance between the source and the target domain without adaptation in Stage II ($0.188$) and with our full method ($0.109$) confirms that rotation recognition is indeed helpful to reduce the domain gap.
\begin{figure}[t!]
\centering
\includegraphics[width=0.28\linewidth]{img/os.pdf}
\includegraphics[width=0.28\linewidth]{img/UNK.pdf}
\includegraphics[width=0.28\linewidth]{img/HOS.pdf}
\caption{Accuracy (\%) averaged over the three openness configurations.}
\label{fig:decreaseknownclass}
\end{figure}
\paragraph{Is our method effective on problems with a high degree of openness?}
The standard open set setting adopted in so far, presents a relatively balanced number of shared and private target classes with openness close to $0.5$. Specifically it is  $\mathbb{O}=1-\frac{10}{21}=0.52$ for Office-31 and $\mathbb{O}=1-\frac{25}{65}=0.62$ for Office-Home. In real-world problems, we can expect the number of unknown target classes to largely exceed the number of known classes, with openness approaching $1$. We investigate this setting using Office-Home and, starting from the classes sorted with ID from 0 to 64 in alphabetic order, we define the following settings with increasing openness:
\textbf{25} known classes $\mathbb{O}=0.62$, ID:\{0-24, 25-49, 40-64\},
\textbf{10} known classes $\mathbb{O}=0.85$, ID:\{0-9, 10-19, 20-29\},
\textbf{5} known classes $\mathbb{O}=0.92$, ID:\{0-4, 5-9, 10-14\}.
Figure~\ref{fig:decreaseknownclass} shows that the performance of our best competitors, STA and OSBP, deteriorates with larger $\mathbb{O}$ due to their inability to recognize the unknown samples. On the other hand, ROS maintains a consistent performance.

\section{Discussion and conclusions}
\label{sec:conclusions}
In this paper, we present ROS: a novel method that tackles OSDA by using the self-supervised task of predicting image rotation. We show that, with simple variations of the rotation prediction task, we can first separate the target samples into known and unknown, and then align the target samples predicted as known with the source samples. Additionally, we propose HOS: a new OSDA metric defined as the harmonic mean between the accuracy of recognizing the known classes and rejecting the unknown samples. HOS overcomes the drawbacks of the current metric OS where the contribution of the unknown classes vanishes with increasing number of known classes.

We evaluate the perfomance of ROS
on the standard Office-31 and Office-Home benchmarks, showing that it
outperforms the competing methods.
In addition, when tested on
settings with increasing openness, ROS is the only method that maintains a steady performance. HOS reveals to be crucial in this evaluation to correctly assess the performance of the methods on both known and unknown samples.
Finally, the failure in reproducing the reported results of existing methods exposes an important issue in OSDA that echoes the current reproducibility crisis in machine learning. We hope that our contributions can help laying a more solid foundation for the field.\\

\noindent\textbf{Acknowledgements} This work was partially founded by the ERC grant 637076 RoboExNovo (SB), by the H2020 ACROSSING project grant 676157 (MRL) and took advantage of the NVIDIA GPU Academic Hardware Grant (TT).

\clearpage

\section*{Appendix}


\section{Selected Work on Intonational Meaning}
Introductions to speech research and acoustic phonetics can be found
in \cite{Ladefoged62,Denes73,Fry79}. For survey-level introductions to
studies of English intonation, see
\cite{Halliday67b,Crystal69,Crystal75,Cruttenden86,Couper-Kuhlen86}.
\cite{Ladd80,Bolinger86,Bolinger89} also represent good introductions
to studies of intonational meaning.  Works oriented toward particular
aspects of intonation theory and models of intonation, also primarily
for English are
\cite{Bolinger51,Liberman75,Garding83,Hirst83,Pierrehumbert80,Vaissiere83,Liberman84,Ladd88,tHart90}
\nocite{Ladd83a,Ladd83b,Ladd83c,Ladd86,Pike45,O'Connor61}
\nocite{Buxton83,Edwards88}
An idea of some of the topics and methods of research on intonational
representation and intonational meaning can
be gained from \cite{Cutler83a,Gibbon84}.
\nocite{Duncan77}

The meaning of particular intonational contours has been studied by
\cite{Liberman74,Sag75,Ladd77,Ladd78,Bing79b,Ladd80,Bouton82,Ward85b}.
Attempts to determine the relative contribution of intonational
contours and other intonational features to utterance interpretation
include \cite{Ladd85,Pierrehumbert90,Hirschberg91b}.

For intonational studies of adult intonation in speech directed to
children, see the overview in
\cite{Fernald91b}.  For experimental work on infants' perception of
phrasing see \cite{Hirsh-Pasek85,Ratner86}, and for children's
turn-taking devices, see
\cite{Ervin-Tripp79}.  \cite{Morgan87} studies prosodic and
morphological cues to language acquisition.
\nocite{Fernald91a}

There is a considerable literature on the relationship between
intonational phrasing and syntactic phenomena.  General and
theoretical work includes \cite{Downing70,Bresnan71,Selkirk84a,Cooper80}.
Empirical work on acoustic correlates of intonation boundaries
includes experimental work (production and perception) such as
\cite{Grosjean79,O'Malley73,Lehiste73,Klatt75a,Lehiste76,Cooper77,Streeter78,Wales79,Lehiste80a,Gee83,Umeda82,Beach91}.
\nocite{Martin70,Grosjean87}
Corpus-based research includes \cite{Quirk64,Altenberg87}.  Work in
computational linguistics proposes parsing strategies including
prosodic components \cite{Marcus85,Marcus90,Steedman90}.
Text-to-speech applications inspire some of the work on intonational
boundary predictions \cite{Gee83,Altenberg87,Bachenko90}.  Recognition
applications have motivated recent work
\cite{Ostendorf90,Wang92}.
Also see \cite{Wilkenfeld81} for research on prosody and orthography.
\nocite{Silva-Corvalan83,Bruce90}
\nocite{Taft84,O'Shaughnessy89}
\nocite{Wang91a}
Also see \nocite{Misono90,Sugito90}

Early debates on intonational prominence (stress or pitch accent) are
summarized in
\cite{Bolinger58,Crystal69,Liberman77,Ladd80,Bolinger86}.  More recent
contributions include \cite{Beckman86c,Pierrehumbert87b}.  Constraints
on sentence (nuclear) stress are discussed in
\cite{Cutler77b,Erteschik83,Schmerling76,Schmerling74,Bardovi83a}.
Despite Bolinger's seminal article on the unpredictability of accent
\cite{Bolinger72a}, attempts to predict accent from other features of
the uttered text include \cite{Altenberg87,Hirschberg90}.  A number of
authors have examined the relationship between accent and various
characterizations of information status: Work on the focal domains of
accent and the representation and interpretation of intonational focus
and presupposition includes
\cite{Lakoff71,Schmerling71,Jackendoff72,Ball77,Wilson79,Enkvist79,Gussenhoven83a,Culicover83,Rooth85,Rochemont90,Rooth91,Horne85,Horne87,Baart87,Dirksen92,Zacharski92}.
Topic/comment, given/new, theme/rheme distinctions are discussed with
respect to accent by
\cite{Schmerling75b,Bardovi83b,GBrown83a,Gundel78,Lehman77,Fuchs80,Chafe76,Nooteboom82,Fuchs84,Terken84,Terken85,Terken87,Fowler87,Horne91a,Horne91b}.
\nocite{Allerton79,Kruyt85}
And contrastive stress is examined by
\cite{Bolinger61,Harries-Delisle78,Couper-Kuhlen84}.
Others have looked at the interpretation of accent with particular
attention to anaphora
\cite{Gleitman61,Akmajian70,Williams80,Lujan85,Hirschberg91a}.
See \cite{Ladd87a,Sproat90} for discussions of the phrasing and
accenting of complex nominals.
\nocite{Cantrall69,Cantrall75}
\nocite{Hirschberg90a}
\nocite{Wilson79}
\nocite{Pierrehumbert89}
\nocite{Liberman77}

For general discussion of the intonational characteristics of longer
discourses,
see \cite{Brazil80,GBrown80,GBrown83b}.  Halliday \cite{Halliday76}
also has insightful comments on intonation and discourse cohesion.
For work on intonation and discourse structure, see
\cite{Yule80,Silverman86,Avesani88,Ayers92,Hirschberg92b,Grosz92}, and see
\cite{Kumpf87} for discussion of
pitch phenomena and stories.  \cite{Hirschberg87a,Litman90} investigate
the intonation of cue phrases.
\cite{Butterworth75,Butterworth77,Schegloff79b} investigate
intonational characteristics of speech-only communication channels.

\nocite{Goodwin81}
\nocite{Kutik83}
\nocite{Rees76}
\nocite{Hirschberg86} \nocite{Gazdar80b}
\nocite{Gumperz77}

A good overview of work in speech synthesis is \cite{Klatt87}.
Current work in the area can be sampled in \cite{Autrans90}.
\nocite{Olive85,Anderson84,JHouse90a}
Some experimental message-to-speech systems are described in
\cite{Young79,LWitten77,Danlos86,Davis88,JHouse90b}.
A good introduction to past and recent work on speech recognition can
be found in \cite{Waibel90a}.  More specialized work on the use of
prosody in recognition includes
\cite{Lea75,Lea79,Pierrehumbert83,Waibel79,Ljolje86,Ljolje87a,Waibel88,DHouse89,Silverman92a}
\nocite{Pierrehumbert81,Hirschberg89}
\nocite{Silverman87}

\nocite{Hemphill90,Price88}
\nocite{Talkin89} \nocite{Nash80} \nocite{Carlson83} \nocite{Cutler83b}
\nocite{Prince84} \nocite{Lehiste80b} \nocite{DHouse90a}
\nocite{Roe91}
\nocite{Cutler77a}
\nocite{Scherer84}
\nocite{Taylor87}
\nocite{Bolinger82}
\nocite{Oakeshott84}
\nocite{Cruz-Ferreira83}
\nocite{Stockwell71}
\nocite{Beckman86a}
\nocite{Beckman86b}
\nocite{Fry58}