\section{Task and Dataset}
\label{sec:task_and_data}

Below, we describe the CONSTRAINT 2022 shared task and the corresponding dataset provided by the task organizers. More detail can be found in the shared task report \cite{sharma2022report}.

\subsection{Task}
\label{ssec:Task}
The CONSTRAINT 2022 shared task asked participating systems to detect the role of the entities in the meme, given the meme and a list of these entities. Figure~\ref{fig:task_example} shows an example of an image with the extracted OCR text, implicit (image showing Salman Khan, who is not mentioned in the text), and explicit entities and their roles. The example illustrates various challenges: (\emph{i})~an implicit entity, (\emph{ii})~text extracted from the label of the vial, which has little connection to the overlaid written text, (\emph{iii})~unclear target entity in the meme (\emph{Vladimir Putin}). Such complexities are not common in the multimodal tasks we discussed above. The textual representation of the entities and their roles are different than for typical CoNLL-style semantic role labeling tasks \cite{carreras2005introduction}, which makes it more difficult to address the problem in the same formulation.

By observing these challenges, we first attempted to address the problem in the same formulation: as a sequence labeling problem by converting the data to CoNLL format (see Section~\ref{sss:sequence_labeling}). Then, we further tried to address it as a classification task, i.e., predict the role of each entity in a given meme--entity pair.

\begin{figure}
    \centering
    \includegraphics[width=0.47\textwidth]{figures/task_example.png}
    \caption{An example image showing the implicit (\emph{Salman Khan}) and the explicit entities (from a text perspective) and their roles.
    }
    \label{fig:task_example}
\end{figure}

\subsection{Data}
\label{ssec:data}

We use the dataset provided for the CONSTRAINT 2022 shared task.
It contains harmful memes, OCR-extracted text from these memes, and manually annotated entities with four roles: \textit{hero}, \textit{villian}, \textit{victim}, and \textit{other}.
The datasets cover two domains: COVID-19 and US Politics. The COVID-19 domain consists of 2,700 training and 300 validation examples, while US Politics has 2,852 training and 350 validation examples. The test dataset combines examples from both domains, COVID-19 and US Politics, and has a total of 718 examples.

For the experiments, we combined the two domains, COVID-19 and US Politics, which resulted in 5,552 training and 650 validation examples.

The class distribution of the entity roles, aggregated over all memes, in the combined COVID-19 + US Politics dataset is highly imbalanced as shown in Table~\ref{tab:data_distribution}. We can see that overall the role of \emph{hero} represents only 2\%, and the role of \emph{victim} covers only 5\% of the entities. We can further see that the vast majority of the entities are labeled with the \emph{other} role. This skewed distribution adds additional complexity to the modeling task.

\begin{table}[]
\centering
\setlength{\tabcolsep}{2.0pt}
\scalebox{1.0}{
\begin{tabular}{@{}lrrrrrr@{}}
\toprule
\multicolumn{1}{c}{\textbf{Class label}} & \multicolumn{2}{c}{\textbf{Train}} & \multicolumn{2}{c}{\textbf{Val}} & \multicolumn{2}{c}{\textbf{Test}} \\ \midrule
\multicolumn{1}{c}{\textbf{}} & \multicolumn{1}{c}{\textbf{Count}} & \multicolumn{1}{c}{\textbf{\%}} & \multicolumn{1}{c}{\textbf{Count}} & \multicolumn{1}{c}{\textbf{\%}} & \multicolumn{1}{c}{\textbf{Count}} & \multicolumn{1}{c}{\textbf{\%}} \\ \midrule
Hero & 475 & 2 & 224 & 3 & 52 & 2 \\
Villain & 2,427 & 10 & 886 & 10 & 350 & 14 \\
Victim & 910 & 5 & 433 & 5 & 114 & 5 \\
Others & 13,702 & 83 & 6,937 & 82 & 1,917 & 79 \\ \midrule
Total & 17,514 &  & 8,480 &  & 2,433 & \\ \bottomrule
\end{tabular}
}
\caption{Distribution of the entity roles in the combined COVID-19 + US politics datasets.}
\label{tab:data_distribution}
\end{table}

\section{Results and Discussion}
\label{sec:results}

Below, we first discuss our sequence labeling and classification experiments. We then perform some analysis, and finally, we put our results in a broader perspective in the context of the shared task.

\subsection{Sequence Labeling Results}

Table~\ref{tab:sequence_classification_results} shows the evaluation results on the test set for our sequence labeling reformulation of the problem. We performed two experiments: one where we used as input the entire meme text (i.e.,~all tokens), and another one where we used the concatenation of the target entities only. We can see that the latter performed marginally better, but overall the macro-F1 score is quite low in both cases.

\begin{table}[t]
\centering
\setlength{\tabcolsep}{3.0pt}
\scalebox{0.95}{
\begin{tabular}{@{}lrrrr@{}}
\toprule
\multicolumn{1}{c}{\textbf{Exp.}} & \multicolumn{1}{c}{\textbf{Acc}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} \\ \midrule
 All tokens & 0.51 & 0.32 & 0.21 & 0.24 \\
 Only entities & 0.77 & 0.40 & 0.27 & 0.25 \\
\bottomrule
\end{tabular}
}
\caption{Evaluation results on the test set for the sequence labeling reformulation of the problem.}
\label{tab:sequence_classification_results}
\end{table}

\subsection{Classification Results}

Table \ref{tab:classification_results} shows the evaluation results on the test set for our classification reformulation of the problem. We computed the \textit{majority class} baseline (row 0), which always predicts the most frequent label in the training set. Due to time limitations, our official submission used the image modality only, which resulted in a very low macro-F1 score of 0.23, as shown in row 1.
For our text modality experiments, we used the meme text and the entities. We experimented with BERT and XLM-RoBERTa, obtaining better results using the former. Using the BLOCK fusion technique on unimodal (text + entity) and multimodality (text + image + entity) yielded sizable improvements. The combination of image + text (rows 6 and 9) did not yield much better results compared to using text only (row 4).
Next, we added attention on top of block fusion, which improved the performance, but there was no much difference between the different combinations (rows 7--9).
Considering only the text and the entity, we observe an improvement using text augmentation. Among the different augmentation techniques, there was no performance difference between WordNet and BERT, and combining them yielded worse results.

\begin{table}[h]
\centering
\setlength{\tabcolsep}{2.0pt}
\scalebox{0.9}{
\begin{tabular}{@{}llrrrr@{}}
\toprule
 & \multicolumn{1}{c}{\textbf{Exp.}} & \multicolumn{1}{c}{\textbf{Acc}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} \\ \midrule
\multicolumn{6}{c}{\textbf{Baseline}} \\ \midrule
0 & Majority & 0.79	& 0.20 & 0.25 & 0.22 \\ \midrule
\multicolumn{6}{c}{\textbf{Image modality}} \\ \midrule
\it 1 & \it EffNet feat + SVM & \it 0.72 & \it 0.24 & \it 0.25 & \it 0.23 \\ \midrule
\multicolumn{6}{c}{\textbf{Text modality}} \\ \midrule
2 & BERT & 0.76 & 0.42 & 0.36 & 0.37 \\
3 & XLM-RoBERTa & 0.75 & 0.38 & 0.32 & 0.32 \\ \midrule
\multicolumn{6}{c}{\textbf{Multimodality/Fusion}} \\ \midrule
\multicolumn{6}{l}{\textbf{BLOCK fusion}} \\
4 & Entity + Text & 0.74 & 0.44 & 0.43 & 0.43 \\
5 & Entity + Image & 0.74 & 0.39 & 0.39 & 0.39 \\
6 & Entity + (Text + Image) & 0.75 & 0.43 & 0.42 & 0.41 \\ \midrule
\multicolumn{6}{l}{\textbf{Attention}} \\
7 & Entity + Text & 0.72 & 0.42 & 0.48 & 0.44 \\
8 & Entity + Image & 0.71 & 0.42 & 0.48 & 0.44 \\
9 & Entity + (Text + Image) & 0.71 & 0.42 & 0.49 & 0.44 \\\midrule
\multicolumn{6}{l}{\textbf{Augmentation}} \\
10 & Entity + Text (WordNet aug) & 0.76 & 0.48 & 0.46 & \textbf{0.46} \\
11 & Entity + Text (BERT aug) & 0.74 & 0.46 & 0.46 & \textbf{0.46} \\
12 & Entity + Text (Mix aug) & 0.77 & 0.49 & 0.41 & 0.43 \\ \bottomrule
\end{tabular}
}
\caption{Evaluation results on the test set for our classification reformulation of the problem. Our official submission for the shared task is shown in \emph{italic}.}
\label{tab:classification_results}
\end{table}

\begin{table*}[h]
\centering
\setlength{\tabcolsep}{4.0pt}
\scalebox{0.92}{
\begin{tabular}{lrrrrrr|rrrrrr}
\toprule

\multicolumn{1}{c}{\textbf{}} &
\multicolumn{3}{c}{\textbf{E+I, w/o Att.}}  & \multicolumn{3}{c|}{\textbf{E+I, w/ Att.}} & \multicolumn{3}{c}{\textbf{E+[I+T], w/o Att.}}
& \multicolumn{3}{c}{\textbf{E+[I+T], w/ Att.}}\\ \midrule
 \multicolumn{1}{c}{\textbf{Role}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c|}{\textbf{F1}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} \\ \midrule
Hero & 0.06 &	0.02 &	0.03 &	0.09 &	0.15 &	\textbf{0.12} &	0.22 &	0.12 &	0.15 &	0.09 &	0.21 &	0.12 \\
Villain & 0.35 &	0.44 &	0.39 &	0.40 &	0.51 &	\textbf{0.45} &	0.39 &	0.54 &	0.45 &	0.39 &	0.54 &	0.45 \\
Victim &  0.30 &	0.25 &	0.28 &	0.33 &	0.39	& \textbf{0.35} &	0.23 &	0.18 &	0.20 &	0.31 &	0.45 &	\textbf{0.36} \\
Other &  0.86 &	0.84 &	0.85 &	0.88 &	0.81	& 0.84 & 0.87 &	0.84 &	0.85 &	0.89 &	0.77 &	0.82 \\
\bottomrule
\end{tabular}
}
\caption{Role-level results on the test set with (w/) or without (w/o) attention between the context (text, image) and the entity. (E: Entity, I: Image, Att.: Attention, T: Text)}
\label{tab:attention-result}
\end{table*}

\begin{table*}[h]
\centering
\setlength{\tabcolsep}{4.0pt}
\scalebox{0.95}{
\begin{tabular}{lrrrrrrrrr}
\toprule
\multicolumn{1}{c}{\textbf{}} &
\multicolumn{3}{c}{\textbf{No Aug.}}  & \multicolumn{3}{c}{\textbf{Aug. WordNet}} & \multicolumn{3}{c}{\textbf{Aug. BERT}} \\ \midrule
 \multicolumn{1}{c}{\textbf{Role}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} & \multicolumn{1}{c}{\textbf{P}} & \multicolumn{1}{c}{\textbf{R}} & \multicolumn{1}{c}{\textbf{F1}} \\ \midrule
Hero & 0.21 &	0.12 &	0.15 &	0.33 &	0.21 &	\textbf{0.26} &	0.30 &	0.25 &	\textbf{0.27} \\
Villain & 0.36 &	0.49 &	0.42 &	0.41 &	0.52 &	\textbf{0.46} &	0.39 &	0.51 &	\textbf{0.44} \\
Victim & 0.31 &	0.27 &	0.29 &	0.30 &	0.27 &	0.29 &	0.29 &	0.27 &	0.28 \\
Other & 0.87 & 0.83 &	0.85 &	0.87 &	0.84 &	0.86 &	0.87 &	0.83 &	0.85 \\
\bottomrule
\end{tabular}
}
\caption{Role-level results on the test set for the entity + text combination with and without augmentation.}
\label{tab:augmentation-result}
\end{table*}

\subsection{Role-Level Analysis}

Next, we studied the impact of using attention and data augmentation on the individual entity roles: \emph{hero}, \emph{villain}, \emph{victim}, and \emph{other}.

Table~\ref{tab:attention-result} shows the impact of using attention on (a)~entity + image (left side), and (b)~entity + [image + text] (right side) combinations. We can observe a sizable gain for the \emph{hero} (+0.09), the \emph{villain} (+0.06), and the \emph{victim} (+0.07) roles in the former case (a). However, for case (b), there is an improvement for the \textit{victim} role only; yet, this improvement is quite sizable: +0.16.

Table~\ref{tab:augmentation-result} shows the impact of data augmentation using WordNet or BERT on the individual roles. We can observe sizable performance gains of +0.11 for the \emph{hero} role, and +0.04 for the \emph{villain} role, when using WordNet-based data augmentation. Similarly, BERT-based data augmentation yields +0.12 for the \emph{hero} role, and +0.02 for the \emph{villain} role. However, the impact of either augmentation on the \emph{victim} and on the \emph{other} role is negligible.

\subsection{Official Submission}

For our official submission for the task, we used the image modality system from line 1 in Table~\ref{tab:classification_results}, which was quite weak, with a macro-F1 score of 0.23. Our subsequent experiments and analysis pointed to several promising directions: (\emph{i})~combining the textual and the image modalities, (\emph{ii})~using attention, (\emph{iii})~performing data augmentation. As a result, we managed to improve our results to 0.46. Yet, this is still far behind the F1-score of the winning system: 0.5867.

\section{Related Work}
\label{sec:related_work}

Below, we discuss previous work on semantic role labeling and harmful content detection, both in general and in a multimodal context.

\subsection{Semantic Role Labeling}

\paragraph{Textual semantic role labeling} has been widely studied in NLP, where the idea is to understand who did what to whom, when, where, and why. Traditionally, the task has been addressed using sequence labeling, e.g.,~\citet{fitzgerald2015semantic} used local and structured learning, experimenting with PropBank and FrameNet, and \citet{larionov2019semantic} investigated recent transformer models.

\paragraph{Visual semantic role labeling} has been explored for images and video. \citet{Yatskar_2016_CVPR} addressed situation recognition, and developed a large-scale dataset containing over 500 activities, 1,700 roles, 11,000 objects, 125,000 images, and 200,000 unique situations. The images were collected from Google and the authors addressed the task as a situation recognition problem.
\citet{pratt2020grounded} developed a dataset for situation recognition consisting of 278,336 bounding-box groundings to the 11,538 entity classes.
\citet{gupta2015visual} developed a dataset of 16K examples in 10K images with actions and associated objects in the scene with different semantic roles for each action.

\citet{yang2016grounded} worked on integrating language and vision with explicit and implicit roles. \citet{silberer2018grounding} learned frame–semantic representations of the images. \citet{sadhu2021visual} approached the same problem for video, developing a dataset of 29K 10-second movie clips, annotated with verbs and semantics roles for every two seconds of video content.

\subsection{Harmful Content Detection in Memes}

There has been significant effort for identifying misinformation, disinformation, and malinformation online \cite{Schmidt2017survey,bondielli2019survey,zhou2020survey,da2020survey,alam2021survey,afridi2021multimodal,coordinated:communities:2022,10.1007/978-3-030-99739-7_52}. Most of these studies focused on textual and multimodal content. Compared to that, modeling the harmful aspects of memes has not received much attention.

Recent effort in this direction include categorizing hateful memes \cite{kiela2020hateful}, detecting antisemitism \cite{chandra2021subverting}, detecting the propagandistic techniques used in a meme \cite{dimitrov2021detecting}, detecting harmful memes and the target of the harm \cite{pramanick-etal-2021-momenta-multimodal}, identifying the protected categories that were attacked~\cite{zia-etal-2021-racist}, and identifying offensive content \cite{suryawanshi-etal-2020-multimodal}. Among these studies, the most notable low-level efforts that advanced research by providing high-quality datasets to experiment with include shared tasks such as the \emph{Hateful Memes Challenge} \cite{kiela2020hateful}, the SemEval-2021 shared task on detecting persuasion techniques in memes \cite{SemEval2021-6-Dimitrov}, and the troll meme classification task \cite{dravidiantrollmeme-eacl}.

\citet{chandra2021subverting} investigated antisemitism along with its types as a binary and a multi-class classification problem using  pretrained transformers and convolutional neural networks (CNNs) as modality-specific encoders along with various multimodal fusion strategies. \citet{dimitrov2021detecting} developed a dataset with 22 propaganda techniques and investigated the different state-of-the-art pretrained models, demonstrating that joint vision--language models performed better than unimodal ones. \citet{pramanick-etal-2021-momenta-multimodal} addressed two tasks: detecting harmful memes and identifying the social entities they target, using a multimodal model with local and global information.

\citet{zia-etal-2021-racist} went one step further than a binary classification of hateful memes, focusing on a more fine-grained categorization based on the protected category that was being attacked (i.e.,~race, disability, religion, nationality, sex) and the type of attack (i.e.,~contempt, mocking, inferiority, slurs, exclusion, dehumanizing, inciting violence) using the dataset released in the WOAH 2020 Shared Task.\footnote{\url{http://github.com/facebookresearch/fine_grained_hateful_memes}}
\citet{fersini2019detecting} studied sexist memes and investigated the textual cues using late fusion. They also developed a dataset of 800 misogynistic memes covering different manifestations of hatred against women (e.g.,~body shaming, stereotyping, objectification, and violence), collected from different social media~\citep{france_mys2021}.

\citet{kiela2021hateful} summarized the participating systems in the Hateful Memes Challenge, where the best systems fine-tuned unimodal and multimodal pre-training transformer models such as VisualBERT \citep{li2019visualbert} VL-BERT \citep{su2019vl}, UNITER \citep{chen2019uniter}, VILLA \citep{gan2020large}, and built ensembles on top of them.

The SemEval-2021 propaganda detection shared task \citep{SemEval2021-6-Dimitrov} focused on detecting the use of propaganda techniques in the meme, and the participants' systems showed that multimodal cues were very important.

In the troll meme classification shared task \cite{dravidiantrollmeme-eacl}, the best system used ResNet152 and BERT with multimodal attention, and most systems used pretrained transformers for the text, CNNs for the images, and early fusion to combine the two modalities.

\paragraph{Combining modalities} causes several challenges, which arise due to representation issues (i.e.,~symbolic representation for language vs. signal representation for the visual modality), misalignment between the modalities, and fusion and transferring knowledge between the modalities. In order to address multimodal problems, a lot of effort has been paid to developing different fusion techniques such as (\emph{i})~{\em early fusion}, where low-level features from different modalities are learned, fused, and fed into a single prediction model \cite{jin2016novel,yang2018ti,zhang2019multi,spotfake,zhou2020safe,kang2020multi}, (\emph{ii})~{\em late fusion}, where unimodal decisions are fused with some mechanisms such as averaging and voting \cite{agrawal2017multimodal,qi2019exploiting}, and (\emph{iii})~{\em hybrid fusion}, where a subset of the learned features are passed to the final classifier (early fusion), and the remaining modalities are fed to the classifier later (late fusion) \cite{jin2017multimodal}. Here, we use early fusion and joint learning for fusion.

\section{Introduction}
\label{sec:intro}

Social media have become one of the main communication channels for sharing information online. Unfortunately, they have been abused by malicious actors to promote their agenda using manipulative content, thus continuously plaguing political events, and the public debate, e.g.,~regarding the ongoing COVID-19 infodemic \cite{alam-etal-2021-fighting-covid,10.1007/978-3-030-99739-7_52}.
Such type of content includes harm and hostility \cite{brooke-2019-condescending,joksimovic-etal-2019-automated},
hate speech \citep{fortuna2018survey}, offensive language \cite{zampieri-etal-2019-predicting,SOLID}, abusive language \citep{mubarak2017abusive}, propaganda \cite{EMNLP19DaSanMartino,da2020survey}, cyberbullying \citep{van-hee-etal-2015-detection},
cyber-aggression \citep{kumar2018benchmarking}, and other kinds of harmful content \citep{pramanick-etal-2021-momenta-multimodal,Survey:2022:Harmful:Memes}.

The propagation of such content is often done by coordinated groups \cite{coordinated:communities:2022} using automated tools and targeting specific individuals, communities, and companies. There have been many research efforts to develop automated tools to detect such kind of content.
Several recent surveys have highlighted these aspects, which include fake news \cite{zhou2020survey}, misinformation and disinformation \citep{alam2021survey,survey:media:2021,survey:stance:2022}, rumours \citep{bondielli2019survey}, propaganda \citep{da2020survey}, hate speech \citep{fortuna2018survey,Schmidt2017survey}, cyberbullying \citep{7920246}, offensive \citep{husain2021survey} and harmful content \citep{Survey:2022:Harmful:Memes}.

The content shared on social media comes in different forms: textual, visual, or audio-visual. Among other social media content, recently, \textit{internet memes} became popular. Memes are defined as ``a group of digital items sharing common characteristics of content, form, or stance, which were created by associating them and were circulated, imitated, or transformed via the Internet by many users''~\citep{shifman2013memes}. Memes typically consist of images containing some text \citep{shifman2013memes,suryawanshi-etal-2020-multimodal,suryawanshi-etal-2020-dataset}. They are often shared for the purpose of having fun. However, memes can also be created and shared with bad intentions. This includes attacks on people based on characteristics such as ethnicity, race, sex, gender identity, disability, disease, nationality, and immigration status \cite{zannettou2018origins,kiela2020hateful}. There has been research effort to develop computational methods to detect such memes, such as detecting hateful memes \cite{kiela2020hateful}, propaganda \cite{dimitrov2021detecting}, offensiveness \cite{suryawanshi-etal-2020-multimodal}, sexist memes \cite{fersini2019detecting}, troll memes \cite{dravidiantrollmeme-eacl}, and generally harmful memes \cite{pramanick-etal-2021-momenta-multimodal,DISARM}.

Harmful memes often target individuals, organizations, or social entities. \citet{pramanick-etal-2021-momenta-multimodal} developed a dataset where the annotation consists of (\emph{i})~whether a meme is harmful or not, and (\emph{ii})~whether it targets an individual, an organization, a community, or society. The CONSTRAINT-2022 shared task follows a similar line of research \cite{sharma2022report}. The entities in a meme are first identified and then the task asks participants to predict which entities are glorified, vilified, or victimized in the meme. The task is formulated as \textit{``Given a meme and an entity, determine the role of the entity in the meme: hero vs. villain vs. victim vs. other.''} More details are given in Section~\ref{sec:task_and_data}.

Memes are multimodal in nature, but the textual and the visual content in a meme are sometimes unrelated, which can make them hard to analyze for traditional multimodal approaches. Moreover, context (e.g., where the meme was posted) plays an important role for understanding its content. Another important factor is that since the text in the meme is overlaid on top of the image, the text needs to be extracted using OCR, which can result in errors that require additional manual post-editing \cite{dimitrov2021detecting}.

Here, we address a task about entity role labeling for harmful memes based on the dataset released in the CONSTRAINT-2022 shared task; see the task overview paper for more detail \cite{sharma2022report}. This task is different from traditional semantic role labeling in NLP \cite{palmer2010semantic}, where understanding \textit{who} did \textit{what} to \textit{whom}, \textit{when}, \textit{where}, and \emph{why} is typically addressed as a sequence labeling problem \cite{he-etal-2017-deep}. Recently, this has also been studied for visual content~\cite{sadhu2021visual}, i.e.,~situation recognition \cite{Yatskar_2016_CVPR,pratt2020grounded}, visual semantic role labeling \cite{gupta2015visual,silberer2018grounding,li2020cross}, and human-object interaction \cite{chao2015hico,chao2018learning}.

To address the entity role labeling for a potentially harmful meme, we investigate textual, visual, and multimodal content using different pretrained models such as BERT \cite{text_bert}, VGG16 \cite{simonyan2014very}, and other vision--language models \cite{BlockFusion2019}. We further explore different textual data augmentation techniques and attention methods. For the shared task participation, we used only the image modality, which resulted in an underperforming system in the leaderboard.

Further studies using other modalities and approaches improved the performance of our system, but it is still lower (0.464 macro F1) than the best system (0.586). Yet, our investigation might be useful to understand which approaches are useful for detecting the role of an entity in harmful memes.

Our contributions can be summarized as follows:
\begin{itemize}
\item we addressed the problem both as sequence labeling and as classification;
\item we investigated different pretrained models for text and images;
\item we explored several combinations of multimodal models, as well as attention mechanisms, and various augmentation techniques.
\end{itemize}

The rest of the paper is organized as follows: Section~\ref{sec:related_work} presents previous work, Section~\ref{sec:task_and_data} describes the task and the dataset, Section~\ref{sec:classification} formulates our experiments, Section~\ref{sec:results} discusses the evaluation results. Finally, Section~\ref{sec:conclusion} concludes and points to possible directions for future work.
\section{Conclusion and Future Work}
\label{sec:conclusion}

We addressed the problem of understanding the role of the entities in harmful memes, as part of the CONSTRAINT-2022 shared task. We presented a comparative analysis of the importance of different modalities: the text and the image. We further experimented with two task reformulations ---sequence labeling and classification---, and we found the latter to work better. Overall, we obtained improvements when using BLOCK fusion, attention between the image and the text representations, and data augmentation.

In future work, we plan to combine the sequence and the classification formulations in a joint multimodal setting. We further want to experiment with multi-task learning using other meme analysis tasks and datasets. Last but not least, we plan to develop better data augmentation techniques to improve the performance on the low-frequency roles.

\section*{Acknowledgments}
The work is part of the Tanbih mega-project, which is developed at the Qatar Computing Research Institute, HBKU, and aims to limit the impact of ``fake news,'' propaganda, and media bias by making users aware of what they are reading, thus promoting media literacy and critical thinking.

\section{Experiments}
\label{sec:classification}

\paragraph{Settings:} We addressed the problem both as a sequence labeling and as a classification task. Below, we discuss each of them in detail.

\paragraph{Evaluation measures:} In our experiments, we used accuracy, macro-average precision, recall, and F$_1$ score. The latter was the official evaluation measure for the shared task.

\subsection{Sequence Labeling}
\label{sss:sequence_labeling}
For the sequence labeling experiments, we first converted the OCR text and the entities to the CoNLL BIO-format. An example is shown in Figure~\ref{fig:data_iob_format}. To convert them, we matched the entities in the text and we assigned the same tag (role label) to the token in the text. For the implicit entity that is not in the text, we added them at the end of the text and we assigned them the annotated role; we labeled all other tokens with the O-tag.

We trained the model using Conditional Random Fields (CRFs) \cite{lafferty2001conditional}, which has been widely used in earlier work. As features, we used part-of-speech tags, token length, tri-grams, presence of digits, use of special characters, token shape, w2vcluster, LDA topics, words present in a vocabulary list built on the training set, and in a name list, etc.\footnote{More details about the feature set can be found at \url{https://github.com/moejoe95/crf-vs-rnn-ner}}
We ran two sets of experiments: (\emph{i})~using the same format, and (\emph{ii})~using only entities as shown in Figure~\ref{fig:data_iob_format}.

\begin{figure}
    \centering
    \includegraphics[width=0.47\textwidth]{figures/data_iob_format.png}
    \caption{Example with text in BIO format.}
    \label{fig:data_iob_format}
\end{figure}

\subsection{Classification}
\label{sss:classification}
For the classification experiments, we first converted the dataset into a classification problem. As it contains all examples with one or more entities, we reorganized the dataset so that an example contains an entity, OCR text, image, and entity role. Hence, the dataset size is now the same as the number of entity instances rather than memes. We ended up with 17,514 training examples, which is the number of training entities as shown in Table~\ref{tab:data_distribution}.

We then ran different unimodal and multimodal experiments: (\emph{i})~only text, (\emph{ii})~only meme, and (\emph{iii})~text and meme together. For each setting, we also ran several baseline experiments. We further ran advanced experiments such as adding attention to the network and text-based data augmentation. Figure~\ref{fig:experimental_pipeline} shows our experimental pipeline for this classification task. For the unimodal experiments, we used individual modalities, and we trained them using different pre-trained models.

Note that for the text modality, we ran several combinations of fusion (e.g., text and entity) experiments. For the multimodal experiments, we combined embedding from both modalities, and we ran the classification on the fused embedding, as shown in Figure~\ref{fig:experimental_pipeline}.

\begin{figure}
    \centering
    \includegraphics[width=0.48\textwidth]{figures/system_architecture.png}
    \caption{Diagram of our experimental pipeline.}
    \label{fig:experimental_pipeline}
\end{figure}

\subsubsection{Text Modality}
For the text modality, we experimented using BERT \cite{text_bert} and XLM-RoBERTa \cite{liu2019roberta}. We performed ten reruns for each experiment using different random seeds, and then we picked the model that performed best on the development set. We used a batch size of 8, a learning rate of 2e-5, a maximum sequence length of 128, three epochs, and categorical cross-entropy as the loss function. We used the Transformer toolkit
to train the transformer-based models.

Using the text-only modality, we also ran a different combination of experiments using the text and the entities, where we used  bilinear fusion to combine them. We discuss this fusion technique in more detail in Section~\ref{sssec:multimodal}.

\subsubsection{Image Modality}
For our experiments using the image modality, we extract features from a pre-trained model, and then we trained an SVM classifier using these features. In particular, we extracted features from the penultimate layer of the EfficientNet-b1 (EffNet) model \cite{tan2019efficientnet}, which was trained using the ImageNet dataset. For training the model using the extracted features, we used SVM with its default parameter settings, with no further optimization of its hyper-parameter values. We chose EffNet as it was shown to achieve better performance for some social media image classification tasks \cite{alam2021medic,alam2021social}.

\subsubsection{Multimodal: Text and Image}
\label{sssec:multimodal}

For the multimodal experiments, we used the BLOCK Fusion~\cite{BlockFusion2019} approach, which was originally proposed for question answering (QA). Our motivation is that an entity can be seen like a question about the meme context, asking for its role as an answer. In a QA setting, there are three elements: (\emph{i})~a context (image or text), (\emph{ii})~a question, and (\emph{iii})~a list of answers. The goal is to select the right answer from the answer list. Similarly, we have four types of answers (i.e.,~roles). The task formation is that for an entity and a context (image or text), we need to determine the role of the entity in that context.

BLOCK fusion is a multi-modal framework based on block-superdiagonal tensor decomposition, where tensor blocks are decomposed into blocks of smaller sizes, with the size characterized by a set of mode-$n$ ranks \cite{Lathauwer2008}.
It is a bilinear model that takes two vectors $x^1 \in R^I$  and $x^2 \in R^J$ as input and then projects them to a $K$-dimensional space with tensor products: $y=\mathcal{T}\times x^1\times x^2$,
where $y\in R^K$.  Each component of $y$ is a quadratic form of the inputs, $\forall k \in [1;K]$:

\begin{equation}
    y_k = \sum^I_{i=1}\sum^J_{j=1}\mathcal{T}_{ijk} x^1_i x^j_2
\label{eq:block}
\end{equation}

BLOCK fusion can model bilinear interactions between groups of features, while limiting the complexity of the model, but keeping expressive high dimensional mono-model representations \cite{BlockFusion2019}.
We used BLOCK fusion in different settings: (\emph{i})~for image and entity, (\emph{ii})~for text and entity, and (\emph{iii})~for text, image with entity.

\paragraph{Text and entity:}
We extracted embedding representation for the entity and the text using a pretrained BERT model. We then fed both embedding representations into linear layers of 512 neurons each. The output of two linear layers is taken as input to the trainable block fusion network. Then, a regularization layer and linear layer are used before the final layer.

\paragraph{Image and entity:}
To build embedding representations for the image and the entity, we used a vision transformer (ViT) \cite{Dosovitskiy2021} and BERT pretrained models. The output of two different modalities was then used as input to the block fusion network.

\paragraph{Image, text, and entity:}
In this setting, we first built embedding representations for the text and the image using a pretraind BERT and ViT models, respectively. Then, we concatenated these representations (text + image) and we passed them to a linear layer with 512 neurons. We then extracted embedding representation for the target entity using the pretraind BERT model. Afterwards, we merged the text + image and the entity representations and we fed them into the fusion layer. In this way, we combined the image and the text representations as a unified context, aiming to predict the role of the target entity in this context.

In all the experiments, we uses a learning rate of $1e^{-6}$, a batch size of 8, and a maximum length of the text of 512.

\subsubsection{Additional Experiments}
We ran two additional sets of experiments using attention mechanism and augmentation, as using such approaches has been shown to help in many natural language processing (NLP) tasks.

\paragraph{Attention:}
In the entity + image block fusion network, we used block fusion to merge the entity and the image representations. Instead of using the image representation directly, we used attention mechanism on the image and then we fed the attended features along with the entity representation into the entity + image block. To compute the attention, we used the PyTorchNLP library.\footnote{\url{http://github.com/PetrochukM/PyTorch-NLP}}
In a similar fashion, we applied the attention mechanism to the text and to the combined text + image representation.

\paragraph{Augmentation:}
Text data augmentation has recently gained a lot of popularity as a way to address data scarceness and class imbalance \cite{feng2021survey}. We used three types of text augmentation techniques to balance the distribution of the different class: (\emph{i})~synonym augmentation using WordNet, (\emph{ii})~word substitution using BERT, and (\emph{iii})~a combination thereof. In our experiments, we used the NLPAug data augmentation package.\footnote{\url{https://github.com/makcedward/nlpaug}}
Note that we applied six times augmentation for the \emph{hero} class, twice for the \emph{villain} class, and three times for the \emph{victim} class. These numbers were empirically set and require further investigation in future work.


\section{Introduction}

Van der Waals complexes containing open-shell species are of great
current interest. In particular, complexes containing atoms or
molecules with orbital angular momentum necessarily involve
multiple electronic states.\cite{DUBERNET:1991,
DUBERNET:open:1994} They provide a test-bed for studying
electronically non-adiabatic effects, which are important in the
theory of reaction dynamics. \cite{alexander2004, retail2005,
Ziemkiewicz:2005} In addition, the observation of pre-reactive
Van der Waals complexes trapped in bound levels \cite{Loomis:1997,
Liu:1999, wheeler2000, Lester:2001, Merritt:2005} can shed light
on intermolecular forces in the entrance and exit channels of
chemical reactions.\cite{Dubernet:clhcl:1994, Meuwly:2003,
klos2004, Fishchuk:2006} The form of these shallow, long-range
wells can be important in determining reaction outcomes
\cite{Skouteris:1999, wang2002} and transition-state geometries.
\cite{neumark2002, Neumark:2005}

In this paper, we consider the complex consisting of an open-shell
SH radical and an Ar atom. New \emph{ab initio} potential energy
surfaces (PESs) for the \emph{X} $^2\Pi$ state are presented, and
used in calculations of the bound rotation-vibration levels. We
discuss the possibility of employing an approximate approach in
the bound-state calculations, using a single adiabatic PES rather
than the two surfaces used in the standard method.
\cite{DUBERNET:1991} Finally, the bound-state energies and
wavefunctions are used to simulate the vibrationally resolved
electronic spectrum.

The Ar--SH cluster was first detected experimentally by Miller and
co-workers \cite{miller92} using laser-induced fluorescence
excitation spectroscopy. Subsequently this group developed
empirical potential energy surfaces for the complex, for both the
\emph{A} state \cite{miller97} and the \emph{X} state,
\cite{miller99} by fitting model functions to reproduce
laser-induced fluorescence (LIF) results. The region of the
\emph{A} state PES corresponding to the Ar--S-H configuration
(Jacobi angles between $\sim 90$ and 180 degrees) was determined
only approximately, because the fluorescence experiments did not
probe this zone. More recently, Hirst \emph{et al.}\
\cite{hirst2004} have presented a PES for the \emph{A} state based
on \emph{ab-initio} calculations at the RCCSD(T) level with the
aug-cc-pV5Z basis set. This surface was used to predict bound
vibrational levels in the Ar--S-H configuration \cite{hirst2004}
which have not, to our knowledge, been observed in experiment so
far. A possible reason why these levels have eluded detection is
discussed in section IV of this paper.

Sumiyoshi \emph{et al.}\ \cite{sumiyoshi2000} have recorded
high-resolution spectra for Ar--SH in the ground electronic state
using Fourier-transform microwave (FTMW) spectroscopy. These
authors also produced PESs for the \emph{X} state based on fitting
a function to reproduce their experimental results,
\cite{sumiyoshi2000} and these surfaces were later improved with
the aid of some \emph{ab initio} results.\cite{sumiyoshi2003}
Most recently, results from microwave-millimeter-wave
double-resonance spectroscopy \cite{sumiyoshi2005i} were employed
to determine new three-dimensional PESs for the \emph{X} state.
\cite{sumiyoshi2005ii}

The family of weakly bound clusters containing a rare gas atom and
either the OH or SH radical have been reviewed by Carter \emph{et
al.} \cite{miller2000} in 2000 and by Heaven \cite{heaven2005} in
2005.

The structure of the present paper is as follows. In Section II we
present new PESs for Ar--SH ($^2\Pi$) based entirely on \emph{ab
initio} calculations at the RCCSD(T) level with an aug-cc-pV5Z
basis set. In Section III we describe bound rotation-vibration
level calculations using these surfaces. We also investigate the
wavefunctions and introduce a new adiabatic approximation for the
bound states, including spin-orbit coupling. In Section IV the
results are combined with those of a previous study of the
\emph{A} $^2\Sigma^+$ state, in order to produce a high-quality
simulation of the vibrationally resolved electronic spectrum.

\section{Potential energy surfaces}
The geometry of the complex is specified in terms of body-fixed
Jacobi coordinates $r$, $R$ and $\theta$. $R$ is the length of the
vector $\mathbf{R}$ which links the center of mass of the SH
fragment to the Ar nucleus. The vector $\mathbf{r}$ links the S
nucleus to the H nucleus: its modulus $r$ is the SH bond length.
The angle between $\mathbf{R}$ and $\mathbf{r}$ is $\theta$, so
that $\theta=0$ corresponds to a linear Ar--H-S configuration. For
this work the bond length $r$ was held constant at the
experimentally determined equilibrium value of 1.3409 \AA,\
\cite{herzbergIV} which is justified because the vibrational
motion of the diatom is very weakly coupled to the relatively
low-frequency Van der Waals modes of interest.

Energies were calculated for a regular grid of geometry points
using the MOLPRO quantum chemistry program.\cite{molpro} These
points are at every distance $R$ from 3.25 {\AA } to 5.5 {\AA } in
steps of 0.25 {\AA } and for every angle $\theta$ from 0$^{\circ}$
to 180$^{\circ}$ in steps of 15$^{\circ}$. This gives a total of
130 points. We used the RCCSD(T) method (restricted coupled
cluster with single, double and non-iterative triple excitations)
\cite{knowles93, knowles2000} with the aug-cc-pV5Z basis set.
\cite{dunning89, dunning92, dunning94} The counterpoise procedure
of Boys and Bernardi \cite{boys70} was used to correct for basis
set superposition error (BSSE). This is the same level of theory
and basis set as were used in recent calculations of the PES for
the $A$-state of Ar--SH.\cite{hirst2004}

Two potential surfaces were obtained from the \emph{ab initio}
calculations. These correspond to two adiabatic electronic states:
one symmetric ($A'$) and one antisymmetric ($A''$) with respect to
reflection in the plane of the nuclei. The two states are
degenerate at linear geometries but nondegenerate at nonlinear
geometries: the splitting is an example of the Renner-Teller
effect. The interaction energies for each state were interpolated
using a 2D spline function, and contour plots of the resulting
surfaces are shown in Fig.\ \ref{fig:adiabaticPESs}. The $A'$ and
$A''$ surfaces result from the electronic Hamiltonian without
spin-orbit coupling: a discussion of surfaces including spin-orbit
coupling is presented in section IV.

\begin{figure}
\includegraphics[width=85mm]{one}
\caption{\emph{Ab initio} potential energy surface contour plots
for Ar--SH ({\em X}) in the $A'$ state (upper plot) and the $A''$
state (lower plot). Solid contour lines are shown at 10 cm$^{-1}$
intervals, ranging from 0 cm$^{-1}$ to $-150$ cm$^{-1}$ inclusive
for the $A'$ surface and 0 cm$^{-1}$ to $-120$ cm$^{-1}$ inclusive
for the $A''$ surface. Dashed contour lines are shown at 100
cm$^{-1}$ intervals from +100 cm$^{-1}$ to +500 cm$^{-1}$
inclusive for both surfaces. The linear Ar--H-S conformation
corresponds to $\theta $ = 0$^{\circ}$. \label{fig:adiabaticPESs}}
\end{figure}

The adiabatic surfaces (adiabats) are qualitatively similar to those
reported for He--SH and Ne--SH complexes.\cite{Cybulski:2000} The
latter were calculated at the RCCSD(T) level, using the smaller
aug-cc-pVTZ basis set, but with an additional set of bond functions,
and counterpoise correction. A comparison of the positions and
energies of the minima on the Ar--SH surfaces presented here with
those for $X$-state He--SH and Ne--SH is given in Table \ref{tab1}.
For all the $A''$ surfaces there is a global minimum in the linear
X--S-H configuration (where X is He, Ne or Ar) and a local minimum
in the linear X--H-S configuration. The X--H-S configurations are
saddle points on the $A'$ surfaces, which have shallow local minima
at $\theta=180^\circ$ and global minima at nonlinear configurations.
The $A'$ global minima are deeper than those on the $A''$ surfaces,
because in the $A'$ state the SH $\pi$ hole is directed towards the
Ar atom, resulting in reduced repulsion. The global minima for the
$A'$ state occur at angles $\theta$ that increase with the atomic
number of the rare gas atom. Also, as expected, the minima are
deeper for clusters containing heavier (and more polarizable) rare
gas atoms.

\begin{table}
\caption{\label{tab1} Positions and well depths of potential
minima on the $A'$ and $A''$ adiabatic surfaces for the $X$-state
of SH--rare gas clusters. The results for Ne--SH and He--SH
clusters are from ref.\ \onlinecite{Cybulski:2000}.}
\begin{tabular}{ccccc}
\hline\hline
Cluster & State & $R$/\AA & \qquad $\theta/^{\circ}$ \qquad & depth/cm$^{-1}$\\
\hline
Ar--SH & $A'$ & 3.678 & 66.6 & 157.69\\
Ne--SH & $A'$ & 3.611 & 57.2 & 57.05\\
He--SH & $A'$ & 3.639 & 54.4 & 25.97\\
\hline
Ar--SH & \quad $A'$ and $A''$ \quad & 3.801 & 180 & 128.54\\
Ne--SH & \quad $A'$ and $A''$ \quad & 3.593 & 180 & 54.27\\
He--SH & \quad $A'$ and $A''$ \quad & 3.593 & 180 & 25.27\\
\hline
Ar--SH & $A''$ & 4.274 & 0 & 125.22\\
Ne--SH & $A''$ & 4.101 & 0 & 45.75\\
He--SH & $A''$ & 4.126 & 0 & 21.16\\
\hline\hline
\end{tabular}
\end{table}

In order to perform dynamical calculations on Ar--SH, we need to
evaluate the matrix elements of the potential between electronic
states labelled with an angular momentum quantum number $\lambda$.
For this purpose it is convenient to re-express the potential
energy surfaces as the sum ($V_0$) and difference ($V_2$)
potentials
\begin{displaymath}
V_0(R,\theta)=\frac{1}{2}\left[{V_{A'}(R,\theta)+V_{A''}(R,\theta)}\right],
\end{displaymath}
\begin{displaymath}
V_2(R,\theta)=\frac{1}{2}\left[{V_{A'}(R,\theta)-V_{A''}(R,\theta)}\right].
\end{displaymath}
Contour plots of these surfaces are shown in Fig.\
\ref{fig:avdiffpots}. They are quite similar to those recently
presented by Sumiyoshi \emph{et al.}\ \cite{sumiyoshi2005ii} The
latter were fitted to reproduce experimental results, with
starting values for the potential parameters obtained from
\emph{ab initio} calculations (RCCSD(T)/aug-cc-pVTZ). The form of
our sum potential is also qualitatively similar to those recently presented
for Ne--SH and Kr--SH by Suma \emph{et al.}\cite{suma04}

\begin{figure}
\includegraphics[width=85mm]{two}
\caption{Contour plots for the Ar--SH ({\em X}) difference (upper
plot) and sum (lower plot) potential energy surfaces. Solid
contour lines are shown at 10 cm$^{-1}$ intervals, ranging from 0
cm$^{-1}$ to $-300$ cm$^{-1}$ inclusive for the difference surface
and from 0 cm$^{-1}$ to $-120$ cm$^{-1}$ inclusive for the sum
surface. Dashed contour lines are shown at 100 cm$^{-1}$ intervals
from +100 cm$^{-1}$ to +500 cm$^{-1}$ for the sum surface only.
The linear Ar--H-S conformation corresponds to $\alpha$ =
0$^{\circ}$. \label{fig:avdiffpots}}
\end{figure}

\section{Bound-state calculations}

\subsection{Coupled channel calculations}

The bound states of a complex such as Ar--SH (\emph{X}$^2\Pi$)
involve \emph{both} potential energy surfaces. In the present work
we use a coupled-channel approach to calculate the bound states.
In a body-fixed axis system the Hamiltonian operator is
\begin{equation}
H= -\frac{\hbar^2}{2\mu}R^{-1}\left(\frac{\partial^2}{\partial
R^2}\right) R + H_{\rm mon} + \frac{\hbar^2(\hat J
-\hat\jmath)^2}{2\mu R^2}+\hat V, \label{ham}
\end{equation}
where $H_{\rm mon}$ is the monomer Hamiltonian and $\hat V$ is the
intermolecular potential. In a full treatment including overall
rotation, the total wavefunction of the complex may be expanded
 \cite{DUBERNET:1991}
\begin{equation}
\Psi^{JM}_n=R^{-1}\sum_{jP\lambda\sigma}
\Phi^{JM}_{jP;\lambda\sigma}\,\chi^{J}_{Pn;j;\lambda\sigma}(R),
\label{fullwave} \end{equation} where the channel basis functions
are
\begin{eqnarray}
\Phi^{JM}_{jP;\lambda\sigma} = \varphi_\sigma \varphi_\lambda&&
\left({2j+1\over4\pi}\right)^\frac{1}{2}
D^{j*}_{P\omega}(\phi,\theta,0) \nonumber\\
\times&&\left({2J+1\over4\pi}\right)^\frac{1}{2}
D^{J*}_{MP}(\alpha,\beta,0), \label{chanbas}
\end{eqnarray}
The monomer basis functions are labelled by Hund's case (a)
quantum numbers $\lambda$ and $\sigma$, the projections of the
electronic orbital and spin angular momentum along the SH axis,
and $\omega=\lambda+\sigma$. \footnote{We adopt the convention of
using {\it lower-case} letters for all quantities that refer to
\emph{monomers}, and reserve upper-case letters for quantities
that refer to the complex as a whole.} $\lambda$, $\sigma$ and
$\omega$ are all signed quantities. The $D$ functions are Wigner
rotation matrices.\cite{Brink} The first $D$ function describes
the rotation of the monomer with respect to body-fixed axes, with
angular momentum quantum number $j$ (including electronic orbital
and spin angular momentum) and projection $P$ along the
intermolecular vector $\mathbf{R}$. The second $D$ function
describes the rotation of the complex as a whole, with total
angular momentum $J$ and projections $M$ and $P$ onto space-fixed
and body-fixed axes respectively. The angles ($\beta,\alpha)$
describe the orientation of the $\mathbf{R}$ vector in space.

The monomer Hamiltonian used here for SH($X^2\Pi$) is \cite{Brown}
\begin{equation}
H_{\rm mon} = b(\hat j-\hat l-\hat s)^2 + H_{\rm so},
\end{equation}
where the rotational constant $b$ is 9.465 cm$^{-1}$.
\cite{herzbergIV} For simplicity the spin-orbit Hamiltonian $H_{\rm
so}$ is taken to be independent of $R$ and $\theta$ and equal to
$a\lambda\sigma$, with $a=-378.5$ cm$^{-1}$.\cite{herzbergIV}

It is convenient to expand the sum and difference potentials in
terms of renormalized spherical harmonics $C_{lm}(\theta,\phi)$,
\begin{eqnarray}
V_0(R,\theta) = \sum_l V_{l0}(R) C_{l0}(\theta,0); \\
V_2(R,\theta) = \sum_l V_{l2}(R) C_{l2}(\theta,0).
\end{eqnarray}
The potential matrix elements between the angular basis functions
may then be written
\begin{eqnarray}&&\langle JM;jP;\lambda \sigma |\hat V|
JM;j'P';\lambda'\sigma'\rangle\nonumber\\
&&= \delta_{PP'} \delta_{\sigma\sigma'} \sum_l
V_{l,|\lambda-\lambda'|}(R) \, g_{l,\lambda-\lambda'}(j\omega;
j'\omega';P),
\end{eqnarray}
where the potential coupling coefficients are
\begin{eqnarray}
&&g_{l,\lambda-\lambda'}(j\omega; j'\omega';P)
\nonumber\\
&&\quad=(-1)^{P-\omega}\left[(2j+1)(2j'+1)
\right]^\frac{1}{2}\nonumber\\
&&\quad\times\left(\matrix{j&l&j'\cr
-\omega&\lambda-\lambda'&\omega'\cr}\right)
\left(\matrix{j&l&j'\cr -P&0&P\cr}\right).
\end{eqnarray}
The potential matrix elements are independent of $J$ and diagonal
in $P$. Nevertheless, in a full treatment the wavefunctions of the
complex are linear combinations of functions with different values
of $P$, because the operator $(\hat J - \hat\jmath)^2$ in Eq.\
\ref{ham} has matrix elements off-diagonal in $P$ ($\Delta
P=\pm1$). However, the full wavefunctions are eigenfunctions of
the parity operator: symmetrized basis functions may be
constructed by taking even and odd linear combinations of
$\Phi^{JM}_{jP;\lambda\sigma}$ and
$\Phi^{JM}_{j{-P};{-\lambda}{-\sigma}}$.

In the present work, the coupled equations are solved using the
BOUND program of Hutson.\cite{Hutson:bound:2006} The wavefunction
log-derivative matrix is propagated outwards from a boundary point
at short range ($R_{\rm min}$) and inwards from a boundary point
at long range ($R_{\rm max}$) to a matching point ($R_{\rm mid}$)
in the classically allowed region. If $E$ is an eigenvalue of the
Hamiltonian, the determinant of the difference between the two
log-derivative matrices at $R_{\rm mid}$ is zero.
\cite{Johnson:1978,Hutson:1994} The BOUND program locates
eigenvalues by searching for zeroes of the lowest eigenvalue of
the matching determinant,\cite{Hutson:1994} using bisection
followed by the secant method. In the present work we use $R_{\rm
min}=3.0$~\AA, $R_{\rm max}=9.5$~\AA, $R_{\rm mid}=4.2$~\AA\ and a
log-derivative sector size of 0.02 \AA. The basis set includes all
SH functions up to $j_{\rm max}=15/2$ in both spin-orbit
manifolds.

The energies obtained from full close-coupling calculations for
the lowest few $J=3/2$ levels of Ar--SH (actually carried out in
the equivalent space-fixed basis set \cite{DUBERNET:1991}) are
shown in Table \ref{tab:energies}. These levels all correlate with
SH $^2\Pi_{3/2}$, $j=3/2$ and are labelled with the projection
quantum number $P$ and Van der Waals stretching quantum number
$n$. We use the convention that levels in which $P$ and $\omega$
for the dominant basis functions have the \emph{same} sign are
labelled with positive $P$ and those where they have
\emph{different} sign are labelled with negative $P$.
\cite{DUBERNET:1991} In order of increasing energy, the lowest
four levels for Ar--SH have $P$ = +3/2, +1/2, $-3/2$, $-1/2$, in
contrast to Ar--OH where the order is +3/2, +1/2, $-1/2$, $-3/2$.
\cite{DUBERNET:1991, Dubernet:1993} The difference is due to the
anisotropy of the sum potential $V_0(R,\theta)$: the ratio
$V_{20}/V_{10}$ is larger for Ar--SH.

\begin{table}
\caption{Bound-state energies for $J=3/2$ levels of Ar--SH from
full close-coupling calculations (average $E_{\rm CC}$ and parity
splitting $\Delta E_{\rm CC}$), helicity decoupling calculations
($E_{\rm HD}$) and single-surface calculations on the lower
adiabatic surface including spin-orbit coupling ($E_{\rm Ad}$).
All energies are relative to the dissociation energy to form SH
($X^2\Pi_{3/2},\ j=3/2$). All energies are given as wavenumbers in
cm$^{-1}$. }\label{tab:energies}
\begin{ruledtabular}
\begin{tabular}{cccccc}
$P$ & $n$ & $E_{\rm CC}$ & $\Delta E_{\rm CC}$ & $E_{\rm HD}$ & $E_{\rm Ad}$ \\
$+3/2$ & 0 & $-102.745$ & $+3.5\times10^{-5}$ & $-102.652$ & $-102.725$  \\
$+1/2$ & 0 &  $-97.766$ & +0.144              &  $-97.593$ &  $-97.667$  \\
$-3/2$ & 0 &  $-94.940$ & $-1.1\times10^{-3}$ &  $-94.894$ &  $-95.035$  \\
$-1/2$ & 0 &  $-92.116$ & $-0.138$            &  $-92.222$ &  $-92.293$  \\
\\
$+3/2$ & 1 &  $-77.292$ & $+2.6\times10^{-5}$ &  $-77.111$ &  $-77.258$   \\
$+1/2$ & 1 &  $-72.276$ & +0.134              &  $-72.100$ &  $-72.148$   \\
$-3/2$ & 1 &  $-69.356$ & $-1.3\times10^{-3}$ &  $-69.265$ &  $-69.382$   \\
$-1/2$ & 1 &  $-67.207$ & $-0.124$            &  $-67.291$ &  $-67.315$   \\
\end{tabular}
\end{ruledtabular}
\end{table}

The close-coupling results may be compared with the microwave
experiments of Sumiyoshi \emph{et al.}\ ,\cite{sumiyoshi2000} who
obtained a rotational constant $B^{\rm eff}=1569.66$ MHz (0.05236
cm$^{-1}$) and parity doubling constant $q_J=0.32873$ MHz
($1.10\times10^{-5}$ cm$^{-1}$) for the ground state ($P=+3/2$).
These correspond to a $J=3/2-5/2$ separation of 0.262 cm$^{-1}$
and a $J=3/2$ parity splitting of $6.6\times10^{-5}$ cm$^{-1}$,
which compare with calculated values of 0.263 cm$^{-1}$ and
$3.5\times10^{-5}$ cm$^{-1}$ respectively. The very good agreement
for the rotational spacing suggests that the equilibrium distance
of our \emph{ab initio} potential is quite accurate. The
difference of almost a factor of 2 in the parity splitting is less
satisfactory, but Dubernet \emph{et al.}\ \cite{Dubernet:1992} have
shown that such terms involve complicated combinations of
high-order terms involving the difference potential, spin
uncoupling and Coriolis perturbations. Small differences between
the energies of excited states can have a large effect on the
parity splitting. Sumiyoshi \emph{et al.}\ \cite{sumiyoshi2005i}
have very recently measured microwave--millimetre-wave
double-resonance spectra of the $P=+1/2\leftarrow +3/2$ band of
Ar--SH. The centre of gravity of the parity components of the
$J=3/2\leftarrow 1/2$ line is 81.8 GHz (2.73 cm$^{-1}$). The
corresponding calculated quantity from our potential is 4.805
cm$^{-1}$. In addition, the measured parity splitting for the
$J=3/2$, $P=+1/2$ level is about 5300 MHz (0.177 cm$^{-1}$), which
compares with 0.144 cm$^{-1}$ from our calculations. An
interesting possibility for future work would be to adjust the
\emph{ab initio} potential to improve the fit to the spectroscopic
parameters using the morphing procedure of Meuwly and Hutson.
\cite{Meuwly:1999}

\subsection{Wavefunctions}

The full wavefunctions (Eq.\ \ref{fullwave}) contain contributions
from all possible values of $P$ and $\omega$ and are not separable
between the body-fixed angles ($\theta,\phi$) and the space-fixed
angles ($\beta,\alpha$). This makes them hard to visualize. In
addition, since the mixings depend on the total angular momentum
$J$, they are not convenient for calculating band intensities. We
therefore introduce two approximations to simplify the description
of the wavefunctions for this purpose. First, we introduce the
\emph{helicity decoupling} approximation, where matrix elements of
$(\hat J - \hat\jmath)^2$ off-diagonal in $P$ are neglected.
Secondly, we neglect matrix elements of $H_{\rm mon}$ off-diagonal
in $\sigma$ (spin-uncoupling terms). The coupled equations then
simplify to
\begin{widetext}
\begin{eqnarray}
\left[-\frac{\hbar^2}{2\mu} \frac{d^2}{dR^2} + E_{\omega j}^{\rm
mon} \right. &+& \left. \frac{\hbar^2}{2\mu
R^2}\left(J(J+1)+j(j+1)-2\omega^2\right)-E^J_{Pn}\right]
\chi^{J}_{Pn;j;\lambda\sigma}(R) \nonumber\\ &=& -\sum_{j'\lambda'}
\left<JM;j'P\lambda' \sigma\right|\hat V\left|JM;jP\lambda
\sigma\right> \chi^{J}_{Pn;j';\lambda'\sigma}(R). \label{eqcoup}
\end{eqnarray}
\end{widetext}
Since all matrix elements off-diagonal in $P$ and $\sigma$ have
been neglected, states with quantum numbers $(P,\omega)$ and
$(-P,-\omega$) are uncoupled and it is not necessary to take
combinations of definite total parity. However, states with
$(P,\omega)$ and $(-P,\omega)$ or $(P,-\omega)$ have different
potential energies and are nondegenerate.

The energy levels obtained from helicity decoupling calculations
for Ar--SH are included in Table \ref{tab:energies}. The
approximation is accurate to about 0.2 cm$^{-1}$ for $n=0$ and 1,
but is less reliable for higher states. In particular, the region
between $-60$ and $-40$ cm$^{-1}$ contains both $j=3/2,\ n=2$ and
$j=5/2,\ n=0$ levels. In the presence of the resulting
near-degeneracies, the terms that are omitted in the approximate
Hamiltonian can cause quite significant level shifts.

In the helicity decoupling approximation, $P$ is a good quantum
number. However, $\omega$ is not because $V_2$ mixes levels with
$\Delta\lambda=\pm2$ (but $\Delta\sigma=0$) and thus mixes
$\omega=+3/2$ with $-1/2$ and $\omega=+1/2$ with $-3/2$. However,
in the absence of terms off-diagonal in $\sigma$ the two sets are
not mixed with one another. Each wavefunction thus has only
\emph{two} components corresponding to different values of
$\omega$. The wavefunctions may be written
\begin{equation} \Psi^J_{Pn} = \sum_\omega
\chi^J_{Pn;\omega}(R,\theta) \, \Phi^{JM}_{P;\lambda\sigma},
\label{eqpsip}
\end{equation} where the basis functions now exclude the
$\theta$-dependence,
\begin{eqnarray}
\Phi^{JM}_{P;\lambda\sigma} = \varphi_\sigma \varphi_\lambda
\left(\frac{2J+1}{8\pi^2}\right)^{1/2}
D^{J*}_{MP}(\alpha,\beta,\phi),
\end{eqnarray}
and the 2-dimensional functions that characterize the components
of the wavefunction for each $\omega$ are
\begin{equation}
\chi^J_{Pn;\omega}(R,\theta) = \sum_j
\left(j+\textstyle\frac{1}{2}\right)^{1/2} d^j_{P\omega}(\theta)
\chi^{J}_{Pn;j;\lambda\sigma}(R), \label{eqchi}
\end{equation}
where $d^j_{j\omega}(\theta)$ is a reduced rotation matrix.
\cite{Brink}

We have adapted the BOUND program \cite{Hutson:bound:2006} to
calculate wavefunctions for this case by back-substituting into
the log-derivative propagation equations, as described for the
closed-shell (single surface) case by Thornley and Hutson.
\cite{Thornley:1994} Examples of the resulting wavefunctions
$\chi^J_{Pn;\omega}(R,\theta)$ are shown in Fig.\ \ref{wfnbig}. It
may be seen that the components for different values of $\omega$
have quite different radial and angular distributions.

\begin{figure*}
\includegraphics[width=\textwidth]{three}
\caption{[Colour Figure] Contour plots of the wavefunction
components, superimposed on the average potential.\label{wfnbig}}
\end{figure*}

The Ar--SH wavefunctions are qualitatively similar to those for
Ne--SH obtained by Cybulski {\em et al.}\ \cite{Cybulski:2000}
Since the potential anisotropy for Ar--SH is only a few tens of
cm$^{-1}$ in the well region, there is only weak mixing of SH
rotational functions with different values of $j$. For this reason
the wavefunctions are dominated by the functions
$d^{3/2}_{P\omega}(\theta)$, as described by Dubernet and Hutson
\cite{DUBERNET:1991} for the case of Ar--OH. The $d$ functions are
shown, for example, in Fig.\ 7 of ref.\ \onlinecite{DUBERNET:1991}
and the angular parts of the wavefunctions of Fig.\ \ref{wfnbig}
follow them quite closely.

\subsection{Adiabatic approximations}

In a basis set of Hund's case (a) functions with signed values of
$\lambda=\pm1$ and $\sigma=\pm1/2$, we can define new adiabatic
surfaces (adiabats) including spin-orbit coupling as eigenvalues
of the Hamiltonian matrix at each value of $R$ and $\theta$,
\begin{equation} \left(\matrix{V_0+\frac{1}{2}a & 0 & V_2 & 0 \cr
              0 & V_0-\frac{1}{2}a & 0 & V_2 \cr
              V_2 & 0 & V_0-\frac{1}{2}a & 0 \cr
              0 & V_2 & 0 & V_0+\frac{1}{2}a \cr}\right),
\end{equation}
where again the spin-orbit Hamiltonian is taken to be simply
$a\lambda\sigma$. This clearly factorizes into two equivalent
$2\times2$ matrices, one containing $\omega=+3/2$ and $-1/2$ and
the other containing $\omega=+1/2$ and $-3/2$. The resulting
adiabats may be designated $V_+(R,\theta)$ and $V_-(R,\theta)$
with corresponding electronic functions $\psi_+(R,\theta)$ and
$\psi_-(R,\theta)$ given by
\begin{eqnarray} &&\left(\matrix{\psi_+(R,\theta) \cr \psi_-(R,\theta) \cr}\right)
\nonumber\\&&\quad=
\left(\matrix{\cos\alpha_{\rm ad}(R,\theta) & \sin\alpha_{\rm
ad}(R,\theta) \cr -\sin\alpha_{\rm ad}(R,\theta) & \cos\alpha_{\rm
ad}(R,\theta)}\right) \left(\matrix{\varphi_{\pm3/2} \cr
\varphi_{\mp1/2} \cr}\right), \label{anglead}
\end{eqnarray}
where $\varphi_\omega=\varphi_\lambda\varphi_\sigma$. The adiabats
for Ar--SH are shown in Fig.\ \ref{fig:so_adiabats}. Since for
Ar--SH $V_2(R,\theta)$ is small compared to $a$ in the well
region, the lower adiabat is always predominantly $\omega=\pm3/2$
in character and the upper adiabat is always predominantly
$\omega=\mp1/2$ in character. The corresponding mixing angle
$\alpha_{\rm ad}$ is 0 at $\theta=0$ and $180^\circ$ (where
$V_2(R,\theta)=0$) and less than $20^\circ$ at other angles for
$R>3.5$~\AA. A contour plot of the mixing angle is shown in Fig.\
\ref{mix_ang_surf}.

\begin{figure}
\includegraphics[width=85mm]{four}
\caption{Contour plots for Ar--SH ({\em X}) adiabats including
spin-orbit coupling. Contour lines are shown at 10 cm$^{-1}$
intervals, ranging from 70 cm$^{-1}$ to $+a/2$ for the upper
surface and from $-310$ cm$^{-1}$ to $-a/2$ for the lower surface.
\label{fig:so_adiabats}}
\end{figure}

\begin{figure}
\includegraphics[width=85mm]{five}
\caption{Contour plot of the adiabatic mixing angle
$\alpha_{\textrm{ad}}$. This angle is derived from the adiabats
and is defined in Eq.\ \ref{anglead}. The contour lines are spaced
at $2^\circ$ intervals.\label{mix_ang_surf}}
\end{figure}

A further consequence of the large spin-orbit coupling constant is
that both adiabats resemble the {\it sum} potential
$V_0(R,\theta)$ much more than the $A'$ and $A''$ potentials. The
spin-orbit coupling has in effect quenched the splitting between
the $A'$ and $A"$ states. This explains why there is no tendency
for the wavefunctions shown in Fig.\ \ref{wfnbig} to ``fall into"
the non-linear minimum of the $A'$ state.

The existence of adiabats including spin-orbit coupling suggests a
Born-Oppenheimer separation in which the total wavefunctions are
written approximately as
\begin{equation} \Psi_{iPn} \approx R^{-1} \psi_i(R,\theta)
\chi_{iPn}(R,\theta), \label{psibo} \end{equation} where $\psi_i$
is one of the functions of Eq.\ \ref{anglead} and
$\chi_{in}(R,\theta)$ is a solution of an effective Schr\"odinger
equation of the form
\begin{eqnarray} &&\left[ -\frac{\hbar^2}{2\mu}\frac{\partial^2}{\partial R^2} + H_{\rm
rot}+ \frac{\hbar^2(\hat J -\hat\jmath)^2}{2\mu R^2} \right.
\nonumber\\ && +\ \Biggl. V_i(R,\theta) - E_{iPn}
\Biggr]\chi_{iPn}(R,\theta) = 0. \label{eqad}
\end{eqnarray} However, the appropriate angular operator $H_{\rm rot}$
to use in such a calculation is hard to define. The reduced rotation
matrices $d^j_{P\omega}(\theta)$ that describe the free SH molecule
are eigenfunctions of $H_{\rm rot} = b(\hat\jmath^2-2\omega^2)$,
where
\begin{equation}
\hat\jmath^2 = \left[-\frac{1}{\sin\theta}
\frac{\partial}{\partial\theta} \left(\sin\theta
\frac{\partial}{\partial\theta} \right) + \frac{P^2 + \omega^2
-2P\omega\cos\theta}{\sin^2\theta} \right]. \label{eqhrot}
\end{equation}
This contains singularities at $\theta=0$ and/or $180^\circ$ that
depend on the values of $P$ and $\omega$. However, there is no
single value of $\omega$ that is appropriate at all
configurations. The simplest approach is to replace $\omega$ in
Eq.\ \ref{eqhrot} with the value that is appropriate at $\theta=0$
and $180^\circ$ and solve Eq.\ \ref{eqad} in a basis set of $d$
functions for each value of $P$. This is equivalent to solving the
coupled equations using a basis set containing only functions with
a single value of $\omega$. The results obtained with this
approximation are included in Table \ref{tab:energies}: it may be
seen that it gives energies that are generally slightly too low
(compared to the helicity decoupling results), by 0.05 to 0.15
cm$^{-1}$. A slightly better but significantly more complicated
approximation would be to replace $\omega$ with
$\langle\omega\rangle$ and $\omega^2$ with
$\langle\omega^2\rangle$ in Eq.\ \ref{eqhrot} to give an improved
effective potential.

One approach that is clearly {\it not} appropriate is to carry out
a bound-state calculation on a single adiabat $V_\pm(R,\theta)$
assuming that the SH molecule behaves as a closed-shell rigid
rotor. Such a calculation would give substantially incorrect
energies and wavefunctions.

It is in fact true that {\it no wavefunction of the form
(\ref{psibo}) can have the correct behavior near both linear
geometries}. To see this, consider an alternative definition of
the mixing angle that can be obtained from a single wavefunction
in the helicity decoupling approximation,
\begin{equation} \tan\alpha^J_{Pn}(R,\theta) =
\frac{\chi^J_{Pn;\mp1/2}(R,\theta)}{\chi^J_{Pn;\pm3/2}(R,\theta)}.
\label{anglewav}
\end{equation}
This quantity is plotted for $n=0$ and all four $P$ values
corresponding to $j=3/2$ in Fig.\ \ref{mix_ang_wfns}. The mixing
angles for $P=+3/2$ and $P=+1/2$ bear some similarity to
$\alpha_{\rm ad}(R,\theta)$ (Fig.\ \ref{mix_ang_surf}) at small
$\theta$, but tend to $90^\circ$ instead of zero at $\theta =
180^\circ$. Conversely, the mixing angles for $P=-3/2$ and
$P=-1/2$ tend to $90^\circ$ at $\theta=0$. This is easy to explain
in terms of the reduced rotation matrices that appear in Eq.\
\ref{eqchi}. For example, the functions
$d^j_{\pm3/2,\pm3/2}(\theta)$ all behave as $\cos^3 (\theta/2)$ as
$\theta\rightarrow 180^\circ$, while the functions
$d^j_{\mp1/2,\pm3/2}(\theta)$ behave as $\cos (\theta/2)$. This
corresponds to $\tan\alpha^J_{3/2,n} \rightarrow \infty$ as
$\theta\rightarrow 180^\circ$ so $\alpha^J_{3/2,n} \rightarrow
90^\circ$ in that limit. The point here is that the component of
the $P=+3/2$ wavefunction on the $\omega=-1/2$ surface goes to
zero more slowly than that on the $\omega=+3/2$ surface as
$\theta\rightarrow 180^\circ$. Fig.\ \ref{mix_ang_surf} shows that
the coupled-channel wavefunctions (\ref{eqpsip}) for $P=+3/2$ and
$+1/2$ are predominantly in the $\omega=-1/2$ state near
$\theta=180^\circ$, which corresponds to the {\it upper} adiabat
rather than the lower one. The $P=-3/2$ and $-1/2$ wavefunctions
show similar behavior around $\theta=0$. This is not the behavior
implied by Eq.\ \ref{psibo}.

\begin{figure}
\includegraphics[width=85mm]{six}
\caption{[Colour Figure] Comparison of the adiabatic mixing angle
$\alpha_{\mathrm{ad}}$ (thick black line) with angles obtained
from wavefunctions correlating with $j=3/2$, $\omega=+3/2$ for
$P=+3/2$ (red), $P=+1/2$ (blue), $P=-1/2$ (green) and $P=-3/2$
(black). The mixing angles are shown as cuts through the
corresponding surfaces taken at $R=3.9$ \AA.
\label{mix_ang_wfns}}
\end{figure}

\section{Electronic spectrum calculation}

\subsection{Transition wavenumbers}

In order to calculate the line positions in the vibrationally
resolved $A\ ^2\Sigma^+ \leftarrow X\ ^2\Pi$ electronic spectrum,
we require the bound-state energies for the excited electronic
state, as well as those for the ground state. For the \emph{A}
$^2\Sigma^+$ state we make use of the recent PES presented by
Hirst \emph{et al.}\ \cite{hirst2004} This surface has a global
minimum of $-742.5$ cm$^{-1}$ at the linear Ar--H-S conformation
($\theta = 0^{\circ}$), and a secondary minimum of $-673.7$
cm$^{-1}$ for linear Ar--S-H ($\theta = 180^{\circ}$). The two
minima are separated by a barrier more than $600$ cm$^{-1}$ high
and the lowest-energy vibrational levels are localised within one
or other of the two wells.

The bound states of this PES have been analysed previously
\cite{hirst2004} and only a brief discussion is given here.
Bound-state energies were calculated as eigenvalues of the spin-free
triatomic Hamiltonian in Jacobi coordinates. Discrete variable
representations (DVRs) were employed for both the intermolecular
distance $R$ and the angle $\theta$. For $R$, 120 sinc-DVR functions
\cite{colbertmiller} were used, with DVR points ranging from 2.5
\AA\ to 8.5 \AA. For $\theta$, a 64-point DVR based on Legendre
polynomials was used. With this basis set, the bound levels of
interest were converged to at least seven significant figures. The
resulting levels are labeled by quantum numbers
($v_{\mathrm{SH}},b^K,n$), where $v_{\mathrm{SH}}$ and $n$ are
quantum numbers for the SH stretch and the atom--diatom stretch
respectively. $K$ is the projection of the total angular momentum of
the diatom, neglecting spin, onto the body-fixed $z$-axis, and $b$
is the number of nodes in the intermolecular angle $\theta$. The
resulting energies for levels with total angular momentum $N = 0$
(neglecting spin) are in precise agreement with previous results.
\cite{hirst2004} To facilitate the calculation of band intensities,
for $N = 1$ the helicity decoupled approximation was employed, in
which the Coriolis terms coupling different $K$-levels are ignored.
The helicity-decoupled energies are within 0.5 cm$^{-1}$ of the full
close-coupled results.\cite{hirst2004}

For the purpose of calculating transition frequencies, the
asymptotic separation of the potentials is taken to be the
experimental excitation energy from the $v=0,\ j=3/2$ level of the
$^2\Pi_{3/2}$ state to the lowest $v=0,\ j=1/2$ level of the
$^2\Sigma^{+}$ state of isolated SH, which is 30832.68 cm$^{-1}$.
\cite{ramsay52} All transitions of the complex were assumed to
originate from the $P=+3/2$ level of Ar--SH ($^2\Pi$). The
lowest-energy transition frequency for the complex is calculated
to be 30488.5 cm$^{-1}$, which is 31.5 cm$^{-1}$ greater than the
experimental value of 30457 cm$^{-1}$.\cite{miller97} This
agreement is reasonable, considering the level of theory used in
the calculation of the potentials.

\subsection{Transition dipole moments}
Calculations of spectroscopic intensities require transition
dipole moments $\mu_{\mathrm{tot}}^{\mathrm{if}}$, where
\begin{equation}
\mu_{\mathrm{tot}}^{\mathrm{if}} = \left< \mathrm{i}|
\mu_{\mathrm{el}} | \mathrm{f}\right>.
\end{equation}
The integrals involve the initial (i) and final (f) wavefunctions
as determined from bound-state calculations. In this work we
evaluated transition dipoles over \emph{internal} coordinates
($R,\theta$), neglecting overall rotation. This gives transition
dipoles that correspond to band intensities between intermolecular
vibrational states. The electronic dipole moment,
$\mu_{\mathrm{el}}$, is in general a parametric function of the
nuclear coordinates. In the body-fixed frame it may be expanded in
terms of reduced rotation matrices,
\begin{equation}
\label{eqn:elecdipmom} \mu_{\mathrm{el}}(R,\theta) =\sum_j
\mu_{\mathrm{el},j}^{\Delta\lambda}(R) d^j_{\Delta P, \Delta
\lambda}(\theta),
\end{equation}
where $\Delta P = P_{\mathrm{i}}-P_{\mathrm{f}}$ and $\Delta
\lambda = \lambda_{\mathrm{i}}-\lambda_{\mathrm{f}}$. In the
excited electronic state, $P=K\pm\textstyle{\frac{1}{2}}$.
In this work it is assumed that $\mu_{\mathrm{el}}$ consists
purely of contributions from the SH monomer, so that only $j=1$
contributes in Eq.\ \ref{eqn:elecdipmom} and the coefficients
$\mu_{\mathrm{el},j}^{\Delta\lambda}$ are independent of $R$.
Since we are dealing with a perpendicular transition in SH,
$\Delta\lambda=\pm1$. The transition dipoles were calculated as
one-dimensional Gaussian quadratures in $\theta$, then integrated
over $R$.

\subsection{Intensities and lifetime factors}

The signals in a pulsed-laser fluorescence excitation experiment
decay exponentially following each pulse, with a lifetime equal to
that of the excited state being probed. Intensities are typically
measured as the area under the decay curve, and so the
experimental intensities are proportional to the lifetime of the
excited state. Ar--SH is somewhat unusual in the large range of
lifetimes exhibited by different vibrational levels in the
\emph{A} state. It is known that the presence of the Ar atom
blocks the electronic predissociation of the SH radical, leading
to a greatly increased lifetime of up to 600~ns for low-lying
bound levels, compared to $\sim 1$ ns for the uncomplexed species.
\cite{mccoy98, miller97} However, the actual lifetime depends on
the degree of vibrational excitation, and lifetimes specific to
particular levels have been calculated by McCoy.\cite{mccoy98}

The top panel of Fig.\ \ref{fig:simspec} shows a spectrum
calculated directly from the squares of transition dipoles, while
the center panel shows a spectrum in which the intensities have
been multiplied by McCoy's lifetime values. Clearly this is
possible only for levels for which lifetime data exist, and
transitions to other levels are omitted in the center panel
(\emph{i.e.}, it is assumed that their lifetimes are small). Also
shown is an experimental spectrum from Ref.\
\onlinecite{miller97}. The spacings between the peaks in the
calculated Ar--SH spectrum are consistently $\sim 5$\% smaller
than in experiment. It is clear that the intensity distributions
are significantly different in the two calculated spectra, and
that the one that includes lifetime factors gives considerably
better agreement with experiment. The agreement in intensities is
quite good, especially considering that the experimental spectrum
was most likely not normalised for dye laser power.
\cite{millerperscomm} From our results it seems likely that the
small peak at $\sim 30810$ cm$^{-1}$ in the experimental spectrum
can be assigned to the transition to $(0,0^0,6)$.

\begin{figure}
\includegraphics[width=85mm]{seven}
\caption{Calculated vibrationally resolved fluorescence-excitation
spectrum of Ar--SH for the \emph{A} $^2\Sigma^+$ -- \emph{X}
$^2\Pi$ electronic transition. The lifetime weighting of the intensities
is absent for the the top panel, and present for the
middle panel. Lines labeled with $+$ indicate transitions to the
$\theta=180^{\circ}$ well of the \emph{A} state.
For comparison, the experimental spectrum is shown in the
bottom panel, taken from Ref.\ \onlinecite{miller97}.
Note that the experimental spectrum contains contributions from
Ar$_2$--SH and larger clusters as well as Ar--SH.\label{fig:simspec}}
\end{figure}

Even without the lifetime weighting, transitions to levels localized
in the $\theta=180^{\circ}$ well of the \emph{A} state, which are
labelled with $+$ symbols in Fig.\ \ref{fig:simspec}, are weak in
the simulated spectrum. This arises because of poor overlap with the
$P=+3/2$ ground-state wavefunction (which is concentrated around
$\theta=0$). In addition, it is likely that such levels have short
lifetimes close to that of uncomplexed SH \cite{hirst2004} and so
will have even lower intensities in the fluorescence excitation
spectrum. These levels have not been observed experimentally to our
knowledge.

\section{Summary}

We have obtained new \emph{ab initio} potential energy surfaces
for the Ar--SH complex in its ground $^2\Pi$ electronic state and
used them to calculate bound-state energies and wavefunctions
using coupled-channel methods. We have also described a new
adiabatic approximation that includes spin-orbit coupling and can
be used to calculate bound states on a single potential energy
surface. However, the adiabatic wavefunctions fail to reproduce
some features of the true wavefunctions. We have used our results
to simulate the vibrationally resolved laser-induced fluorescence
excitation spectrum of Ar--SH, with intensities modelled using
calculated transition dipole moments and calculated lifetimes. The
inclusion of the lifetime factor is important to obtain
satisfactory agreement with the experimental intensities.

\section{Acknowledgements}
The authors would like to thank Terry Miller for providing his
experimental spectrum. RJD also thanks Dr. Stuart Mackenzie for
helpful discussions, and is grateful to the Engineering and
Physical Sciences Research Council (EPSRC) for funding.