@inproceedings{poth-etal-2023-ml,
    title = "{ML} Mob at {S}em{E}val-2023 Task 1: Probing {CLIP} on Visual Word-Sense Disambiguation",
    author = "Poth, Clifton  and
      Hentschel, Martin  and
      Werner, Tobias  and
      Sterz, Hannah  and
      Bongard, Leonard",
    editor = {Ojha, Atul Kr.  and
      Do{\u{g}}ru{\"o}z, A. Seza  and
      Da San Martino, Giovanni  and
      Tayyar Madabushi, Harish  and
      Kumar, Ritesh  and
      Sartori, Elisa},
    booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.semeval-1.201",
    doi = "10.18653/v1/2023.semeval-1.201",
    pages = "1463--1469",
    abstract = "Successful word sense disambiguation (WSD)is a fundamental element of natural languageunderstanding. As part of SemEval-2023 Task1, we investigate WSD in a multimodal setting,where ambiguous words are to be matched withcandidate images representing word senses. Wecompare multiple systems based on pre-trainedCLIP models. In our experiments, we findCLIP to have solid zero-shot performance onmonolingual and multilingual data. By em-ploying different fine-tuning techniques, we areable to further enhance performance. However,transferring knowledge between data distribu-tions proves to be more challenging.",
}

Successful word sense disambiguation (WSD)is a fundamental element of natural languageunderstanding. As part of SemEval-2023 Task1, we investigate WSD in a multimodal setting,where ambiguous words are to be matched withcandidate images representing word senses. Wecompare multiple systems based on pre-trainedCLIP models. In our experiments, we findCLIP to have solid zero-shot performance onmonolingual and multilingual data. By em-ploying different fine-tuning techniques, we areable to further enhance performance. However,transferring knowledge between data distribu-tions proves to be more challenging.
Poth, Clifton Hentschel, Martin Werner, Tobias Sterz, Hannah Bongard, Leonard
ML Mob at SemEval-2023 Task 1: Probing CLIP on Visual Word-Sense Disambiguation