@inproceedings{basile-etal-2023-uinauil,
    title = "{UINAUIL}: A Unified Benchmark for {I}talian Natural Language Understanding",
    author = "Basile, Valerio  and
      Bioglio, Livio  and
      Bosca, Alessio  and
      Bosco, Cristina  and
      Patti, Viviana",
    editor = "Bollegala, Danushka  and
      Huang, Ruihong  and
      Ritter, Alan",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-demo.33",
    doi = "10.18653/v1/2023.acl-demo.33",
    pages = "348--356",
    abstract = "This paper introduces the Unified Interactive Natural Understanding of the Italian Language (UINAUIL), a benchmark of six tasks for Italian Natural Language Understanding. We present a description of the tasks and software library that collects the data from the European Language Grid, harmonizes the data format, and exposes functionalities to facilitates data manipulation and the evaluation of custom models. We also present the results of tests conducted with available Italian and multilingual language models on UINAUIL, providing an updated picture of the current state of the art in Italian NLU.",
}

This paper introduces the Unified Interactive Natural Understanding of the Italian Language (UINAUIL), a benchmark of six tasks for Italian Natural Language Understanding. We present a description of the tasks and software library that collects the data from the European Language Grid, harmonizes the data format, and exposes functionalities to facilitates data manipulation and the evaluation of custom models. We also present the results of tests conducted with available Italian and multilingual language models on UINAUIL, providing an updated picture of the current state of the art in Italian NLU.
Basile, Valerio Bioglio, Livio Bosca, Alessio Bosco, Cristina Patti, Viviana
UINAUIL: A Unified Benchmark for Italian Natural Language Understanding