@inproceedings{medina-grespan-etal-2023-logic,
    title = "Logic-driven Indirect Supervision: An Application to Crisis Counseling",
    author = "Medina Grespan, Mattia  and
      Broadbent, Meghan  and
      Zhang, Xinyao  and
      Axford, Katherine  and
      Kious, Brent  and
      Imel, Zac  and
      Srikumar, Vivek",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.654",
    doi = "10.18653/v1/2023.acl-long.654",
    pages = "11704--11722",
    abstract = "Ensuring the effectiveness of text-based crisis counseling requires observing ongoing conversations and providing feedback, both labor-intensive tasks. Automatic analysis of conversations{---}at the full chat and utterance levels{---}may help support counselors and provide better care. While some session-level training data (e.g., rating of patient risk) is often available from counselors, labeling utterances requires expensive post hoc annotation. But the latter can not only provide insights about conversation dynamics, but can also serve to support quality assurance efforts for counselors. In this paper, we examine if inexpensive{---}and potentially noisy{---}session-level annotation can help improve label utterances. To this end, we propose a logic-based indirect supervision approach that exploits declaratively stated structural dependencies between both levels of annotation to improve utterance modeling. We show that adding these rules gives an improvement of 3.5{\%} f-score over a strong multi-task baseline for utterance-level predictions. We demonstrate via ablation studies how indirect supervision via logic rules also improves the consistency and robustness of the system.",
}

Ensuring the effectiveness of text-based crisis counseling requires observing ongoing conversations and providing feedback, both labor-intensive tasks. Automatic analysis of conversations{---}at the full chat and utterance levels{---}may help support counselors and provide better care. While some session-level training data (e.g., rating of patient risk) is often available from counselors, labeling utterances requires expensive post hoc annotation. But the latter can not only provide insights about conversation dynamics, but can also serve to support quality assurance efforts for counselors. In this paper, we examine if inexpensive{---}and potentially noisy{---}session-level annotation can help improve label utterances. To this end, we propose a logic-based indirect supervision approach that exploits declaratively stated structural dependencies between both levels of annotation to improve utterance modeling. We show that adding these rules gives an improvement of 3.5{\%} f-score over a strong multi-task baseline for utterance-level predictions. We demonstrate via ablation studies how indirect supervision via logic rules also improves the consistency and robustness of the system.
Medina Grespan, Mattia Broadbent, Meghan Zhang, Xinyao Axford, Katherine Kious, Brent Imel, Zac Srikumar, Vivek
Logic-driven Indirect Supervision: An Application to Crisis Counseling