@inproceedings{zhu-etal-2023-efficient,
    title = "An Efficient Conversational Smart Compose System",
    author = "Zhu, Yun  and
      Chen, Xiayu  and
      Shu, Lei  and
      Tan, Bowen  and
      Song, Xinying  and
      Liu, Lijuan  and
      Wang, Maria  and
      Chen, Jindong  and
      Ruan, Ning",
    editor = "Bollegala, Danushka  and
      Huang, Ruihong  and
      Ritter, Alan",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-demo.43",
    doi = "10.18653/v1/2023.acl-demo.43",
    pages = "456--462",
    abstract = "Online conversation is a ubiquitous way to share information and connect everyone but repetitive idiomatic text typing takes users a lot of time. This paper demonstrates a simple yet effective cloud based smart compose system to improve human-to-human conversation efficiency. Heuristics from different perspectives are designed to achieve the best trade-off between quality and latency. From the modeling side, the decoder-only model exploited the previous turns of conversational history in a computation lightweight manner. Besides, a novel phrase tokenizer is proposed to reduce latency without losing the composing quality further. Additionally, the caching mechanism is applied to the serving framework. The demo video of the system is available at \url{https://youtu.be/U1KXkaqr60g.We} open-sourced our phrase tokenizer in \url{https://github.com/tensorflow/text}.",
}

Online conversation is a ubiquitous way to share information and connect everyone but repetitive idiomatic text typing takes users a lot of time. This paper demonstrates a simple yet effective cloud based smart compose system to improve human-to-human conversation efficiency. Heuristics from different perspectives are designed to achieve the best trade-off between quality and latency. From the modeling side, the decoder-only model exploited the previous turns of conversational history in a computation lightweight manner. Besides, a novel phrase tokenizer is proposed to reduce latency without losing the composing quality further. Additionally, the caching mechanism is applied to the serving framework. The demo video of the system is available at \url{https://youtu.be/U1KXkaqr60g.We} open-sourced our phrase tokenizer in \url{https://github.com/tensorflow/text}.
Zhu, Yun Chen, Xiayu Shu, Lei Tan, Bowen Song, Xinying Liu, Lijuan Wang, Maria Chen, Jindong Ruan, Ning
An Efficient Conversational Smart Compose System