@inproceedings{rao-etal-2023-length,
    title = "Length-Aware {NMT} and Adaptive Duration for Automatic Dubbing",
    author = "Rao, Zhiqiang  and
      Shang, Hengchao  and
      Yang, Jinlong  and
      Wei, Daimeng  and
      Li, Zongyao  and
      Guo, Jiaxin  and
      Li, Shaojun  and
      Yu, Zhengzhe  and
      Wu, Zhanglin  and
      Xie, Yuhao  and
      Wei, Bin  and
      Zheng, Jiawei  and
      Lei, Lizhi  and
      Yang, Hao",
    editor = "Salesky, Elizabeth  and
      Federico, Marcello  and
      Carpuat, Marine",
    booktitle = "Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada (in-person and online)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.iwslt-1.9",
    doi = "10.18653/v1/2023.iwslt-1.9",
    pages = "138--143",
    abstract = "This paper presents the submission of Huawei Translation Services Center for the IWSLT 2023 dubbing task in the unconstrained setting. The proposed solution consists of a Transformer-based machine translation model and a phoneme duration predictor. The Transformer is deep and multiple target-to-source length-ratio class labels are used to control target lengths. The variation predictor in FastSpeech2 is utilized to predict phoneme durations. To optimize the isochrony in dubbing, re-ranking and scaling are performed. The source audio duration is used as a reference to re-rank the translations of different length-ratio labels, and the one with minimum time deviation is preferred. Additionally, the phoneme duration outputs are scaled within a defined threshold to narrow the duration gap with the source audio.",
}

This paper presents the submission of Huawei Translation Services Center for the IWSLT 2023 dubbing task in the unconstrained setting. The proposed solution consists of a Transformer-based machine translation model and a phoneme duration predictor. The Transformer is deep and multiple target-to-source length-ratio class labels are used to control target lengths. The variation predictor in FastSpeech2 is utilized to predict phoneme durations. To optimize the isochrony in dubbing, re-ranking and scaling are performed. The source audio duration is used as a reference to re-rank the translations of different length-ratio labels, and the one with minimum time deviation is preferred. Additionally, the phoneme duration outputs are scaled within a defined threshold to narrow the duration gap with the source audio.
Rao, Zhiqiang Shang, Hengchao Yang, Jinlong Wei, Daimeng Li, Zongyao Guo, Jiaxin Li, Shaojun Yu, Zhengzhe Wu, Zhanglin Xie, Yuhao Wei, Bin Zheng, Jiawei Lei, Lizhi Yang, Hao
Length-Aware NMT and Adaptive Duration for Automatic Dubbing