grapheme to phoneme
220284805
The SIGMORPHON 2020 Shared Task on Multilingual Grapheme to Phoneme Conversion
We describe the design and findings of the SIGMORPHON 2020 shared task on multilingual grapheme to phoneme conversion. Participants were asked to submit systems which take in a sequence of graphemes in a given language as input, then output a sequence of phonemes representing the pronunciation of that grapheme sequence. Nine teams submitted a total of 23 systems, at best achieving a 18% relative reduction in word error rate (macro averaged over languages) versus strong neural sequence to sequence baselines. To facilitate error analysis, we publicly release the complete outputs for all systems a first for the SIGMORPHON workshop.
SIGMORPHON
Kyle Gorman Lucas F E Ashby Aaron Goyzueta Arya D McCarthy Shijie Wu Daniel You
214698711
Multilingual Grapheme To Phoneme Conversion with Byte Representation
Grapheme to phoneme (G2P) models convert a written word into its corresponding pronunciation and are essential components in automatic speech recognition and text to speech systems. Recently, the use of neural encoder decoder architectures has substantially improved G2P accuracy for mono and multi lingual cases. However, most multilingual G2P studies focus on sets of languages that share similar graphemes, such as European languages. Multilingual G2P for languages from different writing systems, e.g. European and East Asian, remains an understudied area. In this work, we propose a multilingual G2P model with byte level input representation to accommodate different grapheme systems, along with an attention based Transformer architecture. We evaluate the performance of both character level and byte level G2P using data from multiple European and East Asian locales. Models using byte representation yield 16.2% 50.2% relative word error rate improvement over character based counterparts for mono and multi lingual use cases. In addition, byte level models are 15.0% 20.1% smaller in size. Our results show that byte is an efficient representation for multilingual G2P with languages having large grapheme vocabularies.
ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Mingzhi Yu Hieu Duy Nguyen Alex Sokolov Jack Lepird Kanthashree Mysore Sathyendra Samridhi Choudhary Athanasios Mouchtaris Siegfried Kunzmann
102352294
Token Level Ensemble Distillation for Grapheme to Phoneme Conversion
Grapheme to phoneme (G2P) conversion is an important task in automatic speech recognition and text to speech systems. Recently, G2P conversion is viewed as a sequence to sequence task and modeled by RNN or CNN based encoder decoder framework. However, previous works do not consider the practical issues when deploying G2P model in the production system, such as how to leverage additional unlabeled data to boost the accuracy, as well as reduce model size for online deployment. In this work, we propose token level ensemble distillation for G2P conversion, which can (1) boost the accuracy by distilling the knowledge from additional unlabeled data, and (2) reduce the model size but maintain the high accuracy, both of which are very practical and helpful in the online production system. We use token level knowledge distillation, which results in better accuracy than the sequence level counterpart. What is more, we adopt the Transformer instead of RNN or CNN based models to further boost the accuracy of G2P conversion. Experiments on the publicly available CMUDict dataset and an internal English dataset demonstrate the effectiveness of our proposed method. Particularly, our method achieves 19.88% WER on CMUDict dataset, outperforming the previous works by more than 4.22% WER, and setting the new state of the art results.
INTERSPEECH
Hao Sun Xu Tan Jun-Wei Gan Hongzhi Liu Sheng Zhao Tao Qin Tie-Yan Liu
202754247
Transformer based Grapheme to Phoneme Conversion
Attention mechanism is one of the most successful techniques in deep learning based Natural Language Processing (NLP) The transformer network architecture is completely based on attention mechanisms, and it outperforms sequence to sequence models in neural machine translation without recurrent and convolutional layers. Grapheme to phoneme (G2P) conversion is a task of converting letters (grapheme sequence) to their pronunciations (phoneme sequence) It plays a significant role in text to speech (TTS) and automatic speech recognition (ASR) systems. In this paper, we investigate the application of transformer architecture to G2P conversion and compare its performance with recurrent and convolutional neural network based approaches. Phoneme and word error rates are evaluated on the CMUDict dataset for US English and the NetTalk dataset. The results show that transformer based G2P outperforms the convolutional based approach in terms of word error rate and our results significantly exceeded previous recurrent approaches (without attention) regarding word and phoneme error rates on both datasets. Furthermore, the size of the proposed model is much smaller than the size of the previous approaches.
INTERSPEECH
Sevinj Yolchuyeva Geza Nemeth Balint Gyires-Toth
117732095
Grapheme to Phoneme Conversion with Convolutional Neural Networks
Grapheme to phoneme (G2P) conversion is the process of generating pronunciation for words based on their written form. It has a highly essential role for natural language processing, text to speech synthesis and automatic speech recognition systems. In this paper, we investigate convolutional neural networks (CNN) for G2P conversion. We propose a novel CNN based sequence to sequence (seq2seq) architecture for G2P conversion. Our approach includes an end to end CNN G2P conversion with residual connections and, furthermore, a model that utilizes a convolutional neural network (with and without residual connections) as encoder and Bi LSTM as a decoder. We compare our approach with state of the art methods, including Encoder Decoder LSTM and Encoder Decoder Bi LSTM. Training and inference times, phoneme and word error rates were evaluated on the public CMUDict dataset for US English, and the best performing convolutional neural network based architecture was also evaluated on the NetTalk dataset. Our method approaches the accuracy of previous state of the art results in terms of phoneme error rate.
Applied Sciences
Sevinj Yolchuyeva Geza Nemeth Balint Gyires-Toth
202740855
Neural Machine Translation for Multilingual Grapheme to Phoneme Conversion
Grapheme to phoneme (G2P) models are a key component in Automatic Speech Recognition (ASR) systems, such as the ASR system in Alexa, as they are used to generate pronunciations for out of vocabulary words that do not exist in the pronunciation lexicons (mappings like "e c h o" to "E k oU" Most G2P systems are monolingual and based on traditional joint sequence based n gram models [1,2] As an alternative, we present a single end to end trained neural G2P model that shares same encoder and decoder across multiple languages. This allows the model to utilize a combination of universal symbol inventories of Latin like alphabets and cross linguistically shared feature representations. Such model is especially useful in the scenarios of low resource languages and code switching/foreign words, where the pronunciations in one language need to be adapted to other locales or accents. We further experiment with word language distribution vector as an additional training target in order to improve system performance by helping the model decouple pronunciations across a variety of languages in the parameter space. We show 7.2% average improvement in phoneme error rate over low resource languages and no degradation over high resource ones compared to monolingual baselines.
INTERSPEECH
Alex Sokolov Tracy M Rohlin Ariya Rastrow
164650662
Incorporating syllabification points into a model of grapheme to phoneme conversion
A model to convert a grapheme into a phoneme (G2P) is crucial in the natural language processing area. In general, it is developed using a probabilistic based data driven approach and directly applied to a sequence of graphemes with no other information. Important research shows that incorporating information of syllabification point is capable of improving a probabilistic based English G2P. However, the information should be accurately provided by a perfect orthographic syllabification. Some noises or errors of syllabification significantly reduce the G2P performance. In this paper, incorporation of syllabification points into a probabilistic based G2P model for Bahasa Indonesia is investigated. This information is important since Bahasa Indonesia is richer than English in terms of syllables. A 5 fold cross validating on 50 k words shows that the incorporation of syllabification points significantly improves the performance of G2P model, where the phoneme error rate (PER) can be relatively reduced by 10.75% This PER is much lower than the G2P model based on an inductive learning algorithm. An important contribution of this research is that the proposed G2P model is quite robust to syllabification errors. A syllable error rate (SER) of 2.5% that comes from an orthographic syllabification model just slightly increases the PER of the proposed G2P model from 0.83% to be 0.90% A higher SER up to 10% just increase the PER to be 1.14%
Int. J. Speech Technol.
Suyanto Suyanto
52285649
Convolutional Sequence to Sequence Model with Non Sequential Greedy Decoding for Grapheme to Phoneme Conversion
The greedy decoding method used in the conventional sequence to sequence models is prone to producing a model with a compounding of errors, mainly because it makes inferences in a fixed order, regardless of whether or not the model's previous guesses are correct. We propose a non sequential greedy decoding method that generalizes the greedy decoding schemes proposed in the past. The proposed method determines not only which token to consider, but also which position in the output sequence to infer at each inference step. Specifically, it allows the model to consider easy parts first, helping the model infer hard parts more easily later by providing more information. We study a grapheme to phoneme conversion task with a fully convolutional encoder decoder model that embeds the proposed decoding method. Experiment results show that our model shows better performance than that of the state of the art model in terms of both phoneme error rate and word error rate.
2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Moon-Jung Chae Kyubyong Park Linhyun Bang Soobin Suh Longhyuk Park Namju Kim Jonghun Park
52188647
Multilingual Grapheme to Phoneme Conversion with Global Character Vectors
Multilingual grapheme to phoneme (G2P) models are useful for multilingual speech synthesis because one model simultaneously copes with multilingual words. We propose a G2P model that combines global character vectors (GCVs) with bidirectional recurrent neural networks (BRNNs) and enables the direct conversion of text (as a sequence of characters) to pronunciation. GCVs are distributional, real valued representations of characters and their contextual interactions that can be learned from a large scale text corpus in an unsupervised manner. With the flexibility of learning GCVs from plain text resources, this method has an advantage: it enables monolingual G2P (MoG2P) and multilingual G2P (MuG2P) conversion. We experiment in four languages (Japanese, Korean, Thai, and Chinese) with learning language dependent (LD) and language independent (LI) GCVs and then build MoG2P and MuG2P models with two hidden layer BRNNs. Our results show that both LDand LI GCV based MoG2P models, whose performances are equivalent, achieved better than 97.7% syllable accuracy, which is a relative improvement from 27% to 90% depending on the language in comparison with Mecab based models. As for MuG2P, the accuracy is around 98% which is a slightly degraded performance compared to MoG2P. The proposed method also has the potential of the G2P conversion of non normalized words, achieving 80% accuracy in Japanese.
INTERSPEECH
Jinfu Ni Yoshinori Shiga Hisashi Kawai
21840890
Massively Multilingual Neural Grapheme to Phoneme Conversion
Grapheme to phoneme conversion (g2p) is necessary for text to speech and automatic speech recognition systems. Most g2p systems are monolingual: they require language specific data or handcrafting of rules. Such systems are difficult to extend to low resource languages, for which data and handcrafted rules are not available. As an alternative, we present a neural sequence to sequence approach to g2p which is trained on spelling pronunciation pairs in hundreds of languages. The system shares a single encoder and decoder across all languages, allowing it to utilize the intrinsic similarities between different writing systems. We show an 11% improvement in phoneme error rate over an approach based on adapting high resource monolingual g2p models to low resource languages. Our model is also much more compact relative to previous approaches.
ArXiv
Ben Peters Jon Dehdari Josef van Genabith