Personalization of CTC Speech Recognition Models

End-to-end speech recognition models trained using joint Connectionist Temporal Classiﬁcation (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to personalize because of their con-ditional independence assumption that prevents output tokens from previous time steps to inﬂuence future predictions. To tackle this, we propose a novel two-way approach that ﬁrst biases the encoder with attention over a predeﬁned list of rare long-tail and out-of-vocabulary (OOV) words and then uses dynamic boosting and phone alignment network during decoding to further bias the subword predictions. We evaluate our approach on open-source VoxPopuli and in-house medical datasets to showcase a 60% improvement in F1 score on domain-speciﬁc rare words over a strong CTC baseline. biasing and on-the-ﬂy decoder biasing techniques and compared their performance under two different data settings with base models trained on both large and small corpora. We demonstrated the efﬁcacy of our proposed approaches with signiﬁcant improvements in F1 score when biased with a large catalog. Our proposed model showed signiﬁcant improvements of 60.3% and 30% in relative F1 score on rare and OOV words respectively on the VoxPopuli dataset with negligible degradation (less than 0.5%) in WER on general utterances.