A multi-feature fusion based confidence estimation model for Chinese end-to-end speech recognition

Speech recognition has shown excellent performance in recent years, but as we known, this kind of AI task is still supervised and needs labelled data to feed models so that it can achieve a better effect and to inference to get model’s metrics. In actual application scenarios, it always needs many people to check speech recognition’s correctness because of lacking labelled data, it is not realistic to label massive online audio. In addition, this method can’t guarantee comprehensiveness of checking, so confidence estimation algorithm is proposed, which can evaluate speech recognition model’s results and predict error transcriptions automatically. This paper proposed a confidence estimation model based on a multi-feature fusion mechanism and it mainly focus on Chinese end-to-end speech recognition tasks in complex application scenarios. Experiments in Aishell-1 dataset and China telecom’s internal dataset have proved that this model can obtain a good performance.