9084974
Isolated word recognition using modular recurrent neural networks
Abstract This paper describes a novel method of using recurrent neural networks (RNN) for isolated word recognition. Each word in the target vocabulary is modeled by a fully connected recurrent network. To recognize an input utterance, the best matching word is determined based on its temporal output response. The system is trained in two stages. First, the RNN speech models (RSM) are trained independently to capture the essential static and temporal characteristics of individual words. This is performed by using an iterative re-segmentation training algorithm which gives the optimal phonetic segmentation automatically for each training utterance. The second-stage involves mutually discriminative training among the RSMs, aiming at minimizing the probability of misclassification. A series of simulation experiments have been performed to demonstrate the effectiveness of the proposed recognition method. For the recognition of (A) 20 English words, (B) 11 Cantonese digits and (C) 58 Cantonese CV syllables, the top-1 accuracy are 91.9, 93.6 and 87.1%, respectively.
2587142
Using tone information in Cantonese continuous speech recognition
In Chinese languages, tones carry important information at various linguistic levels. This research is based on the belief that tone information, if acquired accurately and utilized effectively, contributes to the automatic speech recognition of Chinese. In particular, we focus on the Cantonese dialect, which is spoken by tens of millions of people in Southern China and Hong Kong. Cantonese is well known for its complicated tone system, which makes automatic tone recognition very difficult. This article describes an effective approach to explicit tone recognition of Cantonese in continuously spoken utterances. Tone feature vectors are derived, on a short-time basis, to characterize the syllable-wide patterns of F0 (fundamental frequency) and energy movements. A moving-window normalization technique is proposed to reduce the tone-irrelevant fluctuation of F0 and energy features. Hidden Markov models are employed for context-dependent acoustic modeling of different tones. A tone recognition accuracy of 66.4% has been achieved in the speaker-independent case. The recognized tone patterns are then utilized to assist Cantonese large-vocabulary continuous speech recognition (LVCSR) via a lattice expansion approach. Experimental results show that reliable tone information helps to improve the overall performance of LVCSR.
42876940
An HMM-based speech recognition IC
This paper presents the design, simulation and measurement results of a Hidden Markov Model (HMM) based isolated word recognizer IC with double mixtures. Table look-up technique is employed in this design. The chip operates at 20 MHz at 3.3 V. The recognition time is 0.5 s for a 50-word speech library. The speech IC has been verified with 467 test speech data and the recognition accuracy is 93.8%. A reference software recognizer using the same algorithm and speech library has a recognition accuracy of 94.2%. The new speech IC that uses a table look up to reduce the complexity of the circuit has approximately the same recognition accuracy as an ideal software recognizer.
5832781
Adaptive contrast gain control and information maximization
Contrast gain control is an important mechanism underlying the visual system's adaptation to contrast of luminance in varying visual environments. Our previous work showed that the threshold and saturation determine the preferred contrast sensitivity as well as the maximum information coding capacity of the neuronal model. In this report, we investigated the design principles underlying adaptation behavior in contrast gain control by an adaptive linear-nonlinear model. We found that an adaptive rescaling mechanism predicted by information transmission maximization can explain a variety of observed contrast gain control phenomena in neurophysiological experiments, including the divisive input-output relations, and the inverse power-law relation between response gain and input contrast. Our results suggest that contrast gain control in visual systems might be designed for information maximization.
866852
Unsupervised Classification with Non-Gaussian Mixture Models Using ICA
We present an unsupervised classification algorithm based on an ICA mixture model. The ICA mixture model assumes that the observed data can be categorized into several mutually exclusive data classes in which the components in each class are generated by a linear mixture of independent sources. The algorithm finds the independent sources, the mixing matrix for each class and also computes the class membership probability for each data point. This approach extends the Gaussian mixture model so that the classes can have non-Gaussian structure. We demonstrate that this method can learn efficient codes to represent images of natural scenes and text. The learned classes of basis functions yield a better approximation of the underlying distributions of the data, and thus can provide greater coding efficiency. We believe that this method is well suited to modeling structure in high-dimensional data and has many potential applications.
207739442
Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources
An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub- and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to high-dimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.
207726294
Evolving better population distribution and exploration in evolutionary multi-objective optimization
The aim of multi-objective evolutionary optimization is to minimize the distance between the solution set and the true Pareto front, to distribute the solutions evenly and to maximize the spread of solution set. This paper addresses these issues by presenting two features that enhance the optimization ability of multi-objective evolutionary algorithms. The first feature is a variant of the mutation operator that adapts the mutation rate along the evolution process to maintain a balance between the introduction of diversity and local fine-tuning. In addition, this adaptive mutation operator adopts a new approach to strike a compromise between the preservation and disruption of genetic information. The second feature is an enhanced exploration strategy that encourages the exploration towards less populated areas and hence achieves better discovery of gaps in the generated front. The strategy also preserves non-dominated solutions in the evolving population to achieve a good convergence for the optimization. Comparative studies of some well-known diversity operators, mutation operators and multi-objective evolutionary algorithms are performed on different benchmark problems, which illustrate the effectiveness and efficiency of the proposed features.
5750510
Online Query Relaxation via Bayesian Causal Structures Discovery
We introduce a novel algorithm, TOQR, for relaxing failed queries over databases, that is, over-constrained DNF queries that return an empty result. TOQR uses a small dataset to discover the implicit relationships among the domain attributes, and then it exploits this domain knowledge to relax the failed query. TOQR starts with a relaxed query that does not include any constraint, and it tries to add to it as many as possible of the original constraints or their relaxations. The order in which the constraints are added is derived from the domain's causal structure, which is learned by applying the TAN algorithm to the small training dataset. Our experiments show that TOQR clearly outperforms other approaches: even when trained on a handful of examples, it successfully relaxes more that 97% of the failed queries; furthermore, TOQR'S relaxed queries are highly similar to the original failed query.