19651025
A new neuro-fuzzy identification model of nonlinear dynamic systems
Abstract Multilayer neural networks with error back-propagation learning algorithms have the capability of learning an arbitrary continuous nonlinear function with examples of input and output sample pairs and a great potential for identifying nonlinear dynamic systems with unknown characteristics. A fuzzy system is composed of fuzzification of input, reasoning (or inference) by fuzzy rules, and defuzzification of fuzzy output. In general, there are some difficulties in finding suitable fuzzification and defuzzification methods and fuzzy rules. Formation of fuzzy rules with complex input-output relationships can be replaced by building neural networks with input and output sample pairs. A neuro-fuzzy identifier is proposed to have a cascaded structure of fuzzification, neural network, and defuzzification and additionally is shown to be able to compensate for fuzzification and defuzzification error of fuzzy logic. Computer simulation shows that neuro-fuzzy identification is very effective in modeling the fuzzy system whose fuzzy rules can not be obtained easily.
22839377
Training Algorithm with Incomplete Data for Feed-Forward Neural Networks
A new algorithm is developed to train feed-forward neural networks for non-linear input-to-output mappings with small incomplete data in arbitrary distributions. The developed Training-EStimation-Training (TEST) algorithm consists of 3 steps, i.e., (1) training with the complete portion of the training data set, (2) estimation of the missing attributes with the trained neural networks, and (3) re-training the neural networks with the whole data set. Error back propagation is still applicable to estimate the missing attributes. Unlike other training methods with missing data, it does not assume data distribution models which may not be appropriate for small training data. The developed TEST algorithm is first tested for the Iris benchmark data. By randomly removing some attributes from the complete data set and estimating the values latter, accuracy of the TEST algorithm is demonstrated. Then it is applied to the Diabetes benchmark data, of which about 50% contains missing attributes. Compared with other existing algorithms, the proposed TEST algorithm results in much better recognition accuracy for test data.
11390137
A modified infomax algorithm for blind signal separation
Abstract We present a new algorithm to perform blind signal separation (BSS), which takes a trade-off between the ordinary gradient infomax algorithm and the natural gradient infomax algorithm. Analyzing the algorithm, we show that desired equilibrium points are locally stable by choosing appropriate score functions and step sizes. The algorithm provides better performance than the ordinary gradient algorithm, and it is free from approximation error and the small-step-size restriction of the natural gradient algorithm. In simulations on convolved mixtures, the algorithm provides much better performance than the other algorithms while requiring less computation.
40268280
Merging Back-propagation and Hebbian Learning Rules for Robust Classifications
Abstract By imposing saturation requirements on hidden-layer neural activations, a new learning algorithm is developed to improve robustness on classification performance of a multi-layer Perceptron. Derivatives of the sigmoid functions at hidden-layers are added to the standard output error with relative significance factors, and the total error is minimized by the steepest-descent method. The additional gradient-descent terms become Hebbian, and this new algorithm merges two popular learning algorithms, i.e., error back-propagation and Hebbian learning rules. Only slight modifications are needed for the standard back-propagation algorithm, and additional computational requirements are negligible. This saturation requirement effectively reduces output sensitivity to the input, which results in improved robustness and better generalization for classifier networks. Also distributed representations at hidden-layers are successfully suppressed to accomplish efficient utilization of hidden neurons. Computer simulations demonstrates much faster learning convergence as well as improved robustness for classifications and hetero-associations of binary patterns. Copyright © 1996 Elsevier Science Ltd
8960700
On the Efficient Speech Feature Extraction Based on Independent Component Analysis
A new efficient code for speech signals is proposed. To represent speech signals with minimum redundancy we use independent component analysis to adapt features (basis vectors) that efficiently encode the speech signals. The learned basis vectors are sparsely distributed and localized in both time and frequency. Time-frequency analysis of basis vectors shows the property similar with the critical bandwidth of human auditory system. Our results suggest that the obtained codes of speech signals are sparse and biologically plausible.
14142156
Out-of-Vocabulary Rejection based on Selective Attention Model
Based on a psychological selective attention theory a new algorithm is developed to provide reliable out-of-vocabulary (OOV) rejection for speech recognition systems in noisy environments. The developed attention model is based on Broadbent's ‘early filtering’ theory, and the attention adaptation process utilizes a gradient-descent error minimization algorithm with error backpropagation rule. The developed model is applied to isolated-word recognition tasks, and much higher in-vocabulary recognition rates are achieved with the same out-of-vocabulary rejection rates.
5485745
Performance-guided Neural Network for Self-Organising Network Management
A neural network architecture is introduced for real-time learning of input sequences 
using external performance feedback. Some aspects of Adaptive Resonance Theory (ART) 
networks [1] are applied because they are able to function in a fast real-time adaptive active network environment where user requests and new proxylets (services) are constantly being introduced over time [2,3]. The architecture learns, self-organis es and self-stabilises in response to user requests, mapping the requests according to the types of proxylets available. However, in 
order make the neural networks respond to performance feedback, we introduce a modification to the original ART1 network in the form of the ‘snap-drift’ algorithm, that uses fast convergent, minimalist learning (snap) when the overall network performance is poor, and slow learning (drift towards user request input pattern) when the performance is good. Preliminary simulations 
evaluate the two-tiered architecture using a simple operating environment consisting of simulated training and test data.
17934021
Fast determination of textural periodicity using distance matching function
The periodicity of a texture is one of its important visual characteristics. The inertias of co-occurrence matrices of the texture have been often used to detect the visual periodicity. However, it is time-consuming to explicitly construct these matrices. In this paper, we propose the distance matching function to avoid constructing the matrices due to our new interpretation of an inertia. For a texture of size m n, the inertias of all co-occurrence matrices can be obtained in OOmn log mnU time by simultaneously evaluating the function at all displacement vectors. This is a significant improvement over the previous method using the co-occurrence matrices, that requires OOm 2 n 2 U time. ” 1999 Elsevier Science B.V. All rights reserved.
15590392
Performance-guided neural network for rapidly self-organising active network management
We present a neural network for real-time learning and mapping of patterns using an external performance indicator. In a non-stationary environment where new patterns are introduced over time, the learning process utilises a novel snap-drift algorithm that performs fast, convergent, minimalist learning (snap) when the overall network performance is poor and slower, more cautious learning (drift) when the performance is good. Snap is based on a modified form of Adaptive Resonance Theory (CGIP 37(1987)54); and drift is based on Learning Vector Quantization (LVQ) (Proc. IJCNN 1(1990a)545). The two are combined within a semi-supervised learning system that shifts its learning style whenever it receives a significant change in performance feedback. The learning is capable of rapid re-learning and re-stabilisation, according to changes in external feedback or input patterns. We have incorporated this algorithm into the design of a modular neural network system, Performance-guided Adaptive Resonance Theory (P-ART) (Proc. IJCNN 2(2003)1412; Soft computing systems: Design, Management and application, IOS Press, Netherland, 2002; pp. 21-31). Simulation results show that the system discovers alternative solutions in response to significant changes in the input patterns and/or in the environment, which may require similar patterns to be treated differently over time. The simulations involve attempting to optimise the selection of network services in a non-stationary, real-time active computer network environment, in which the factors influencing the required selections are subject to change.
9536
Current practices of leading e-government countries
IT is transforming the way governments function and valuable lessons can be learned from the pioneering e-government programs that have led the charge.
14828160
Maximum likelihood estimation and model comparison of nonlinear structural equation models with continuous and polytomous variables
Abstract Recently, it is recognized that nonlinear relationships among latent variables in a structural equation model are important. In this article, maximum likelihood (ML) analysis of a general nonlinear structural equation model that contains fixed covariates in the measurement equation and the nonlinear structural equation is investigated. A MCEM algorithm is implemented to obtain the ML estimates, in which the E-step is completed with the help of a hybrid algorithm that combines the Gibbs sampler and the Metropolis–Hastings algorithm whilst the M-step is completed by conditional maximization. The importance sampling is employed to compute the observed-data likelihood in the Bayesian Information Criterion for model comparison. The methodology is illustrated with a simulation study and a real example.