40268280
Merging Back-propagation and Hebbian Learning Rules for Robust Classifications
Abstract By imposing saturation requirements on hidden-layer neural activations, a new learning algorithm is developed to improve robustness on classification performance of a multi-layer Perceptron. Derivatives of the sigmoid functions at hidden-layers are added to the standard output error with relative significance factors, and the total error is minimized by the steepest-descent method. The additional gradient-descent terms become Hebbian, and this new algorithm merges two popular learning algorithms, i.e., error back-propagation and Hebbian learning rules. Only slight modifications are needed for the standard back-propagation algorithm, and additional computational requirements are negligible. This saturation requirement effectively reduces output sensitivity to the input, which results in improved robustness and better generalization for classifier networks. Also distributed representations at hidden-layers are successfully suppressed to accomplish efficient utilization of hidden neurons. Computer simulations demonstrates much faster learning convergence as well as improved robustness for classifications and hetero-associations of binary patterns. Copyright © 1996 Elsevier Science Ltd
8960700
On the Efficient Speech Feature Extraction Based on Independent Component Analysis
A new efficient code for speech signals is proposed. To represent speech signals with minimum redundancy we use independent component analysis to adapt features (basis vectors) that efficiently encode the speech signals. The learned basis vectors are sparsely distributed and localized in both time and frequency. Time-frequency analysis of basis vectors shows the property similar with the critical bandwidth of human auditory system. Our results suggest that the obtained codes of speech signals are sparse and biologically plausible.
43679766
Blind signal separation and independent component analysis
Independent component analysis (ICA) is emerging as a new paradigm in signal processing and data analysis. ICA has been proposed as a solution to the blind source separation problem (BSS) in which sensory signals are given as mixtures of unknown source signals. Since neither the sources nor the mixing environment are known a priori there is no general analytical solution to this problem. However, since the last decade researchers have found good approximate solutions based on assumptions about the source statistics. They were able to formulate solutions or algorithms that were derived from well-known cost functions including maximum likelihood estimation, information maximization and higher-order cumulants. The ICA framework touches many theoretical research areas, and therefore receives attention in several research communities including machine learning, neural networks, statistical signal processing and Bayesian modeling. More recently, there have been numerous applications of ICA in adaptive 0ltering, speech signal processing, biomedical signal processing, computational neuroscience, image coding, text data modeling, and 0nancial data analysis. The standard ICA formulation has, in general, strong assumptions or simpli0cations such as: the number of sensors has to be equal or greater than the number of sources, there is no additive noise signal in the sensors, and the sources are modeled as random variables. However, when applying ICA to real-world problems it becomes evident that those restrictions are cumbersome and although the standard ICA algorithms give good approximate solutions, there is great need in going beyond the standard assumptions and develop new algorithms that can relax some of the assumptions. This special issue on ICA and BSS is a timely publication covering important recent 0ndings in ICA theory as well as new applications of ICA in data analysis. Leading researchers present their work to advance this 0eld beyond traditional borders and show how this 0eld can bridge disciplines and yield new methods that lead to mutual bene0ts and advances in science and engineering. This issue is composed of a review article and two parts: the 0rst part presents new theoretical 0ndings while the second part presents novel applications. Often, there is no such sharp division into these two parts since illustrative simulations support theoretical 0ndings and practical methods include algorithmic modi0cations to design and evaluate the application-speci0c methods. In “Frontiers of research in BSS=ICA”, V. David S: anchez A. formulates some of the basic concepts and problems in the areas of blind signal separation and independent component analysis. He then moves to a concise description of key methods of data preprocessing and algorithms that provide solutions to the instantaneous mixture and the
14142156
Out-of-Vocabulary Rejection based on Selective Attention Model
Based on a psychological selective attention theory a new algorithm is developed to provide reliable out-of-vocabulary (OOV) rejection for speech recognition systems in noisy environments. The developed attention model is based on Broadbent's ‘early filtering’ theory, and the attention adaptation process utilizes a gradient-descent error minimization algorithm with error backpropagation rule. The developed model is applied to isolated-word recognition tasks, and much higher in-vocabulary recognition rates are achieved with the same out-of-vocabulary rejection rates.
22839377
Training Algorithm with Incomplete Data for Feed-Forward Neural Networks
A new algorithm is developed to train feed-forward neural networks for non-linear input-to-output mappings with small incomplete data in arbitrary distributions. The developed Training-EStimation-Training (TEST) algorithm consists of 3 steps, i.e., (1) training with the complete portion of the training data set, (2) estimation of the missing attributes with the trained neural networks, and (3) re-training the neural networks with the whole data set. Error back propagation is still applicable to estimate the missing attributes. Unlike other training methods with missing data, it does not assume data distribution models which may not be appropriate for small training data. The developed TEST algorithm is first tested for the Iris benchmark data. By randomly removing some attributes from the complete data set and estimating the values latter, accuracy of the TEST algorithm is demonstrated. Then it is applied to the Diabetes benchmark data, of which about 50% contains missing attributes. Compared with other existing algorithms, the proposed TEST algorithm results in much better recognition accuracy for test data.
8239327
A filter bank approach to independent component analysis for convolved mixtures
Abstract We present a filter bank approach to perform independent component analysis (ICA) for convolved mixtures. Input signals are split into subband signals and subsampled. A simplified network performs ICA on the subsampled signals, and finally independent components are synthesized. The proposed approach achieves superior performance than the frequency domain approach and faster convergence with less computational complexity than the time domain approach. Furthermore, it requires shorter unmixing filter length and less computational complexity than other filter bank approaches by designing efficient filter banks. Also, a method is proposed to resolve the permutation and scaling problems of the filter bank approach.
933849
Scalability study of the ad hoc on-demand distance vector routing protocol
As mobile networking continues to experience increasing popularity, the need to connect large numbers of wireless devices will become more prevalent. Many recent proposals for ad hoc routing have certain characteristics that may limit their scalability to large networks. This paper examines five different combinations of modifications that may be incorporated into virtually any on-demand protocol in order to improve its scalability. The scalability of current on-demand routing protocols is evaluated through the selection of a representative from this class of protocols. The performance of the un-modified on-demand protocol is compared against that of it combined with each of the scalability modifications. Each scheme's behavior is analyzed in networks as large as 10,000 nodes through detailed simulation. Based on the observations, conclusions are drawn as to the expected scalability improvement that can be achieved by each modification.
27705819
The effect of negative buyer feedback on prices in Internet auction markets
15590392
Performance-guided neural network for rapidly self-organising active network management
We present a neural network for real-time learning and mapping of patterns using an external performance indicator. In a non-stationary environment where new patterns are introduced over time, the learning process utilises a novel snap-drift algorithm that performs fast, convergent, minimalist learning (snap) when the overall network performance is poor and slower, more cautious learning (drift) when the performance is good. Snap is based on a modified form of Adaptive Resonance Theory (CGIP 37(1987)54); and drift is based on Learning Vector Quantization (LVQ) (Proc. IJCNN 1(1990a)545). The two are combined within a semi-supervised learning system that shifts its learning style whenever it receives a significant change in performance feedback. The learning is capable of rapid re-learning and re-stabilisation, according to changes in external feedback or input patterns. We have incorporated this algorithm into the design of a modular neural network system, Performance-guided Adaptive Resonance Theory (P-ART) (Proc. IJCNN 2(2003)1412; Soft computing systems: Design, Management and application, IOS Press, Netherland, 2002; pp. 21-31). Simulation results show that the system discovers alternative solutions in response to significant changes in the input patterns and/or in the environment, which may require similar patterns to be treated differently over time. The simulations involve attempting to optimise the selection of network services in a non-stationary, real-time active computer network environment, in which the factors influencing the required selections are subject to change.
1670024
Scattered Data Interpolation with Multilevel B-Splines
The paper describes a fast algorithm for scattered data interpolation and approximation. Multilevel B-splines are introduced to compute a C/sup 2/ continuous surface through a set of irregularly spaced points. The algorithm makes use of a coarse to fine hierarchy of control lattices to generate a sequence of bicubic B-spline functions whose sum approaches the desired interpolation function. Large performance gains are realized by using B-spline refinement to reduce the sum of these functions into one equivalent B-spline function. Experimental results demonstrate that high fidelity reconstruction is possible from a selected set of sparse and irregular samples.
17250597
Distributed similarity search algorithm in distributed heterogeneous multimedia databases
The collection fusion problem in multimedia databases is concerned with the merging of results retrieved by content based retrieval from distributed heterogeneous multimedia databases in order to find the most similar objects to a query object. We propose distributed similarity search algorithms, two heuristic algorithms and an algorithm using the linear regression, to solve this problem. To our knowledge, these algorithms are the first research results in the area of distributed content based retrieval for heterogeneous multimedia databases.