11390137
A modified infomax algorithm for blind signal separation
Abstract We present a new algorithm to perform blind signal separation (BSS), which takes a trade-off between the ordinary gradient infomax algorithm and the natural gradient infomax algorithm. Analyzing the algorithm, we show that desired equilibrium points are locally stable by choosing appropriate score functions and step sizes. The algorithm provides better performance than the ordinary gradient algorithm, and it is free from approximation error and the small-step-size restriction of the natural gradient algorithm. In simulations on convolved mixtures, the algorithm provides much better performance than the other algorithms while requiring less computation.
22839377
Training Algorithm with Incomplete Data for Feed-Forward Neural Networks
A new algorithm is developed to train feed-forward neural networks for non-linear input-to-output mappings with small incomplete data in arbitrary distributions. The developed Training-EStimation-Training (TEST) algorithm consists of 3 steps, i.e., (1) training with the complete portion of the training data set, (2) estimation of the missing attributes with the trained neural networks, and (3) re-training the neural networks with the whole data set. Error back propagation is still applicable to estimate the missing attributes. Unlike other training methods with missing data, it does not assume data distribution models which may not be appropriate for small training data. The developed TEST algorithm is first tested for the Iris benchmark data. By randomly removing some attributes from the complete data set and estimating the values latter, accuracy of the TEST algorithm is demonstrated. Then it is applied to the Diabetes benchmark data, of which about 50% contains missing attributes. Compared with other existing algorithms, the proposed TEST algorithm results in much better recognition accuracy for test data.
8239327
A filter bank approach to independent component analysis for convolved mixtures
Abstract We present a filter bank approach to perform independent component analysis (ICA) for convolved mixtures. Input signals are split into subband signals and subsampled. A simplified network performs ICA on the subsampled signals, and finally independent components are synthesized. The proposed approach achieves superior performance than the frequency domain approach and faster convergence with less computational complexity than the time domain approach. Furthermore, it requires shorter unmixing filter length and less computational complexity than other filter bank approaches by designing efficient filter banks. Also, a method is proposed to resolve the permutation and scaling problems of the filter bank approach.
40268280
Merging Back-propagation and Hebbian Learning Rules for Robust Classifications
Abstract By imposing saturation requirements on hidden-layer neural activations, a new learning algorithm is developed to improve robustness on classification performance of a multi-layer Perceptron. Derivatives of the sigmoid functions at hidden-layers are added to the standard output error with relative significance factors, and the total error is minimized by the steepest-descent method. The additional gradient-descent terms become Hebbian, and this new algorithm merges two popular learning algorithms, i.e., error back-propagation and Hebbian learning rules. Only slight modifications are needed for the standard back-propagation algorithm, and additional computational requirements are negligible. This saturation requirement effectively reduces output sensitivity to the input, which results in improved robustness and better generalization for classifier networks. Also distributed representations at hidden-layers are successfully suppressed to accomplish efficient utilization of hidden neurons. Computer simulations demonstrates much faster learning convergence as well as improved robustness for classifications and hetero-associations of binary patterns. Copyright © 1996 Elsevier Science Ltd
14142156
Out-of-Vocabulary Rejection based on Selective Attention Model
Based on a psychological selective attention theory a new algorithm is developed to provide reliable out-of-vocabulary (OOV) rejection for speech recognition systems in noisy environments. The developed attention model is based on Broadbent's ‘early filtering’ theory, and the attention adaptation process utilizes a gradient-descent error minimization algorithm with error backpropagation rule. The developed model is applied to isolated-word recognition tasks, and much higher in-vocabulary recognition rates are achieved with the same out-of-vocabulary rejection rates.
43679766
Blind signal separation and independent component analysis
Independent component analysis (ICA) is emerging as a new paradigm in signal processing and data analysis. ICA has been proposed as a solution to the blind source separation problem (BSS) in which sensory signals are given as mixtures of unknown source signals. Since neither the sources nor the mixing environment are known a priori there is no general analytical solution to this problem. However, since the last decade researchers have found good approximate solutions based on assumptions about the source statistics. They were able to formulate solutions or algorithms that were derived from well-known cost functions including maximum likelihood estimation, information maximization and higher-order cumulants. The ICA framework touches many theoretical research areas, and therefore receives attention in several research communities including machine learning, neural networks, statistical signal processing and Bayesian modeling. More recently, there have been numerous applications of ICA in adaptive 0ltering, speech signal processing, biomedical signal processing, computational neuroscience, image coding, text data modeling, and 0nancial data analysis. The standard ICA formulation has, in general, strong assumptions or simpli0cations such as: the number of sensors has to be equal or greater than the number of sources, there is no additive noise signal in the sensors, and the sources are modeled as random variables. However, when applying ICA to real-world problems it becomes evident that those restrictions are cumbersome and although the standard ICA algorithms give good approximate solutions, there is great need in going beyond the standard assumptions and develop new algorithms that can relax some of the assumptions. This special issue on ICA and BSS is a timely publication covering important recent 0ndings in ICA theory as well as new applications of ICA in data analysis. Leading researchers present their work to advance this 0eld beyond traditional borders and show how this 0eld can bridge disciplines and yield new methods that lead to mutual bene0ts and advances in science and engineering. This issue is composed of a review article and two parts: the 0rst part presents new theoretical 0ndings while the second part presents novel applications. Often, there is no such sharp division into these two parts since illustrative simulations support theoretical 0ndings and practical methods include algorithmic modi0cations to design and evaluate the application-speci0c methods. In “Frontiers of research in BSS=ICA”, V. David S: anchez A. formulates some of the basic concepts and problems in the areas of blind signal separation and independent component analysis. He then moves to a concise description of key methods of data preprocessing and algorithms that provide solutions to the instantaneous mixture and the
9536
Current practices of leading e-government countries
IT is transforming the way governments function and valuable lessons can be learned from the pioneering e-government programs that have led the charge.
18759724
An Embedding Framework for Myocardial Velocity Processing with MRI
This paper presents an embedding framework for myocardial velocity processing with MRI based on a hollow, semi-spherical template. The relationship between vector bundle and manifold mapping is analysed and three different mapping methods that include tangent mapping, normal mapping, and normal-tangent mapping are assessed for their practical value of myocardial contractility analysis. The proposed method provides a basis for consistent volume matching and vector correspondence, in addition to the ease of calculating biomechanical indices such as radial, circumferential and longitudinal strain rates without the concern of boundary effects. Detailed analysis results with both synthetic and in vivo MR velocity data sets are provided.
55517887
Creativity in Digital Libraries and Information Retrieval Environments
Inspired by Weisberg’s [20] argument that creativity exists in everyone and on the assumption that every information seeker is creative and undergoes a creative process in information seeking, this paper synthesizes a holistic “creativity” model and “information seeking” model to identify stages in a creative process for information seeking. A pilot study was conducted to verify these stages and also examine if present digital libraries and information retrieval environments provide “creative” features to support creative information seeking. We discussed these initial findings and made recommendations for further work to better support creative information seeking in digital libraries and information retrieval environments.
15590392
Performance-guided neural network for rapidly self-organising active network management
We present a neural network for real-time learning and mapping of patterns using an external performance indicator. In a non-stationary environment where new patterns are introduced over time, the learning process utilises a novel snap-drift algorithm that performs fast, convergent, minimalist learning (snap) when the overall network performance is poor and slower, more cautious learning (drift) when the performance is good. Snap is based on a modified form of Adaptive Resonance Theory (CGIP 37(1987)54); and drift is based on Learning Vector Quantization (LVQ) (Proc. IJCNN 1(1990a)545). The two are combined within a semi-supervised learning system that shifts its learning style whenever it receives a significant change in performance feedback. The learning is capable of rapid re-learning and re-stabilisation, according to changes in external feedback or input patterns. We have incorporated this algorithm into the design of a modular neural network system, Performance-guided Adaptive Resonance Theory (P-ART) (Proc. IJCNN 2(2003)1412; Soft computing systems: Design, Management and application, IOS Press, Netherland, 2002; pp. 21-31). Simulation results show that the system discovers alternative solutions in response to significant changes in the input patterns and/or in the environment, which may require similar patterns to be treated differently over time. The simulations involve attempting to optimise the selection of network services in a non-stationary, real-time active computer network environment, in which the factors influencing the required selections are subject to change.
5697162
Efficient L1 Regularized Logistic Regression
L1 regularized logistic regression is now a workhorse of machine learning: it is widely used for many classification problems, particularly ones with many features. L1 regularized logistic regression requires solving a convex optimization problem. However, standard algorithms for solving convex optimization problems do not scale well enough to handle the large datasets encountered in many practical settings. In this paper, we propose an efficient algorithm for L1 regularized logistic regression. Our algorithm iteratively approximates the objective function by a quadratic approximation at the current point, while maintaining the L1 constraint. In each iteration, it uses the efficient LARS (Least Angle Regression) algorithm to solve the resulting L1 constrained quadratic optimization problem. Our theoretical results show that our algorithm is guaranteed to converge to the global optimum. Our experiments show that our algorithm significantly outperforms standard algorithms for solving convex optimization problems. Moreover, our algorithm outperforms four previously published algorithms that were specifically designed to solve the L1 regularized logistic regression problem.