866852
Unsupervised Classification with Non-Gaussian Mixture Models Using ICA
We present an unsupervised classification algorithm based on an ICA mixture model. The ICA mixture model assumes that the observed data can be categorized into several mutually exclusive data classes in which the components in each class are generated by a linear mixture of independent sources. The algorithm finds the independent sources, the mixing matrix for each class and also computes the class membership probability for each data point. This approach extends the Gaussian mixture model so that the classes can have non-Gaussian structure. We demonstrate that this method can learn efficient codes to represent images of natural scenes and text. The learned classes of basis functions yield a better approximation of the underlying distributions of the data, and thus can provide greater coding efficiency. We believe that this method is well suited to modeling structure in high-dimensional data and has many potential applications.
767826
The Spectral Independent Components of Natural Scenes
We apply independent component analysis (ICA) for learning an efficient color image representation of natural scenes. In the spectra of single pixels, the algorithm was able to find basis functions that had a broadband spectrum similar to natural daylight, as well as basis functions that coincided with the human cone sensitivity response functions. When applied to small image patches, the algorithm found homogeneous basis functions, achromatic basis functions, and basis functions with overall chromatic variation along lines in color space. Our findings suggest that ICAm ay be used to reveal the structure of color information in natural images.
207739442
Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources
An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub- and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to high-dimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.
6741849
Blind Separation of Delayed and Convolved Sources
We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deconvolving mixed signals. While they work well on simulated data, these rules fail in real rooms which usually involve non-minimum phase transfer functions, not-invertible using stable IIR filters. An approach that sidesteps this problem is to perform infomax on a feedforward architecture in the frequency domain (Lambert 1996). We demonstrate real-room separation of two natural signals using this approach.
16028378
Improving the Performance of Infomax Using Statistical Signal Processing Techniques
In this paper, we present a new method that speeds up the convergence of the infomax algorithm proposed by Bell and Sejnowski. One effect of the infomax algorithm is that the 2nd order and 4th order statistical correlations are reintroduced to the signals during the learning process due to the optimization with respect to the complete signal statistics. We show that repetitively forcing 2nd and 4th order correlations to zero speeds up the convergence and improves separating sources with fewer data points.
15979344
A multiobjective evolutionary algorithm toolbox for computer-aided multiobjective optimization
This paper presents an interactive graphical user interface (GUI) based multiobjective evolutionary algorithm (MOEA) toolbox for effective computer-aided multiobjective (MO) optimization. Without the need of aggregating multiple criteria into a compromise function, it incorporates the concept of Pareto's optimality to evolve a family of nondominated solutions distributing along the tradeoffs uniformly. The toolbox is also designed with many useful features such as the goal and priority settings to provide better support for decision-making in MO optimization, dynamic population size that is computed adaptively according to the online discovered Pareto-front, soft/hard goal settings for constraint handlings, multiple goals specification for logical "AND"/"OR" operation, adaptive niching scheme for uniform population distribution, and a useful convergence representation for MO optimization. The MOEA toolbox is freely available for download at http://vlab.ee.nus.edu.sg/-kctan/moea.htm which is ready for immediate use with minimal knowledge needed in evolutionary computing. To use the toolbox, the user merely needs to provide a simple "model" file that specifies the objective function corresponding to his/her particular optimization problem. Other aspects like decision variable settings, optimization process monitoring and graphical results analysis can be performed easily through the embedded GUIs in the toolbox. The effectiveness and applications of the toolbox are illustrated via the design optimization problem of a practical ill-conditioned distillation system. Performance of the algorithm in MOEA toolbox is also compared with other well-known evolutionary MO optimization methods upon a benchmark problem.
1878850
Information Brokering on the World Wide Web
Information Brokering is the process of collecting and re-distributing information. As the rich but unstructured sea of data that comprises the World Wide Web continues to grow, so too will the demand for information brokering services. This paper outlines a methodology and an architecture to support information brokering on the Web. Specifically, the two innovations which are described facilitate the construction and distribution of customized views that integrate data from a number of Web sources. An original technique is presented for the extraction of data from the Web and a solution based on JavaScript and HTML is presented to support the embedding of SQL queries within Web documents.
8163714
Scaling Laws in Natural Scenes and the Inference of 3D Shape
This paper explores the statistical relationship between natural images and their underlying range (depth) images. We look at how this relationship changes over scale, and how this information can be used to enhance low resolution range data using a full resolution intensity image. Based on our findings, we propose an extension to an existing technique known as shape recipes [3], and the success of the two methods are compared using images and laser scans of real scenes. Our extension is shown to provide a two-fold improvement over the current method. Furthermore, we demonstrate that ideal linear shape-from-shading filters, when learned from natural scenes, may derive even more strength from shadow cues than from the traditional linear-Lambertian shading cues.
10997807
Resource bounded symmetry of information revisited
The information contained in a string x about a string y is the difference between the Kolmogorov complexity of y and the conditional Kolmogorov complexity of y given x, i.e., I(x : y)=C(y)-C(y|x). The Kolmogorov-Levin Theorem says that I(x : y) is symmetric up to a small additive term. We investigate if this property also holds for several versions of polynomial time-bounded Kolmogorov complexity.We study symmetry of information for some variants of distinguishing complexity CD where CD(x) is the length of a shortest program which accepts x and only x. We show relativized worlds where symmetry of information does not hold in a strong way for deterministic and nondeterministic polynomial time distinguishing complexities CDpoly and CNDpoly. On the other hand, for nondeterministic polynomial time distinguishing complexity with randomness, CAMDpoly, we show that symmetry of information holds for most pairs of strings in any set in NP. Our techniques extend work of Buhrman et al. (Language compression and pseudorandom generators, in: Proc. 19th IEEE Conf. on Computational Complexity, IEEE, New York, 2004, pp. 15-28) on language compression by AM algorithms, and have the following application to the compression of samplable sources, introduced in Trevisan et al. (Compression of sample sources, in: Proc. 19th IEEE Conf. on Computational Complexity, IEEE, New York, 2004, pp. 1-15): any element x in the support of a polynomial time samplable source X can be given a description of size - log Pr[X = x] + O(log3 n), from which x can be recovered by an AM algorithm.
323922
Decoding V1 Neuronal Activity using Particle Filtering with Volterra Kernels
Decoding is a strategy that allows us to assess the amount of information neurons can provide about certain aspects of the visual scene. In this study, we develop a method based on Bayesian sequential updating and the particle filtering algorithm to decode the activity of V1 neurons in awake monkeys. A distinction in our method is the use of Volterra kernels to filter the particles, which live in a high dimensional space. This parametric Bayesian decoding scheme is compared to the optimal linear decoder and is shown to work consistently better than the linear optimal decoder. Interestingly, our results suggest that for decoding in real time, spike trains of as few as 10 independent but similar neurons would be sufficient for decoding a critical scene variable in a particular class of visual stimuli. The reconstructed variable can predict the neural activity about as well as the actual signal with respect to the Volterra kernels.