1902919
Multi-Conditional Learning: Generative/Discriminative Training for Clustering and Classification
This paper presents multi-conditional learning (MCL), a training criterion based on a product of multiple conditional likelihoods. When combining the traditional conditional probability of "label given input" with a generative probability of "input given label" the later acts as a surprisingly effective rerularizer. When applied to models with latent variables, MCL combines the structure-discovery capabilities of generative topic models, such as latent Dirichlet allocation and the exponential family harmonium, with the accuracy and robustness of discriminative classifiers, such as logistic regression and conditional random fields. We present results on several standard text data sets showing significant reductions in classification error due to MCL regularization, and substantial gains in precision and recall due to the latent structure discovered under MCL.
753865
Dynamic Mixture Models for Multiple Time-Series
Traditional probabilistic mixture models such as Latent Dirichlet Allocation imply that data records (such as documents) are fully exchangeable. However, data are naturally collected along time, thus obey some order in time. In this paper, we present Dynamic Mixture Models (DMMs) for online pattern discovery in multiple time series. DMMs do not have the noticeable drawback of the SVD-based methods for data streams: negative values in hidden variables are often produced even with all non-negative inputs. We apply DMM models to two real-world datasets, and achieve significantly better results with intuitive interpretation.
149826
Generalized component analysis for text with heterogeneous attributes
We present a class of richly structured, undirected hidden variable models suitable for simultaneously modeling text along with other attributes encoded in different modalities. Our model generalizes techniques such as principal component analysis to heterogeneous data types. In contrast to other approaches, this framework allows modalities such as words, authors and timestamps to be captured in their natural, probabilistic encodings. A latent space representation for a previously unseen document can be obtained through a fast matrix multiplication using our method. We demonstrate the effectiveness of our framework on the task of author prediction from 13 years of the NIPS conference proceedings and for a recipient prediction task using a 10-month academic email archive of a researcher. Our approach should be more broadly applicable to many real-world applications where one wishes to efficiently make predictions for a large number of potential outputs using dimensionality reduction in a well defined probabilistic framework.
32712892
Learning Image Manifold Using Web Data
Manifold learning has become a hot research topic in recent years and is widely used in the area of dimension reduction, information retrieval and ranking, etc. However, how to reconstruct the intrinsic manifold from the observed data points, i.e. what is the proper data point distance measure, is still an open problem. In this paper, we propose to take advantages of the information provided by web-pages and the image-related website link structure to learn the Web image manifold, which better approaches to the intrinsic manifold than those learned by previous methods which use Euclidean alike distances to construct the initial affinity matrix. Experimental results prove the effectiveness of our learned Web image manifold.
18717765
A Bayesian solution and its approximations to out-of-sequence measurement problems
Abstract Target tracking using delayed, out-of-sequence measurements is a problem of growing importance due to an increased reliance on networked sensors interconnected via complex communication network architectures. In such systems, it is often the case that measurements are received out-of-time-order at the fusion center. This paper presents a Bayesian solution to this problem and provides approximate, implementable algorithms for both cluttered and non-cluttered scenarios involving single and multiple time-delayed measurements. Such an approach leads to a solution involving the joint probability density of current and past target states. In contrast, existing solutions in the literature modify the sensor measurement equation to account for the time delay and explicitly deal with the resulting correlations that arise in the process noise and current target state. In the Bayesian solution proposed in this paper, such cross correlations are treated implicitly. Under linear Gaussian assumptions, the Bayesian solution reduces to an augmented state Kalman filter (AS-KF) for scenarios devoid of clutter and an augmented state probabilistic data association filter (AS-PDA) for scenarios involving clutter. Computationally efficient versions of AS-KF and AS-PDA are considered in this paper. Simulations are presented to evaluate the performance of these solutions.
1383241
CBC: clustering based text classification requiring minimal labeled data
Semisupervised learning methods construct classifiers using both labeled and unlabeled training data samples. While unlabeled data samples can help to improve the accuracy of trained models to certain extent, existing methods still face difficulties when labeled data is not sufficient and biased against the underlying data distribution. We present a clustering based classification (CBC) approach. Using this approach, training data, including both the labeled and unlabeled data, is first clustered with the guidance of the labeled data. Some of unlabeled data samples are then labeled based on the clusters obtained. Discriminative classifiers can subsequently be trained with the expanded labeled dataset. The effectiveness of the proposed method is justified analytically. Our experimental results demonstrated that CBC outperforms existing algorithms when the size of labeled dataset is very small.
15536346
An Index Structure for Data Mining and Clustering
Abstract. In this paper we present an index structure, called MetricMap, that takes a set of objects and a distance metric and then maps those objects to a k-dimensional space in such a way that the distances among objects are approximately preserved. The index structure is a useful tool for clustering and visualization in data-intensive applications, because it replaces expensive distance calculations by sum-of-square calculations. This can make clustering in large databases with expensive distance metrics practical. We compare the index structure with another data mining index structure, FastMap, recently proposed by Faloutsos and Lin, according to two criteria: relative error and clustering accuracy. For relative error, we show that (i) FastMap gives a lower relative error than MetricMap for Euclidean distances, (ii) MetricMap gives a lower relative error than FastMap for non-Euclidean distances (i.e., general distance metrics), and (iii) combining the two reduces the error yet further. A similar result is obtained when comparing the accuracy of clustering. These results hold for different data sizes. The main qualitative conclusion is that these two index structures capture complementary information about distance metrics and therefore can be used together to great benefit. The net effect is that multi-day computations can be done in minutes.
5101914
Wave Propagation Mediated by GABAB Synapse and Rebound Excitation in an Inhibitory Network: A Reduced Model Approach
A reduction method is used to analyze a spatially structured network model of inhibitory neurons. This network model displays wave propagation of postinhibitory rebound activity, which depends on GABAB synaptic interactions among the neurons. The reduced model allows explicit solutions for the wavefronts and their velocity as a function of various parameters, such as the synaptic coupling strength. These predictions are shown to agree well with the numerical simulations of the conductance-based biophysical model.