The nature of statistical learning theory
7138354
The Nature of Statistical Learning Theory
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?
Statistics for Engineering and Information Science
Vladimir Naumovich Vapnik
206755547
The Nature Of Statistical Learning Theory
If you really want to be smarter, reading can be one of the lots ways to evoke and realize. Many people who like reading will have more knowledge and experiences. Reading can be a way to gain information from economics, politics, science, fiction, literature, religion, and many others. As one of the part of book categories, the nature of statistical learning theory always becomes the most wanted book. Many people are absolutely searching for this book. It means that many love to read this kind of book.
IEEE Trans. Neural Networks
Vladimir Cherkassky
195825425
The Nature of Statistical Learning Theory
Vladimir Naumovich Vapni
198120202
The Nature of Statistical Learning Theory, Second Edition
Statistics for Engineering and Information Science
Vladimir Naumovich Vapnik
121019091
An examination of methods for approximating implicit limit state functions from the viewpoint of statistical learning theory
The reliability analysis of complex structures is hindered by the implicit nature of the limit state function. For their approximation use has been made of the Response Surface Method (RSM) and, more recently, of Neural Networks. From the statistical viewpoint this corresponds to a regression approach. In the structural reliability literature little attention has been paid, however, to the possibility of treating the problem as a classification task. This enlarges the list of methods that are eventually useful to the purpose at hand and justifies an overall examination of their distinguishing features. This task is performed in this paper from the point of view of the Theory of Statistical Learning, which provides a unified framework for all regression, classification and probability density estimation. The classification methods are grouped into three categories and it is shown that only one group is useful for structural reliability, according to some specific criteria. In this category are the Multi Layer Perceptrons and the Support Vector Machines, which are the recommended methods because (a) they can estimate the function on the basis of a few samples, (b) they use flexible and adaptive models and (c) they can overcome the curse of dimensionality. The paper also includes an in depth analysis of the RSM from the point of view of statistical learning. It is shown that the empirically found instability of this method is explained with statistical learning concepts.
Jorge E Alva Hurtado
49323521
Statistical learning of language: Theory, validity, and predictions of a statistical learning account of language acquisition
Abstract Considerable research indicates that learners are sensitive to probabilistic structure in laboratory studies of artificial language learning. However, the artificial and simplified nature of the stimuli used in the pioneering work on the acquisition of statistical regularities has raised doubts about the scalability of such learning to the complexity of natural language input. In this review, we explore a central prediction of statistical learning accounts of language acquisition that sensitivity to statistical structure should be linked to real language processes via an examination of: (1) recent studies that have increased the ecological validity of the stimuli; (2) studies that suggest statistical segmentation produces representations that share properties with real words; (3) correlations between individual variability in statistical learning ability and individual variability in language outcomes; and (4) atypicalities in statistical learning in clinical populations characterized by language delays or deficits.
Lucy Erickson Erik D Thiessen
108339887
Statistical learning methods for ranking theory, algorithms and applications
Multipartite ranking is a statistical learning problem that consists in ordering observations that belong to a high dimensional feature space in the same order as the labels, so that the observations with the highest label appear at the top of the list. This work aims to understand the probabilistic nature of the multipartite ranking problem in order to obtain theoretical guarantees for ranking algorithms. In this context, the output of a ranking algorithm takes the form of a scoring function, a function that maps the space of the observation to the real line which order is induced using the values on the real line. The contributions of this manuscript are the following First, we focus on the characterization of optimal solutions to multipartite ranking. The second research theme is the design of algorithms to produce scoring functions. We offer two methods, the first using an aggregation procedure, the second an approximation scheme. Finally, we return to the binary ranking problem to establish adaptive minimax rate of convergence.
Sylvain Robbiano
8509703
Coping with the Computational and Statistical Bipolar Nature of Machine Learning
Machine Learning is known to have its roots in a broad spectrum of fields including Artificial Intelligence, Pattern Recognition, Statistics or Optimisation. From the earliest stages of Machine Learning, both computational issues and generalisation properties have been identified as central to the field. While the former address the question of computability, complexity (from a fundamental perspective) or computational efficiency (on a more practical standpoint) of learning systems, the latter aim at understanding and characterising how well the solutions they provide perform on new, unseen data. Those last years, the emergence of large scale datasets in Machine Learning has been deeply reshaping the principles of Learning Theory. Taking into account possible constraints on the training time, one has to deal with more complex trade offs than the ones classically addressed by Statistics. As a direct consequence, designing new efficient algorithms (both in theory and practice) able to handle large scale datasets, imposes to jointly deal with the statistical and computational aspects of Learning. The present thesis aims at unravelling, analysing and exploiting some of the connections that naturally exist between the statistical and computational aspects of Learning. More precisely, in a first part, we extend the stability analysis, which relates some algorithmic properties to the generalisation abilities of learning algorithms, to a novel (and fine grain) performance measure, namely the confusion matrix. In a second part, we present a novel approach to learn a kernel based regression function, that serves the learning task at hand and exploits the structure of the problem so that the optimisation procedure is made inexpensive. Finally, we investigate the trade off between convergence rate and computational cost when minimising a composite functional with inexact proximal gradient methods. In that setting, we identify optimisation strategies that provably are computationally optimal.
Pierre Machart
7565247
Learning Theory for Distribution Regression
We focus on the distribution regression problem: regressing to vector valued outputs from probability measures. Many important machine learning and statistical tasks fit into this framework, including multi instance learning and point estimation problems without analytical solution (such as hyperparameter or entropy estimation) Despite the large number of available heuristics in the literature, the inherent two stage sampled nature of the problem makes the theoretical analysis quite challenging, since in practice only samples from sampled distributions are observable, and the estimates have to rely on similarities computed between sets of points. To the best of our knowledge, the only existing technique with consistency guarantees for distribution regression requires kernel density estimation as an intermediate step (which often performs poorly in practice) and the domain of the distributions to be compact Euclidean. In this paper, we study a simple, analytically computable, ridge regression based alternative to distribution regression, where we embed the distributions to a reproducing kernel Hilbert space, and learn the regressor from the embeddings to the outputs. Our main contribution is to prove that this scheme is consistent in the two stage sampled setup under mild conditions (on separable topological domains enriched with kernels) we present an exact computational statistical efficiency trade off analysis showing that our estimator is able to match the one stage sampled minimax optimal rate [Caponnetto and De Vito, 2007; Steinwart et al. 2009] This result answers a 17 year old open question, establishing the consistency of the classical set kernel [Haussler, 1999; Gaertner et. al, 2002] in regression. We also cover consistency for more recent kernels on distributions, including those due to [Christmann and Steinwart, 2010]
J. Mach. Learn. Res.
Zoltan Szabo Bharath K Sriperumbudur Barnabas Poczos Arthur Gretton
126179132
Empirical risk approximation A statistical learning theory of data clustering
k means data clustering estimates a partition of a vectorial data set in an unsupervised way. The partition assigns data to clusters and it is represented by a set of cluster centers. We apply computational learning theory to discuss the nature of phase transitions in data clustering. The requirement that the partition has to generalize from one data sample set to a second sample set drawn from the same distribution yields a new combinatorial parameter selection criterion for cluster splits. Our analysis demonstrates that maximum entropy based approximations schemes are optimal continuation methods for optimization with build in control of model complexity.
Joachim M Buhmann Naftali Tishby