Principal component analysis
7128002
Robust principal component analysis?
This article is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low rank component and a sparse component. Can we recover each component individually? We prove that under some suitable assumptions, it is possible to recover both the low rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the l1 norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.
JACM
Emmanuel J Candes Xiaodong Li Yuliang Ma John Wright
27917863
Principal Component Analysis
International Encyclopedia of Statistical Science
Ian T Jolliffe
2534141
Principal Component Analysis
The Karhunen Lo eve basis functions, more frequently referred to as principal components or empirical orthogonal functions (EOFs) of the noise response of the climate system are an important tool for geophysical studies. Many researchers have used this tool to examine the geophysical and climatological phenomena. Perhaps more frequent use of EOFs in recent studies is in conjunction with the development of the signal detection and estimation methods of the background uctuations of a detection variable serve as an orthogonal basis set and are used to design optimal techniques for detecting and estimating signals. A detection and prediction approach is to design a lter or optimal weights for the signal to be detected. It has been reported that weighted averaging of data over the surface of the Earth improved the detectability of climatic changes (Hasselmann 1979; Stefanick 1981; Bell 1982) Since the signal to noise ratio (SNR) varies geographically, there exists an optimal geographical weighting of the signal which maximizes the SNR. The design of an optimal weighting function may require detailed knowledge on the natural uctuation of the climate system. A conceptually similar approach is to employ a particular pattern (or patterns) of climatic change for detection and prediction (e.g. Barnett and Hasselmann 1979; Hasselmann 1979) The patterns of interest (also called the predictors) may include the principal components (empirical orthogonal functions) (e. von Storch 1990) among others. This approach also requires complete knowledge of the natural variability of the climate system. To test and improve the detection and prediction techniques addressed above, a complete cross spectral covariance matrix, or similarly, a complete set of the principal components of natural uctuations of the climate system for each frequency band of the spectrum is necessary. In reality, a reliable spectrum of observational covariance matrix is not available because observations are not suuciently long and sampling errors contaminate the observational records (Preisendorfer and Barnett 1977; North et al. 1982) Further, inadequate spatial coverage of observations may introduce bias. Therefore, the covariance matrix of the noise response is often estimated from a simple stochastic model. Kim and North (1991, 1992) examined the covariance matrix in terms of various second moment statistics earlier. Examined here are the principal components of the covariance matrix of the surface temperature uctuations in a simple coupled climate model in comparison with observations. The principal components not only are an
Encyclopedia of Database Systems
Heng Tao Shen
122725444
Principal Component Analysis
Introduction Properties of Population Principal Components Properties of Sample Principal Components Interpreting Principal Components: Examples Graphical Representation of Data Using Principal Components Choosing a Subset of Principal Components or Variables Principal Component Analysis and Factor Analysis Principal Components in Regression Analysis Principal Components Used with Other Multivariate Techniques Outlier Detection, Influential Observations and Robust Estimation Rotation and Interpretation of Principal Components Principal Component Analysis for Time Series and Other Non Independent Data Principal Component Analysis for Special Types of Data Generalizations and Adaptations of Principal Component Analysis
Ian T Jolliffe
4737805
Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm
In this paper, we consider the Tensor Robust Principal Component Analysis (TRPCA) problem, which aims to exactly recover the low rank and sparse components from their sum. Our model is based on the recently proposed tensor tensor product (or t product) [14] Induced by the t product, we first rigorously deduce the tensor spectral norm, tensor nuclear norm, and tensor average rank, and show that the tensor nuclear norm is the convex envelope of the tensor average rank within the unit ball of the tensor spectral norm. These definitions, their relationships and properties are consistent with matrix cases. Equipped with the new tensor nuclear norm, we then solve the TRPCA problem by solving a convex program and provide the theoretical guarantee for the exact recovery. Our TRPCA model and recovery guarantee include matrix RPCA as a special case. Numerical experiments verify our results, and the applications to image recovery and background modeling problems demonstrate the effectiveness of our method.
IEEE Transactions on Pattern Analysis and Machine Intelligence
Canyi Lu Jiashi Feng Yudong Chen W Liu Zhouchen Lin Shuicheng Yan
117887765
Principal component analysis in linear systems: Controllability, observability, and model reduction
Kalman's minimal realization theory involves geometric objects (controllable, unobservable subspaces) which are subject to structural instability. Specifically, arbitrarily small perturbations in a model may cause a change in the dimensions of the associated subspaces. This situation is manifested in computational difficulties which arise in attempts to apply textbook algorithms for computing a minimal realization. Structural instability associated with geometric theories is not unique to control; it arises in the theory of linear equations as well. In this setting, the computational problems have been studied for decades and excellent tools have been developed for coping with the situation. One of the main goals of this paper is to call attention to principal component analysis (Hotelling, 1933) and an algorithm (Golub and Reinsch, 1970) for computing the singular value decompositon of a matrix. Together they form a powerful tool for coping with structural instability in dynamic systems. As developed in this paper, principal component analysis is a technique for analyzing signals. (Singular value decomposition provides the computational machinery. For this reason, Kalman's minimal realization theory is recast in terms of responses to injected signals. Application of the signal analysis to controllability and observability leads to a coordinate system in which the "internally balanced" model has special properties. For asymptotically stable systems, this yields working approximations of X_{c} X_{\bar{o} the controllable and unobservable subspaces. It is proposed that a natural first step in model reduction is to apply the mechanics of minimal realization using these working subspaces.
Bradley Alan Moore
54979942
Principal component analysis
Principal Component Analysis (PCA) is a multivariate exploratory analysis method, useful to separate systematic variation from noise. It allows to define a space of reduced dimensions that preserve
Svante Wold Kim H Esbensen Paul Geladi
20101754
Principal component analysis: a review and recent developments
Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori, hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.
Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Ian T Jolliffe Jorge Cadima
15538672
Probabilistic Principal Component Analysis
Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.
Michael E Tipping Christopher M Bishop
122379222
Principal component analysis
Principal component analysis PCA is a multivariate technique that analyzes a data table in which observations are described by several inter correlated quantitative dependent variables. Its goal is to extract the important information from the table, to represent it as a set of new orthogonal variables called principal components, and to display the pattern of similarity of the observations and of the variables as points in maps. The quality of the PCA model can be evaluated using cross validation techniques such as the bootstrap and the jackknife. PCA can be generalized as correspondence analysis CA in order to handle qualitative variables and as multiple factor analysis MFA in order to handle heterogeneous sets of variables. Mathematically, PCA depends upon the eigen decomposition of positive semi definite matrices and upon the singular value decomposition SVD of rectangular matrices. Copyright (c) 2010 John Wiley Sons, Inc.
Herve Abdi Lynne J Williams