# PCA on independent but non-identically distributed multinormal data

In the empirical phase diagram approach the data are given by a set ${\boldsymbol X}$ of input (controlled) variables (such as pH and temperature) and a set ${\boldsymbol Y}$ of output variables. The empirical phase diagram aims to visualize (at least, roughly) the relation between the input and the output.

The strategy runs as follows in case when there are only two input variables:

• firstly we ignore the input ${\boldsymbol X}$ and we run a PCA on ${\boldsymbol Y}$

• keep the first three principal components

• transform the three coordinates on the principal components into a color using the red-green-blue coding

• plot the color in function of the two input variables, and visualize

I am rather new in PCA analysis (so please do not hesitate to tell me my questions are stupid if they are). I know PCA has a better interpretation when it is applied with a sample of i.i.d. random multivariate normal variables. But I don't know what are the possible pitfalls when this assumption does not hold.

Assume for instance a multivariate regression model for which the distribution of a single multivariate response ${\boldsymbol Y}$ is assumed to be: $${\boldsymbol Y}_i = f({\boldsymbol X}_i) + \epsilon_i \quad \textrm{with } \epsilon_i \sim {\cal N}({\boldsymbol 0}, \Sigma).$$ I think that we ideally should run the PCA on the centered responses ${\boldsymbol Y}_i - \hat{f}({\boldsymbol X}_i)$ in such a situation.

So what are the possible pitfalls if we run the above strategy in such a situation ?

-
+1 I think this is a good question not a stupid one. –  Michael Chernick Sep 18 '12 at 11:01

If you run PCA on ${\bf Y}_i - \hat f({\bf X}_i)$, you are visualizing residuals $\epsilon_i$, not the original data ${\bf Y}_i$. In other words, you will analyze the conditional variance of ${\bf Y}_i$, and that may be of interest per se. If it is the variability of ${\bf Y}_i$ that you want to visualize, and you know these outcomes are affected by ${\bf X}_i$'s, then what you are describing appears a moderately sensible visualization, with the caveat that 5D graphs are difficult to read and interpret. You could also look into principal curves if the dependence on ${\bf X}_i$'s is heavily non-linear.

-
There's no conditional distribution in the regression model so I don't see why are you talking about some conditional variance. Moreover I don't want to use any model, my question is about what happens if there is such a model but we don't use it. Thanks for mentioning the principal curves, I didn't know this notion. –  Stéphane Laurent Sep 19 '12 at 5:02
Oops sorry I'm still sleeping ;) You mean that the regression model is a modeling of the conditional distribution of Y given the covariates, right. –  Stéphane Laurent Sep 19 '12 at 5:05
IF the data are iid normals, then you get a sphere (the covariance matrix is $\sigma^2I$). With normals, you always get ellipsoids (in a smaller dimension perhaps, because of non-zero correlations.) –  user765195 Sep 20 '12 at 3:24
the data are always multivariate. With i.i.d. multinormal ${\cal N}({\boldsymbol \mu}, \Sigma)$ one gets the ellipsoid associated to $\Sigma$, which is a sphere when $\Sigma=\sigma^2 I$. This point is clear. –  Stéphane Laurent Sep 20 '12 at 5:11