# 3.2.4 Multiple Outputs

Suppose we have multiple outputs $Y_1, Y_2, ..., Y_k$ that we wish to predict from our inputs $X_0, X_1, ..., X_p$. We assume a linear model for each output (3.34, 3.35):

$$
\begin{align}
Y_k &= \beta_{0k} + \sum_{j=1}^{p} X_j\beta_{jk} + \varepsilon_k\\
&= f_k(X) + \varepsilon_k
\end{align}
$$

We can write the model in matrix notation:

$$
\mathbf{Y} = \mathbf{XB} + \mathbf{E}
$$

Here **Y** is the $N\times K $ response matrix, **X** is the $N\times(p+1)$ input matrix, **B** is the $(p+1)\times K$ matrix and **E** is the $N\times K$ matrix of errors.

A straightforward generalization of the univariate loss functio is (3.37, 3.38):
$$
\begin{align}
RSS(\mathbf{B}) &= \sum_{k=1}^K{\sum_{i=1}^N (y_{ik} - f_k(x_i))^2}\\
&= tr\left[(\mathbf{Y}-\mathbf{XB})^T(\mathbf{Y}-\mathbf{XB})\right]
\end{align}
$$

The least squares estimates is (3.39):
$$\hat{\mathbf{B}} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}$$

If the errors $\varepsilon=(\varepsilon_1,...,\varepsilon_K)$ are correlated, then (3.40):

$$
RSS(B; \Sigma)=\sum_{i=1}^N(y_i - f(x_i))^T\Sigma^{-1}(y_i - f(x_i))
$$

arises from multivariate Gaussian theory.

924828281c9cff57d61abe13c3212cd077a7ead7
ipynb
Jupyter Notebook
chapter-03/3.2.4-multiple-outputs.ipynb
leduran/ESL
fcb6c8268d6a64962c013006d9298c6f5a7104fe
MIT
2019-01-28T14:05:02.000Z
2022-03-27T00:11:21.000Z
chapter-03/3.2.4-multiple-outputs.ipynb
leduran/ESL
fcb6c8268d6a64962c013006d9298c6f5a7104fe
MIT
2020-07-06T16:51:40.000Z
2020-07-06T16:51:40.000Z
chapter-03/3.2.4-multiple-outputs.ipynb
leduran/ESL
fcb6c8268d6a64962c013006d9298c6f5a7104fe
MIT
2019-03-21T23:48:35.000Z
2022-03-31T13:05:10.000Z
Qwen/Qwen-72B
1. YES
2. YES