```python
import numpy as np

import scipy.linalg as la

import sympy as sp
```

Lecture 14. Orthogonality

Orthogonal vectors:   


```python
x = np.array([1,0])
y = np.array([0,1])
np.dot(x.T,y)
```




    0



|x|^2 + |y|^2 = |x+y|^2

x.Tx - length squared

x.Tx+y.Ty = (x+y).T(x+y)

x.Tx+y.Ty = x.Tx+x.Ty+y.Tx+y.Ty

2x.Ty = 0

x.Ty = 0


```python
np.dot(x.T,x)
```




    1




```python
np.dot(y.T,y)
```




    1




```python
np.dot((x+y).T,(x+y))
```




    2



Dot product of orthogonal vectors is 0.

0 vector is orthogonal to everthing.

Now to subspaces

Supspace S is orthogonal to subspace T:

means every vector in S is orthogonal to every vector in T. They don't intersect (except of 0 vector).

Rowspace is orthogonal to a nullspace.

Ax = 0

Means the dot product of all the rows of A and x is 0, and x is a vector in nullspace as a definition.

c1(row1).Tx = 0

"+" = 0

c2(row2).Tx = 0

Nullspace and rowspace are orthogonal complements in R^n.

Nullspace contains all vectors perpedicular to rowspace.

Coming: Ax = b

"solve" when there is no solution.

m > n

b is not in acolumnspace

Practically too many equations with noise on a right side. E.g. finding the best solution.

Drop some equtions is not a solution, as by droping them we literally drop the measurements and loose information.



A.TA - is a square symmetric matrix (A.TA).T = A.TA

A.TAx' = A.Tb  is a way to go. x' is a best solution, not a solution to original system.

N(A.TA) = N(A)

rank of A.TA  = rank of A

A.TA is invertible if A has independant columns.

Lecture 15. Projections.

a.T(b-xa) = 0 - projection of a vector b to a vector a.

xa.Ta = a.Tb

x = (a.Tb)/(a.Ta)

p = ax

p = a(a.Tb)/(a.Ta)

Doubling b will double p - projection.

proj p = Pb

P = aa.T/a.Ta - n x n matrix

C(P)  = a line through a

rank(P) = 1

P.T = P

The projection of a projection is the same projection.

P^2 = P


Why to even project?

Because Ax = b may have no solution, thus we project b to a columnspace of A and this is exactly the closest and the best possible solution.

e = b-p  - error. The reason why we project, and projection sort of eliminates the error.

p = x1'a1 + x2'a2  = Ax'

p = Ax'

Find a solution, such as error vector is perpendicular to the plain. b = Ax'


a1.T(b-Ax' = 0 a2.T(b-Ax')

A.T(b-Ax') = 0

e in N(A.T)

e perpendicular to C(A) - YES!

Key: A.TAx' = A.Tb

Projection matrix.

x'  = (A.TA)^(-1)A.Tb

p = Ax' = A(A.TA)^(-1)A.T b

matrix P = A(A.TA)^(-1)A.T

for squared A; P  = AA^-1(A.T)^-1A.T = I, which will mean it covers the b, and P prohjects b to b.

You can't do this for rectangular.

P.T = P

P^2 = P



Least squares. Fitting 3 points with a best line. Practically a linear regression.

Lecture 16

Example:

C+D = 1

C+2D = 2

C+3D = 2


```python
A = np.array([[1,1],
             [1,2],
             [1,3]])

b = np.array([1,2,2])
```


```python
P = np.dot(np.dot(A, np.linalg.inv(np.dot(A.T,A))),A.T)
```


```python
P
```




    array([[ 0.83333333,  0.33333333, -0.16666667],
           [ 0.33333333,  0.33333333,  0.33333333],
           [-0.16666667,  0.33333333,  0.83333333]])




```python
newb = np.dot(P,b)
newb
```




    array([ 1.16666667,  1.66666667,  2.16666667])



Ax = newb


```python
A
```




    array([[1, 1],
           [1, 2],
           [1, 3]])



Practically just solve A.TAx' = A.Tb


```python
xhat = la.solve(np.dot(A.T,A),np.dot(A.T,b))
xhat
```




    array([ 0.66666667,  0.5       ])




```python
np.around(np.dot(A,xhat) - newb)
```




    array([-0., -0., -0.])




```python
la.inv(np.dot(A.T,A))
```




    array([[ 2.33333333, -1.        ],
           [-1.        ,  0.5       ]])




```python

```

2827e6ee58d2122a9f1126a5a29d045b4e55e96d
ipynb
Jupyter Notebook
LinAl_007.ipynb
rtgshv/linal101
f520987a6f1e468b3b466c14820e43dada565e43
MIT
LinAl_007.ipynb
rtgshv/linal101
f520987a6f1e468b3b466c14820e43dada565e43
MIT
LinAl_007.ipynb
rtgshv/linal101
f520987a6f1e468b3b466c14820e43dada565e43
MIT
Qwen/Qwen-72B
1. YES
2. YES