# EIGENVALUES and EIGENVECTORS
## Prachee Javiya
---
# Notes

  * The most basic equation for eigenvalues is $Ax=\lambda x$. The number $\lambda$ is an eigenvalue of A.
   * To solve the above equation, calculate the roots of $(A-\lambda I)x=0$
   * Calculate the eigenvectors corresponding to an eigenvalue
  * The eigenvectors "x" do NOT change direction when you multiply by A.
  * Eigenvectors make understanding linear transformations easy.
  * The eigenvalue $\lambda$ tells whether the special vector x is stretched or shrunk or reversed or left unchanged-when it is multiplied by A
   * When $\lambda$ is 0, the vector x lies in the nullspace of A.
  * If A is the identity matrix, every vector has Ax= x. All vectors are eigenvectors of I.
  * |A-$\lambda$I|=0 ,always. Reason:For a particular value of $\lambda$, the vector Ax is parallel to $\lambda$x. 
 #### Calculating eigenvectors and eigenvalues :

* The 2x2 identity matrix has $\lambda$=1 which occurs twice.


```python
import sympy as sp
```


```python
A=sp.Matrix([[1,0],[0,1]])
A.eigenvals()
```




    {1: 2}



* Eigenvalues for a 2x2 permutation matrix 
 * any x in the plane is unchanged by P. The equation hence formed is Px=x
 * Any x perpendicular to the plane, equation becomes Px=0


```python
B=sp.Matrix([[0,1],[1,0]])
B.eigenvals()
```




    {-1: 1, 1: 1}



Let λ be an eigenvalue of P for the eigenvector v. You have $\lambda^2v=P^2v=Pv=\lambda v$. Because v≠0 it must be $\lambda^2=\lambda$. The solutions of the last equation are λ1=0 and λ2=1. Those are the only possible eigenvalues the projection might have.

### Properties of eigenvectors and eigenvalues:
 - Trace of the matrix A is equal to the sum of its eigenvalues. </br>
 - Additive property for calculating Evalues of two matrices does not hold.  
      >$ Ax=\lambda x$   
      >$ Bx=\gamma x$   
      >$ (A+B)x\neq (\lambda+\gamma) x $   
 - Because it is not necessary that the vector x will be same for both matrices A and B
 - A matrix A will have zero as an eigenvalue if and only if it is singular. </br>
 - Eigenvalue and eigenvector of $A^{-1}$ $$ Av=\lambda v$$ Multiplying both sides by $A^{-1}$, we get, $$ v=\lambda A^{-1}v$$ $$=A^{-1}v=\frac{1}{\lambda} v $$ 
 
**NOTE** : </br>
 When $\lambda=0, A^{-1}$ does not exist.

- When we take a multiple of I (eg 3I) the eigenvalues get multiplied by a factor 3 and the eigenvectors remain **same**


```python
C=sp.Matrix([[3,1],[1,3]])
C.eigenvals()
```




    {4: 1, 2: 1}




```python
C.eigenvects()
```




    [(2, 1, [Matrix([
       [-1],
       [ 1]])]), (4, 1, [Matrix([
       [1],
       [1]])])]



* Eigenvalues of anti-symmetric matrices are purely imaginary
* Repeated eigenvalues leads to shortage of eigenvectors(not enough linearly independant eigenvectors)</br>
  _DEGENERATE MATRIX_


```python
D=sp.Matrix([[3,1],[0,3]])
D.eigenvals()
```




    {3: 2}




```python
D.eigenvects()
```




    [(3, 2, [Matrix([
       [1],
       [0]])])]



### DIAGONALISATION 
##### A matrix is diagonalisable when there exists a matrix S and $\Lambda$ such that $$A=S\Lambda S^{-1}$$
---
* Λ is the diagonal matrix, with the entries being the corresponding eigenvalues of A
* $AS=S\Lambda$ (Be careful when the matrix S is imaginary,conjugate terms will take part, $A^H={A^T}^C$,where $A^H$ is a hermetian matrix
* If a square matrix A has a full set of eigenvectors, then the matrix can be constructed by this formula.
* Multiply by $S^{-1}$ on both sides to get the diagonal matrix i.e. $$AS=S \Lambda $$ $$S^{-1}AS=\Lambda$$ 

_Diagonalizability is concerned with the number of eigenvectors - too few or enough_

#### AM/GM
---
AM(Algebraic multiplicity) is the number of times an **eigenvalue** appears in the characteristic equation.</br>
GM(Geometric multiplicity) is the number of **eigenvectors** corresponding to an eigenvalue.

### Real Symmetric Matrices-Eigenvalues
---
- When $A=A^T$ for any matrix A, it is known as a symmetric matrix
- Evalues for a symmetric matrix are real
- Eigenvectors of real symmetric matrices are **orthogonal** 
$\therefore$ the matrix S will have orthonormal columns </br>
And, $$A=Q\Lambda Q^{-1}$$
- For orthonormal columns, $A^T=A^{-1}$
$$\therefore A=Q\Lambda Q^T$$
- Every symmetric matrix is a combination of perpendicular projection matrices

### Difference equation
---
$$u_{k+1}=Au_{k}$$
Given vector $u_0$,</br>
$u_1=Au_0, u_2=A^2u_0.... u_k=A^ku_0$
To solve, $$u_0=c_1x_1+c_2x_2+..+c_nx_n=Sc$$
$$Au_0=\lambda_1x_1+c_2\lambda_2x_2+..+c_n\lambda_nx_n$$ 
$$A^{100}u_0=\lambda_1^{100}x_1+c_2\lambda_2^{100}x_2+..+c_n\lambda_n^{100}x_n$$
$$=\Lambda^{100}Sc$$

## Fibonacci equation
---
$$F_{k+2}=F_{k+1}+F_k$$
$$F_{k+1}=F_{k+1}$$
$$ u_k=
\begin{bmatrix}
F_{k+1}\\
F_k
\end{bmatrix}
$$
$$u_{k+1}=
\begin{bmatrix}
1 & 1\\ 1 & 0
\end{bmatrix}u_k
$$

## Positive Definiteness of symmetric matrix
How to tell if a matrix is positive symmetric?
$$\begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
$$
 1. Eigenvalues are positive
 2. $a>0 , ac-b^2>0$
 3. $x^TAx>0$</br>
 
**Borderline case example,**


```python
import sympy as sp
A=sp.Matrix([[2,6],[6,18]])
A
```




$\displaystyle \left[\begin{matrix}2 & 6\\6 & 18\end{matrix}\right]$



Testing:
$$
\begin{bmatrix}
x_1 & x_2
\end{bmatrix}
\begin{bmatrix}
2 & 6\\
6 & 18
\end{bmatrix}
\begin{bmatrix}
x_1 \\
x_2
\end{bmatrix}
$$

$$=2x_1^2+12x_1x_2+18x_2^2$$

Quadratic form : $ax^2+2bxy+cy^2$</BR>
**IF THE ABOVE QUADRATIC IS POSITIVE FOR ALL x1,x2, THEN A IS POSITIVE DEFINITE**</br>
In the above case, the matrix is definite psitive for all values greater than 18

### Left Inverse/Right inverse and Pseude inverse
---
- Left Inverse
  - For a full rank(columns are independant) mxn matrix,  $(A^TA)^{-1}A^T$ is the left inverse of the matrix A.
  - A rectangular matrix cannot have two sided inverse because $m \neq n $ and there will be some free variables around
- Right Inverse 
  - $A^T(AA^T)^{-1}$ is the right inverse.$AA^T(AA^T)^{-1}=I$)
  - We have independant rows and $N(A^T)={0}$.
  - Rank r=m<n
  - Infinite solutions               
- Pseudo Inverse
  - It comess from SVD. $ A=U\Sigma V^T$ and 
$$ \Sigma=
\begin{bmatrix}
\sigma_1 &0&0&0&0\\
0&\sigma_2&0&0&0\\
0&0&\sigma_3&0&0\\
0&0&0&.&. \\
.&.&.&.&\sigma_r
\end{bmatrix}
_{m\times n}
$$
then,Pseudo inverse of $\Sigma$ will be,
$$\Sigma^+=
\begin{bmatrix}
\frac{1}{\sigma_1} &0&0&0&0\\
0&\frac{1}{\sigma_2}&0&0&0\\
0&0&\frac{1}{\sigma_3}&0&0\\
0&0&0&.&. \\
.&.&.&.&\frac{1}{\sigma_r}
\end{bmatrix}
_{n\times m}
$$


```python

```

33ad966b6984a698a04a81673345a4e554747731
ipynb
Jupyter Notebook
Evalues.ipynb
PracheeJaviya/Linear-Algebra
7dbe32a55cdf53d5ef063cc25ec19af4cab85b11
MIT
Evalues.ipynb
PracheeJaviya/Linear-Algebra
7dbe32a55cdf53d5ef063cc25ec19af4cab85b11
MIT
Evalues.ipynb
PracheeJaviya/Linear-Algebra
7dbe32a55cdf53d5ef063cc25ec19af4cab85b11
MIT
Qwen/Qwen-72B
1. YES
2. YES