# Homework - 1 

#####Vectors and Matrices 

Consider the matrix X and the vectors y and z below:

$$ 
\mathbf{X} = 
\begin{bmatrix} 
2&4 \\
1&3
\end{bmatrix}
$$

$$\mathbf{y} = \begin{bmatrix} 1 \\ 3 \end{bmatrix}$$

$$\mathbf{z} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$$




**1. What is the inner product of the vectors y and z? (this is also sometimes called the dot product, and is sometimes written $y^Tz$)**

Inner Product of y and z ($y^T z$) = $(1 * 2) + ( 3 * 3) = 11$ 


```
# Inner Product of two vectors using Numpy
import numpy as np
y =  np.array([[1], [3]])
z = np.array([[2], [3]])

print np.vdot(y,z)
```

    11


**2. What is the product Xy?**

Xy = $\begin{bmatrix} (2 * 1) + ( 1 * 3) \\(4 * 1) + (3 * 3) \end{bmatrix}$

Xy = $\begin{bmatrix} 5 \\ 13 \end{bmatrix}$


```
# Matrix Vector multiplication using Numpy
X = np.array([[2,1],[4,3]])
y =  np.array([[1], [3]])

print X.dot(y)
```

    [[ 5]
     [13]]


**3. Is X invertible? If so, give the inverse, and if no, explain why not.**

A n x n square matrix A is said to be invertible or nonsingular if there exists any n x n square matrix B such that $$ AB = BA = I_n $$ , where $I_n$ is a n x n Identity matrix.

The inverse matrix of $X$, denoted by $X^{-1}$ is $$\mathbf{X^{-1}} = \begin{bmatrix} 1.5&-0.5 \\ -2&1 \end{bmatrix}$$


```
# Inverse of X in Numpy
from numpy.linalg import inv
X_inv = inv(X)
print 'Inverse of X is:'
print X_inv

print 'X * X_inv is an Identity Matrix'
print X.dot(X_inv)
```

    Inverse of X is:
    [[ 1.5 -0.5]
     [-2.   1. ]]
    X * X_inv is an Identity Matrix
    [[ 1.  0.]
     [ 0.  1.]]


**4. What is the rank of X?**

The rank of X is 2, since the column rank is 2.

##### Calculus

**1. If $y = x^3 + x − 5$ then what is the derivative of y with respect to x?**

Derivative of y w.r.t x is: $ \frac{dy}{dx} = 3x^2 + 1$

**2. If $y = x\:sin(z)\:e^{−x}$ then what is the partial derivative of y with respect to x?**

Using Product Rule and the fact that derivative of $e^{-x} = -e^{-x}$, $x = 1$, we have 

$\frac{\partial y}{\partial x} = sin(z)\:e^{-x} - x\:sin(z)\:e^{-x}$

##### Probability and Statistics

Consider a sample of data S = {1, 1, 0, 1, 0} created by flipping a coin x five times, where 0 denotes that the coin turned up heads and 1 denotes that it turned up tails.

**1. What is the sample mean for this data?**

Sample Mean is $\frac{1 + 1 + 0 + 1 + 0}{5} = \frac{3}{5}$

**2. What is the sample variance for this data?**

Sample Variance is $\frac{6}{25}$

** 3. What is the probability of observing this data, assuming it was generated by flipping a coin with an equal probability of heads and tails (i.e. the probability distribution is p(x = 1) = 0.5, p(x = 0) = 0.5).**

Since it is a sequence of five independent tosses with P(H) = P(T) = 0.5, 
P(S) = $(\frac{1}{2})^5 = \frac{1}{32}$


** 4. Note that the probability of this data sample would be greater if the value of p(x = 1) was not 0.5, but instead some other value. What is the value that maximizes the probability of the sample S. Please justify your answer.**

Answer:
Let p(x=1) = p. Therefore, we have p(x=0) = 1-p. If n is the number of tosses, we have the total probability as $\prod\limits_{i=1}^n p^{x_i} \: (1-p)^{x_i} = p^{\sum_{i=1}^n x_i} \: (1-p)^{n - \sum_{i=1}^n x_i}$

Let y = $ {\sum_{i=1}^n x_i} $. 

The above equation can be now written as $ p^{y} \: (1-p)^{n - y}$

Since we need to find the value of p that maximizes the probability of sample S, we can take the log of the above equation, take its derivative w.r.t p, set it to zero and solve for p.

Taking the log, we have $y\>log(p) + (n-y)\>log(1-p)$, since $log(a^p) = p\>log(a)$ and $log(a * b) = log(a) + log(b)$.

Taking the derivative w.r.t p and setting it to 0 and solving for p, we have:

$$\begin{equation}
\begin{split}
\frac{dl(p)}{dp} & = \frac{y}{p} + \frac{n-y}{(1-p)} = 0 \\
 \implies 0 & = \frac{(1-p)\>y + p\>(n - y)}{p\>(1-p)}  \\
 \implies 0 & = \frac{y - py + py - pn}{p\>(1-p)} \\
 \implies 0 & = \frac{y - pn}{p\>(1-p)} \\
 pn & = y \\
 p & = \frac{1}{n} y \\
 p & = \frac{1}{n} {\sum_{i=1}^n x_i} \\
\end{split}
\end{equation}$$

Plugging in the values for $x_i$ and n=5, we get $\boxed{p = \frac{3}{5}}$


```
# Custom Styling - Please ignore
# Custom CSS Styling 
from IPython.core.display import HTML

def css_styling():
    styles = open("../../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()

```




<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsx.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsi.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunso.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }  
</style>





e05c8b6d1cb8649cf037333ba6cfbc39747e7a4f
ipynb
Jupyter Notebook
coursework/CMU 10-601/Homework 1.ipynb
mathkann/ML
65ace09c7327c2625ed176bc7d0e7ad46794218e
MIT
2015-08-15T11:16:14.000Z
2015-08-15T11:16:14.000Z
coursework/CMU 10-601/Homework 1.ipynb
mathkann/ML
65ace09c7327c2625ed176bc7d0e7ad46794218e
MIT
coursework/CMU 10-601/Homework 1.ipynb
mathkann/ML
65ace09c7327c2625ed176bc7d0e7ad46794218e
MIT
Qwen/Qwen-72B
1. YES
2. YES