<a href="https://colab.research.google.com/github/bkgsur/StatisticsWithPython/blob/main/MaximumLikelihoodEstimation.ipynb" target="_parent"></a>

**Estimation Using Maximum Likelihood (MLE)** is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference. 
https://en.wikipedia.org/wiki/Maximum_likelihood_estimation


```python
import numpy as np
import scipy.stats
import sympy
import math
```


```python
# sample count
N = 1000
x = sympy.symbols('x', positive= True)
```


```python
def parameterEstimate(p_dist, df, param_true,v):
  print("True parameter value:", param_true)
  # Experiment and samples
  samples = p_dist.rvs(N)
  #likelihood function 
  L = np.product([df.subs(x,sample) for sample in samples])
  print("Likelihood Function:", L)
  # solve for Likelihood 
  #Note: log of the likelihood fincton makes the maximization problem tractable.

  Log_L = sympy.expand_log(sympy.log(L))
  param_est, = sympy.solve(sympy.diff(Log_L, v),v)

  print("Estimated parameter value:", float(param_est))
```


```python
print("Bernoulli Distribution")
p_true = 0.7
p_dist = scipy.stats.bernoulli(p_true)
# variables using sympy
p = sympy.symbols('p', positive= True)
#probability function
df = p**x * (1-p)**(1-x)
parameterEstimate(p_dist, df, p_true,p)
```

    Bernoulli Distribution
    True parameter value: 0.7
    Likelihood Function: p**741*(1 - p)**259
    Estimated parameter value: 0.741



```python
print("Exponential Distribution")
p_dist = scipy.stats.expon()
p_true = p_dist.mean()
# variables using sympy
t = sympy.symbols('t', positive= True)

#probability function
df = (1/t) *  math.e ** (-x/t)

parameterEstimate(p_dist, df, p_true,t)

```

    Exponential Distribution
    True parameter value: 1.0
    Likelihood Function: 2.71828182845905**(-1022.71172198597/t)/t**1000
    Estimated parameter value: 1.02271172198597



```python
print("Normal Distribution")
mu_true = 2
sigma_true=0
p_dist = scipy.stats.norm(mu_true,sigma_true)
# variables using sympy
mu, sigma = sympy.symbols('mu, sigma', positive= True)

#probability function
df =  ((2 * math.pi* sigma*sigma) ** (-0.5)) * (math.e ** (-0.5 * (x-mu)* (x-mu)/(sigma * sigma)))
parameterEstimate(p_dist, df, mu_true,mu)

```

    Normal Distribution
    True parameter value: 2
    Likelihood Function: 8.12953716728196e-400*2.71828182845905**(1000*(2.0 - mu)*(0.5*mu - 1.0)/sigma**2)*sigma**(-1000.0)
    Estimated parameter value: 2.0



```python
print("Poisson Distribution")
k_true= 2
p_dist = scipy.stats.poisson(k_true)
k = sympy.symbols('k', positive= True)
df = k**x*sympy.exp(-k)/sympy.factorial(x)
parameterEstimate(p_dist, df, k_true,k)

```

    Poisson Distribution
    True parameter value: 2
    Likelihood Function: k**1997*exp(-1000*k)/142220199860881056663191396650367732478138060843772563671397683466569187541722759507915845631107969632545273741424688952787500633027625617630113037289315761090010517971364415451864150528618267221053183525752106609291883404681346679086957134357390537918919070304662927052260585891725831988996973817758666931146942020746209214900436818491150673157476389615461350353676686715592963879195841207050750154875803762161535927975936000000000000000000000000000000000000000000000
    Estimated parameter value: 1.997


bb013d7d288d0e6d9d54a28bd6e42f1d4ebb4634
ipynb
Jupyter Notebook
Statistics/MaximumLikelihoodEstimation.ipynb
bkgsur/FinanceModelingComputationWithPython
e526d53fc06776b91202c87005ace3e4f946e80c
MIT
Statistics/MaximumLikelihoodEstimation.ipynb
bkgsur/FinanceModelingComputationWithPython
e526d53fc06776b91202c87005ace3e4f946e80c
MIT
Statistics/MaximumLikelihoodEstimation.ipynb
bkgsur/FinanceModelingComputationWithPython
e526d53fc06776b91202c87005ace3e4f946e80c
MIT
Qwen/Qwen-72B
1. YES
2. YES