Write detailed note from this summary include all the equations (in a sudo latex form): Notes: LMs: Smoothing & Evaluation
Recap:
* N-gram language models use probability decomposition and smoothed estimates from corpora.
* Smoothing methods: Laplace, Lidstone, Interpolation.
NAME_1 Smoothing:
* Powerful and commonly used technique.
* Combines discounting, fertility, and interpolation.
Practical note:
* Use log probabilities to avoid multiplying small probabilities.
	p1 ×p2 ×p3 ×p4 =exp(logp1 +logp2 +logp3 +logp4 )
Evaluating a language model:
* Extrinsic evaluation: Measure performance on a downstream application.
* Intrinsic evaluation: Design a measure independent of applications.
Intrinsic evaluation setup:
* Divide data into train (80%), development (10%), and test (10%).
* Compute language models on training data.
* Set parameters (e.g., k for Lidstone smoothing or Interpolation weights) using development data.
* Test data should be used only once.
Perplexity of test data:
* Inverse probability of the test set, normalized by the number of words.
	Perplexity(W)=P(w1 w2 ...wN )−N1
* Lower perplexity is better.
Intuition of Perplexity:
* Average branching factor of the model.
* How many options are there at each stage?
Perplexity of test data under a language model:
* Compute probability under the respective language model (e.g., bigram model).
Intuitive interpretation of perplexity:
* Average branching factor of the model.
* How many words can follow a word on average?
Comparison of perplexity values:
*