# M10. Amdahl's Law

The most useful corollaries to what is now known as *Amdahl's Law* are hardly profound. The notion of prioritizing the improvements *with the greatest bearing on the overall result* is almost common sense (which is maybe to suggest that it isn't common at all). Violations of the Law are prevalent (though not ubiquitous):

- Behavior characterized by the Britishism: "Penny wise, pound foolish"&mdash;people who are thrifty when it comes to small expenses but who are indulgent or forgetful of larger ones.
- Spending legislative energy on banning plastic straws when they constitute 0.02 *percent* (by mass) of annual plastic waste deposited in the oceans.

The original version of the Law, advanced by Gene Amdahl in 1967, was nowhere near as ambitious in its scope (nor did it pretend to be). It merely provided an upper bound on speedups afforded by one type of improvement, parallelization, and it was formulated as follows:

Suppose a fixed sequential workload of latency $W$ can be partitioned into a parallelizable fraction $p$ and a non-parallelizable fraction $1-p$. Then, the speedup in latency offered by running the workload on $n$ processors is given by 

\begin{align}
S(n) &= \frac{W}{\frac{pW}{n} + (1-p)W}\\
&= \frac{1}{\frac{p}{n}+(1-p)}
\end{align}

and hence we obtain the upper bound

$$\lim_{n\to\infty} S(n) = \frac{1}{1-p}.$$

Amdahl's Law is a 'real law' (much as Moore's Law is *not*), but it is somewhat naive and primitive in the sense that its operative assumption is that workloads can be partitioned into different regimes, which is often easier said than done. In fact, Amdahl's Law is sometimes seen being used to *back-calculate* the fraction $p$ of a workload exposed to a given improvement.  

One word of caution: the fraction $p$ referred to has nothing to do with the *frequency* of the improvable portion of a workload and everything to do with its *latency*. Things which occur infrequently may still be catastrophically time-intensive when they do occur. On the other hand, there are many things which *do* happen often and yet contribute little to overall execution time.  




76328778b3295307b089cf6516bb36c8abe806c0
ipynb
Jupyter Notebook
M10_Amdahl's_Law.ipynb
brekekekex/computer_organization_memoranda
d16dc251075d3da49aaf01f148676f857d05dc4b
Unlicense
2020-01-17T16:34:17.000Z
2020-02-23T22:06:07.000Z
M10_Amdahl's_Law.ipynb
brekekekex/computer_organization_memoranda
d16dc251075d3da49aaf01f148676f857d05dc4b
Unlicense
M10_Amdahl's_Law.ipynb
brekekekex/computer_organization_memoranda
d16dc251075d3da49aaf01f148676f857d05dc4b
Unlicense
Qwen/Qwen-72B
1. YES
2. YES