\section{Introduction}

 Stellar Observations Network Group (SONG) is an initiative started in
Denmark to create a global network of highly specialized
1m telescopes aimed at doing time-domain astronomy. In particular the
goals are to: produce exquisite data for asteroseismic studies of stars
across most of the HR diagram (with focus on solar-like stars), and to
search for, and characterize, the population of low-mass planets in orbit
around other stars via the microlensing and radial--velocity methods.

 A ground-based network with a sufficient number of nodes will ensure
observations with a high duty-cycle, needed to obtain stellar oscillation
spectra with high frequency precision and without aliasing problems.
Furthermore, the light-curve anomalies in microlensing events
occur at unpredictable times and thus, to find and characterize these,
near-continuous observations are needed.

 The group behind SONG has obtained funding for the design and
construction of a full prototype network node which shall be completed
in late 2011. In the following we describe some of the aspects of the
ongoing work and the expected performance of the prototype.

\section{Observational goals}

Below we shall briefly account for the main goals and requirements that the
SONG instruments must fulfil in order to meet the requirements to
do asteroseismology and measure microlensing light curves.

\subsection{Asteroseismology}

To study efficiently the oscillations of solar-like
stars, the best strategy for ground-based  observations is to measure
the change in radial velocity of their surface. This requires a high radial
velocity precision in the few  m/s range.
For the best ground-based instruments
this level of precision is now routine, and some are at the sub-meter per
second precision level \citep{harps, iodine1}
over short timescales. For the SONG project the aim is
to achieve a velocity precision better than 1m/s for a $V=0$ star per
minute of observation. This requires an efficient, high-resolution
spectrograph and a CCD camera with a fast readout in order to allow a high
observing duty cycle.

Our aim is to obtain a network duty cycle near 80\%. This will be achieved
by having 8 nodes distributed at existing northern and southern hemisphere
sites, and well distributed in longitude. Experience from BiSON
\citep{bison1}
show that it is realistic to reach this level with 6--8 stations,
consistent with the results of \citet{mosser}.
It is clear that targets in the equatorial zone will have the highest degree
of coverage since these can be observed from both northern and southern
nodes. With both northern and southern sites in the network, full-sky
coverage will be possible.

\subsection{Gravitational microlensing}

For the study of microlensing events towards the bulge of the Milky Way,
SONG will employ the lucky-imaging method \citep{lucky1}. By observing the
target field with a CCD camera which can read out at high speeds
($\approx30$Hz), and then only selecting
the images of best quality for subsequent co-addition, it is possible
to obtain images with high spatial resolution. Since the bulge fields are
in general quite crowded, this offers a big advantage in the achievable
photometric precision and depth. Observations for this purpose only
require a small field of view, and the current design foresees a field
of 46\arcsec$\times$46\arcsec.
The microlensing observations have similar requirements as the asteroseismic
observations with respect to the observing duty-cycle. \citet{lensing1}
discusses the prospects for microlensing studies with SONG in more detail.

\subsection{Other possibilities}

One of the possibilities we are considering for SONG is to use the
spectrograph to observe the oscillations of the Sun during daytime. This
is done by pointing the telescope to the blue sky and measuring the
velocities in exactly the same way (through the iodine cell) as the
stars are observed at night. In this way we will observe
the sun-as-a-star and complement the existing facilities (ground and
space) for solar observations. See \citet{sun1} for an example of this
with the HARPS and UCLES spectrographs.

\section{Layout of the instrumentation}

The instrumentation at each site consists of a 1m telescope equipped with
a high-resolution spectrograph at a Coud{\'e} focus and two lucky-imaging
cameras at one of the two Nasmyth foci. This allows simultaneous two-colour
imaging.

\subsection{Dome and enclosure}
The telescope will be housed in a dome of approximately 4.5m diameter,
and the Coud{\'e} room will be located in a 20 foot shipping container with
a significant level of insulation. Computers and hard drives for observatory
control and data reduction will be located in this container as well. The
use of a container for housing instrumentation and computers allows
cost savings and provides for a rather small footprint,
since the only permanent buildings needed will be the telescope concrete
pier and the footings on which the container is attached to the ground. A
separate concrete foundation will carry the support for the spectrograph such
that it is mechanically de-coupled from the container.
Figure~\ref{fig:pier} shows the general layout.

Since any potential SONG node should be at an existing observatory, access
to electrical power and internet will already be available. A weather station
with a cloud monitor is also part of the instrumentation.
The expected date for arrival of the telescope to the first site is mid-2010.

\begin{figure}[!ht]
\plotone{grundahl_fig1.ps}
\caption{
\label{fig:pier}
The basic structures of a SONG node, comprising a concrete pier
for supporting the telescope, a 20 foot container and a dome support
structure. In the final version the structure carrying the dome will be
approximately 1m higher with the dome floor at the level of the container
roof. Side-ports will be installed in the extra 1m height to improve
nighttime ventilation of the dome. The inside of the container will be
split in two compartments, one for the instruments nearest to the telescope
pier, and one at the other end for control computers and other electronics. }
\end{figure}

\subsection{Telescope optics and imaging}

Lucky-imaging for the microlensing science will be carried out with two
cameras located at one of the two Nasmyth stations. We decided to not
place these at the Coud{\'e} focus since this would lead to less than optimal
image quality (many optical surfaces), lower overall efficiency and higher
costs. At the Nasmyth station a mirror slide with several positions will
send the light to the two imaging cameras, or allow the light to pass on
to the Coud{\'e} train.

For lucky-imaging it is important to have correction for atmospheric
dispersion (ADC), as well as field de-rotation (the telescope is alt-az
mounted). The field de-rotation will be done with an optical de-rotator
of the Abbe-K{\"o}nig type for both imaging and spectroscopy. The imaging
at Nasmyth will be in two wavelength regions, with a split at 6700{\AA}. In
the ``short'' wavelength channel a beam splitter will be placed, which
sends a small fraction of the light to a (cheap) CCD camera for continuous
focus monitoring. In this way the telescope will always be at its optimal
focus. Both of the two imaging channels will be equipped with filter wheels.

The pixel scale for the lucky-imaging cameras is $\sim0\farcs09$ in order
to provide adequate sampling (in the red channel) for  nearly
diffraction-limited imaging. At a seeing of 1\arcsec\, FWHM, the 1m SONG
telescope has $D/r_0$ of 5--6  at a wavelength of 8000{\AA}. In this regime
the lucky-imaging method works extremely well, which implies that a very
significant improvement in image quality can be expected via the use of
this method. At the best observing sites, imaging with a FWHM close to
0\farcs3 can be expected.

We should note that the CCD cameras will of course also offer the
possibility to do photometry of other objects, such as variable stars
and gamma-ray bursts.

\subsection{The Coud{\'e} train and spectrograph focal plane}

The second main instrument for SONG is the spectrograph which is located
at a Coud{\'e} focus. By removing the mirror that directs the light to the
Nasmyth station the light from M3 will instead go to other mirrors
(M4, M5, M6, M7 and M8, see Fig.~\ref{fig:optical_layout}) and ultimately
end in the instrument
container where the spectrograph is located. Radial velocities are measured
using the iodine technique \citep{iodine1}. This implies that only a limited
wavelength range, 5000--6200{\AA} is needed (discussed in more
detail below). With such a short wavelength interval to be ``transported''
to the Coud{\'e} focus, highly optimized anti-reflective and
reflective coatings can be used. This results in the total efficiency of
the Coud{\'e} train being nearly 95\%.

At the pre-slit assembly several functions will be available; these include
calibration light (ThAr and halogen lamps), slit viewing, focus monitoring,
tip-tilt correction and telescope pupil monitoring and correction. In
addition to this, a temperature stabilized iodine cell is also available
for providing an accurate velocity reference. The spectrograph will be
located in a temperature-stabilized box for improved stability.

\begin{figure}[!ht]
\plotone{grundahl_fig2.ps}
\caption{
\label{fig:optical_layout}
A schematic layout for the SONG prototype node. The relative sizes
of the components are not completely correct, but only intended to show the
layout. }
\end{figure}

The light path from M4 to the Coud{\'e} room will be in vacuum tubes -- this
will ensure a minimal level of maintenance of the optics as well as a low
impact of thermal differences along the light path from the instrument
room to the dome. The instrument room is expected to be kept at a
temperature of 15--20\deg C.

\subsection{Auxiliary instrumentation}

The telescope has two Nasmyth ports of which only one will be used initially.
In order to allow for future instrument upgrades, the telescope will, however,
be designed such that installation of a rotating tertiary mirror will be
relatively simple. This allows auxiliary instrumentation to be placed at the
second platform. The secondary mirror is designed to allow a field of view of
10\arcmin$\times$10\arcmin.

\section{Spectrograph}

For the asteroseismic observations, the Coud{\'e} spectrograph is the
principal in\-stru\-ment. In order to determine the required high-precision
velocities an iodine cell will be used for wavelength reference. The
spectro\-graph will have a (2 pixel) resolution of 120.000 for a 1\arcsec\,
slit and employ an R4 echelle and a collimated, F/6, beam diameter of 75mm.
The slit is 10\arcsec\, long, and via the instrument rotator it can
be rotated in any desired direction allowing the possibility of doing
asteroseismology for close binary stars such as $\alpha$\,Cen\,A and B.

The spectrograph is designed to cover a wavelength range
of 4800--6800{\AA}. The orders are fully covered at wavelengths shorter
than 5200{\AA}; in the future a larger detector will allow full coverage
of the spectral orders.

The spectrograph has been designed by P. Span{\`o}, with inspiration from
the design of UVES, HARPS and other modern high-resolution spectrographs.
It is very compact, roughly $50\times90\times15$cm (without mounts), which
is expected to be an advantage with respect to temperature control.

In order to provide a well behaved instrumental profile, emphasis was put
on the design of the spectroscopic camera. This resulted in a design
with an instrumental profile which is nearly diffraction limited over
the entire area of the detector. The detector system will be a 2K$\times$2K
system from Andor of Belfast. This camera has an advanced peltier cooling
system that does not need liquid nitrogen or closed cycle cooling, and
furthermore the vacuum is ``permanent'' -- so the camera is essentially
maintenance free. The electronics allow full-frame readout in 5s with
a readout noise less than  10\,electrons, thus giving only very small overheads
for the spectroscopic time-series observations. The CCD camera is capable
of reading out frames at a speed of 5Mpix.

\subsection{Spectrograph efficiency and performance}

Due to the small wavelength range covered by the spectrograph it is possible
to employ coatings with very high efficiency for both reflective and
transmissive optics. Many optical companies can deliver optics with
a reflectivity higher than 99.5\% and anti-reflection coatings with
transmissions better than 99.5\% over the spectrograph design wavelength
interval. Our calculations of the spectrograph efficiency (no slit or
detector included), show that a throughput in excess of 50\% can be expected.
With a relatively small telescope and a 75mm beam the slit product is
a generous 120.000, much higher than for larger telescopes, implying
smaller slit losses under typical seeing conditions.

\begin{figure}[!ht]
\plotone{grundahl_fig3.ps}
\caption{
\label{fig:velocity}
The expected velocity precision for SONG. The calculation is based
on the assumption that the velocity precision scales with signal-to-noise
ratio as found by \cite{iodine2}. We note that at the present time the
{\it i}SONG iodine reduction software has achieved a level of precision of
$\approx0.8$m/s on UVES archival data for $\alpha$\,Cen\,A; thus with the
current generation of the software a 1m/s precision can realistically be
expected.
}
\end{figure}

We have carried out calculations of the {\it total} efficiency of the
spectroscopic system, including all sources of light loss (from atmosphere
to detector) and these
indicate that in 1\arcsec\, seeing the total system throughput is in
excess of 8\%.

For the design of the prototype we have included provisions for continuous
control of the location of the telescope pupil on the grating as well as
allowing tip-tilt correction to provide a very stable illumination of the
slit.

On the assumption that our data reduction code reaches a level of performance
as that of \citet{iodine2} for $\alpha$\,Cen\,A (their Fig.~5) we
arrive at a predicted velocity precision of better than 1m/s per minute of
observation for stars brighter than $V\approx1$. Stars with lower metallicity,
shallower lines and higher rotational velocities will not allow such a
high precision to be obtained. In Fig.~\ref{fig:velocity}  the
calculated velocity precision
vs. stellar magnitude is shown. For stars where a sample time of 1\,minute
(5s read $+$ 55s exposure) is sufficient it will be feasible to carry
out asteroseismic campaigns to $V\approx6$.

It should be noted that the iodine cell is not located permanently in the
light path; therefore ``conventional'' spectroscopy with ThAr calibration
exposures prior to, and after, science exposure will be possible.
Thus programmes
which do not demand the highest velocity precision are possible in the
same way as for other telescopes.

\subsection{Iodine-based velocities with {\it i}SONG}

We have developed an IDL-based data reduction code, called {\it i}SONG, for
the extraction of
velocities based on iodine-cell observations. At this point, the code has
been tested on the \citet{iodine2} UVES observations of {$\alpha$}\,Cen\,A
(the raw data were retrieved from the ESO archive) and the SARG observations
of $\mu$\,Herculis \citep{muher}. For
the analysis of the {$\alpha$}\,Cen\,A data we achieve a velocity precision of
77cm/s per data point; only slightly poorer than the 70cm/s achieved by
Butler et al. on the same 688 frames. The current (incomplete) analysis of
the data for $\mu$\,Her indicates that we reach a similar precision as that
of \citet{muher}. For an example of velocity time-series with
this code we refer the reader to \citet{grundahl1}.

\section{Summary and status of SONG}

As of late 2008 the SONG project has obtained full funding for the
development of a prototype node. Our design revolves around a
high-performance 1m telescope equipped with a dual-colour lucky-imaging
camera system and a highly efficient high-resolution spectrograph which
will allow 1m/s precision velocities to be obtained for the brightest stars
in the sky. Better than 10m/s precision is expected for stars brighter than
$V=6$ per minute of observation.

SONG is now in its detailed design phase. The optical design is completed
thus allowing final design of the remaining elements. Mechanical design
of the spectrograph, Nasmyth focal plane and pre-slit systems is
on-going. These systems, with control software, will be tested and integrated
in Aarhus during 2010, with expected delivery to the telescope on the
first site (Tenerife) in late 2010. The telescope is expected to be
delivered to the site in mid 2010.

The ongoing prototype work aside, the main challenge for the coming years
will be to develop the international consortium that will enable the setup
of the full network. Work towards this purpose will be kick-started with
a workshop in Aarhus in March 2009 (see {\tt http://astro.phys.au.dk/SONG}).

\acknowledgements

Funding for the SONG project is provided through substantial grants from
the Villum Kann Rasmussen Foundation, The Danish Natural Sciences Research
Council and the Carlsberg foundation. FG wishes to acknowledge many fruitful
conversations with Michael I. Andersen. Henrik K. Bechtold and Anton Norup
S{\o}rensen are thanked for providing Figs 1 and 2. G. Marcy is thanked
for advice and encouragement during the development of the {\it i}SONG
code. S. Leccia and A. Bonanno kindly permitted the use of the raw
$\mu$\,Her\, data for further tests of the code.


\section{Introduction}
Physical processes unfold over time. Our minds grasp physical mechanisms largely via narrative. So it is not surprising that some of the most vivid physics demonstrations also play out over time. Simulations of physics that unfold over time are similarly powerful; interactive simulations are better; and simulations created by the student can be best of all. This view is gaining ground in introductory courses
\cite{Bchab15a}, but the benefits of animated simulation extend farther than this. Here we wish to show that the  behavior of strongly nonequilibrium statistical systems can be illustrated via stochastic simulations that are simple enough to serve as undergraduate projects. Recently developed, free, open-source programming resources sidestep the laborious coding chores that were once required for such work.
In particular, we believe that the error-correction mechanism known as kinetic proofreading can be more clearly understood when a student views its linear temporal sequence, as opposed to solving deterministic rate equations. Coding this and other simple processes opens the door for the student to study other systems, including those too complex for the rate-equation approach to yield insight.

\section{Double-well hopping}
\subsection{The phenomenon}
We tell students that a simple chemical reaction, for example isomerization of a macromolecule, can be regarded as a barrier-passing process. A micrometer-size bead in a double optical trap serves as a mesoscopic model system with this character \cite{Simon:1992cr}, and it is well worthwhile for students to watch it undergo a few dozen sharp transitions in between episodes of Brownian motion near its two stable positions (see supplementary video \ref{sv1}).
A simple model for this behavior states that the hopping transitions occur at random times drawn from an exponential distribution. That is, many rapid transitions are interspersed with a few long pauses\cut{To put this a different way, "activated processes" which involve crossing barriers significantly greater than $k_B T$ tend to be rare, not slow.}.

\subsection{Simulation: Waiting times drawn from an exponential distribution\label{ss:wtde}}
With this physical motivation, students can explore how to generate simulated waiting times of the sort just described. Any computer math system has a pseudorandom number generator that generates floating-point numbers uniformly distributed between 0 and 1. Many students are surprised (and some are intrigued), to learn that applying a nonlinear function to samples from a random variable yields samples with a different distribution, and in particular that $y=-\tau\ln x$ is exponentially distributed, with mean $\tau$, if $x$ is uniform on (0,1] \cite{Bnels15a}\cut{This construction can be nicely illustrated with a graphic that shows its generality, especially the cdf approach.}.

Starting from that insight, it takes just one line of code to generate a list of simulated waiting times for transitions in a symmetric double well; finding the cumulative sums of that list gives the actual transition times (see supplementary computer code \ref{sc1}).

The freely accessible VPython programming system (or its Web-based version Glowscript) makes it very easy to create an animation of an object whose spatial position is supplied as a function of time \cite{Evpyt17a}. The only challenging part is to pass from a list of irregularly-spaced transition times to particle positions at each of many (regularly-spaced) video frames (see supplementary computer code \ref{sc1}). The payoff is immediate: Visually, the simulated hopping has a very similar character to the actual Brownian hopping of a bead in a double trap (see supplementary video \ref{sv2}).

\subsection{Upgrade to 1D random walks}
It may be of interest to make a small modification of the code: Instead of hopping between two wells (reversing direction on every step), consider one-dimensional diffusion on a symmetric many-well potential, for example, one of the form $U(x)=\sin(x)$. In such a potential, for each transition the system must also decide whether to increase or decrease a ``position'' coordinate. The resulting random walk will display the same long-time scaling behavior as any unbiased 1D walk, but with trajectories that undergo hops at random times, not periodic steps as in the simplest realization \cite{Bnels15a}.

\section{Birth-death process}
We can now generalize from situations with essentially only one kind of transition (or two symmetric kinds), to the more interesting case where several inequivalent choices are possible, and where the relevant probabilities depend on the current state. This general situation can describe a chemical reaction that requires, and depletes, molecules of some substrate.

Most science students know that living cells synthesize each of their messenger RNAs ({mRNAs}) from a single copy (or a small fixed number) of the corresponding gene. Even a constitutive (unregulated) gene must wait for the transcription apparatus to arrive, bind, and begin transcription \cite{Balbe09a}. We consider a situation in which that apparatus is in short enough supply that this waiting is the primary determinant for the initiation of transcription. Once a mRNA transcript has formed, it has a limited lifetime until it is degraded by other cellular machinery. We assume that this process, too, relies on chance encounters with degradation enzymes. Moreover, each of many species of mRNA must all share the attentions of a limited number of degradation enzymes, so each mRNA copy has a a fixed probability per unit time to be removed from the system.

The physical hypotheses in the preceding paragraph amount to a model called the ``birth-death process,'' which has many other applications in physics and elsewhere. As in the 1D walk, we characterize the system's state by an integer, in this case the population of the  mRNA of interest. Synthesis is a transition that increases this number, with a fixed probability per unit time $k_\mathrm{s}$ (called the ``mean rate'' of synthesis). Degradation is a transition that decreases it, with a probability per unit time that is the current population $n$ times another constant $k_\mathrm{d}$ (the ``rate constant'' for degradation)\cut{I suggest including the governing differential equation.  That will help make these statements concrete for students who should be comfortable with dif eqns.}.

\subsection{Simulation\label{ss:bdp.s}}
D.~Gillespie extended and popularized a simple but powerful method, the ``stochastic simulation algorithm,'' for simulating systems of this sort \cite{Gillespie:1976p1686}. In the case just described, the algorithm repeatedly executes the following steps (see supplementary computer code \ref{sc2}):\begin{itemize}
\item Determine the probability per time $k_{\mathrm{ tot}}$ for \emph{any} of the allowed transitions to occur by summing all the mean rates. In a birth-death process, we have $k_\mathrm{tot}=k_\mathrm{s}+nk_\mathrm{d}$.
\item Draw a waiting time from the exponential distribution with mean given by the reciprocal of $k_{\mathrm{ tot}}$, via the method in \ssref{wtde}.
\item Determine which of the allowed processes happens at that transition time. In the birth-death process, we make a Bernoulli trial with probability $p=k_{\mathrm s}/k_\mathrm{tot}$ to increase population $n$ by one, and $1-p$ to decrease it.
\item Update $n$ and repeat.\end{itemize}
The beauty of this algorithm, besides its correctness \cite{Gillespie:1977p1466}, is that no computation is wasted on time steps at which nothing happened: By definition, there is a state transition at \emph{every} chosen time.

\subsection{Convergence to the continuous, deterministic approximation\label{ss:ccda}}
Students will probably find it reasonable that, when $n$ is sufficiently large, we may neglect its discrete character. Students who have been exposed to probability ideas may also find it reasonable that in this case, the relative fluctuations of $n$ from one realization to the next will be small, and so $n$ effectively behaves as a continuous, deterministic variable, subject to the differential equation ${\mathrm d} n/{\mathrm d} t=-k_\mathrm dn+k_\mathrm s$. That equation predicts exponential relaxation from an initial value $n_0$ to the steady value $n_*=k_\mathrm{s}/k_\mathrm{d}$ with e-folding time $k_\mathrm{d}$:
\begin{equation}n(t)=n_*+(n_0-n_*)\ex{-k_\mathrm{d}t}.
	\label{e:bdcts}\end{equation}
The simulation bears out this expectation (\frefpanel{f1}{a,b}).
\begin{figure}\begin{center}\includegraphics{g178BDsim3.pdf}\end{center}
\caption{\label{f:f1}\small
\arttitle{Behavior of a birth-death process.}
\capitem aThe \capelement{bumpy traces} show two examples of simulated time series with $k_\mathrm{s}=12\,\ensuremath{\mathrm{min}}^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$, $k_\mathrm{d}=0.015\,\ensuremath{\mathrm{min}}^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$, and hence $n_*=800$. The initial population was $n_0=3200$.
The \capelement{smooth trace} shows the exponential relaxation predicted by the continuous, deterministic approximation (\eref{bdcts}).
\capitem bAfter the system comes to steady state, there is a tight distribution of $\NX$ values across 100 runs of the simulation (\capelement{bars})\cut{Bars are hard to see.  I suggest reducing vertical scale.  Also, on both axes, label multiple tick marks to avoid ambiguity of values.}. The \capelement{curve} shows the Poisson distribution\Nindex{Poisson!distribution}{} with mean $n_*$
for comparison.
\capitem{c,d}The same but with $k_\mathrm{s}=0.15\,\ensuremath{\mathrm{min}}^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$ and $n_0=40$. Although individual instances (runs of the simulation) deviate strongly from the continuous, deterministic approximation, nevertheless the sample mean of the population $\NX(t)$ over 150 runs does follow that prediction\cut{ (\capelement{green trace} in c)}. The distribution of steady-state fluctuations is again Poisson (\capelement{curve} in d).
(See also \cite{Bnels15a}.)
\instructor{Made by \<transcrip2rxn.py, viewTransc.py>.}
}\end{figure}

Actually, mRNA populations in living cells are often \emph{not} large. Nevertheless, although individual realizations of $n(t)$ may differ significantly, the \emph{ensemble average} of many such trajectories does follow the prediction of the continuous/deterministic idealization (\fref{f1}c). Within individual cells, there will be significant deviation around that mean behavior (\fref{f1}c again). In particular, the ``steady'' state will have fluctuations of $n$ that follow a Poisson distribution (\frefpanel{f1}{d}). That key result is more memorable for students when they discover it empirically in a simulation than it would be if they just watched the instructor prove it with abstract mathematics (by solving a master equation \cite{Bnels15a}).

State fluctuations of the sort just mentioned may suffice to pop a more complex system out of one ``steady'' state and into a very different one. Indeed, even the simplest living cells do make sudden, random state transitions of this sort. Such unpredictable behavior, not seen in the differential-equation approach, seems to be useful to bacteria, implementing a population-level ``bet hedging'' strategy \cite{Choi:2008p2537,Eldar:2010p9554,Lidstrom:2010gj}.

A real bacterium is not simply a beaker of reagents. Bacteria periodically divide, partitioning a randomly chosen subset of each mRNA species into each daughter cell. That extra level of realism is hard to introduce into an analytical model, but straightforward in a simulation. The results are similar to the ones just described, with a larger effective value of the clearance rate constant \cite{Bnels15a}.

\subsection{Upgrade to cover bursting processes\label{ss:ucbp}}
Bacteria are supposedly simple organisms. The birth-death process is simple, too, and it fits with the cartoons we see in textbooks. So it is interesting to follow the recent discovery that the model makes quantitative predictions for mRNA production that were experimentally disproven \cite{Golding:2005p1478,Golding:2008p1486,Bnels15a}.

For example, recent advances in single-molecule imaging permit the direct measurement of $n(t)$ in individual cells, and disproved the model's prediction that the distribution of $n$ in the ``steady'' state should be Poisson (\fref{f2}{c}). Researchers found, however, that a simple modification of the birth-death model could accommodate these and other discrepant data. The required extension amounts to assuming that mRNA transcripts are generated in \emph{bursts,}  that the bursts themselves are initiated with a fixed probability per unit time, and that once initiated, a burst is also terminated with a fixed probability per unit time. Although this ``bursting model'' has two additional parameters compared to the original birth-death model, nevertheless it was overconstrained by the experimental data, so its success was a nontrivial test \cite{Golding:2008p1486,Bnels15a}. Remarkably, detailed biochemical mechanisms for this behavior were found only some years after its indirect inference \cite{Chong:2014p15102,Sevier:2016ew,Sevier:2018ek,klindz}, an important lesson for students to appreciate.
\begin{figure}\begin{center}\includegraphics{g180goldingF2abcOutline.pdf}\end{center}
\caption{\label{f:f2}\small
\arttitle{Indirect evidence for transcriptional bursting.}
\capitem{a}\capelement{Symbols:} Time course of the number of {mRNA}\FNindex{RNA!messenger (mRNA)} transcripts in a cell, $\Nx(t)$, averaged over 50 or more cells in each of three separate, identical experiments. Data from the three trials are shown with three different symbols.
All of the cells were induced\FNindex{induction} to begin gene expression at  time zero.
The \capelement{dashed curve} shows a fit of the birth-death (BD) process (\ereffarcomma{bdcts}) to data, determining the apparent synthesis rate $\NSrate\approx\BDsynVAL/\ensuremath{\mathrm{min}}$ and clearance rate constant $\NrateconX{\Nzilch}\approx0.014/\ensuremath{\mathrm{min}}$. The \capelement{solid curve} shows the corresponding result from a computer simulation\FNindex{simulation!random process (Gillespie algorithm)!bursting model} of the \FIindex{bursting} model discussed in \ssref{ucbp}  (see also \cite{Bnels15a}).
\capitem{b}Semilog plot of the fraction of observed cells that have zero copies of {mRNA} versus elapsed time. \capelement{Symbols} show data from the same experiments as in \capitemrefer{a}. \capelement{Dashed line:} The \FIindex{birth-death process} predicts that initially $\Nprob_{{{\mathrm{\Ndrvbare}}}\rm(t)}(0)$ falls with time as  $\exp(-\NSrate t)$, where $\NSrate$ has the value found by fitting the data in \capitemrefer a. (Degradation was negligible in this experiment.)
The experimental data instead yield initial slope $-0.028/\ensuremath{\mathrm{min}}$. \capelement{Solid line:} Computer simulation of the bursting model.
\capitem{c}Experiments were performed at each of many different levels of gene induction. For each level, a \capelement{cross} shows the variance of the late-time mRNA population $n_\infty$ versus its sample mean. This log-log plot of the data shows that they fall roughly on a line of slope $1$, indicating that the \FIindex{Fano factor} $(\variance\Nx)/\exv{\Nx}$ is roughly a constant. The simple \FIindex{birth-death process} predicts that this constant is equal to 1 (\capelement{dashed line}),  because mean equals variance for any Poisson distribution, but the data instead give the value $\approx5$. The \capelement{circle} shows  the result of the same simulation shown in \capitemrefer{a,b}.
\artfrom{Figure adapted from \cite{Bnels15a}; experimental data from \protect\cite{Golding:2005p1478}\pnonly{Actually taken from \protect\cite{Golding:2008p1486}}\cut{; see \dref{bursting}}.}\instructor{Made by \texttt{bursting/mRNAsimMain.m} et al.}
}\end{figure}

	\section{Kinetic proofreading}\subsection{Word model\label{ss:wmkp}}
\instructor{``Already in 1957, before the advent of molecular biology, Linus Pauling postulated that this intrinsic (thermodynamic) limitation to the selection of similar amino acids would give rise to very large amino acid substitution errors in intracellular proteins [Pauling L: The probability of errors in the process of synthesis of protein molecules. Festschrift Arthur Stoll. Birkh\"ause Verlag; 1957:597--602.]'' -- Johansson etal\\
	``The free
energy differences due to base pair mismatches between the
mRNA codon and the tRNA anticodon are too small to provide
the observed high accuracy of tRNA selection (Ogle and Ramakrishnan,
2005; Xia et al., 1998), even if kinetic proofreading
(Hopfield, 1974; Ninio, 1975) is taken into account'' -- \cite{Savir:2013ez}}%
Ask a student, ``What is the big secret of life?'' and the answer will probably be ``DNA,'' or perhaps ``evolution by natural selection.'' Indeed, DNA's high, but not perfect, degree of stability underlies life's ability to replicate with occasional random modifications. But it is less well appreciated that the \emph{stability} of a {molecule} of DNA does not guarantee the \emph{accuracy} of its {replication} and {transcription.} There is another big secret here, just as essential to life as the well known ones. In fact, a wide range of molecular recognition events must have extremely high accuracy for cells and their organisms to function. Think of our immune cells, which must ignore the vast majority of antigens they encounter (from ``self''), yet reliably attack a tiny subpopulation of foreign antigens differing only slightly from the self.

Translation of mRNA into proteins is an emblematic example of such a puzzle. It is true that artificial machines now exist that can read the sequence of mRNA. Then another artificial machine can take the resulting sequence of base triplets, decode it, and synthesize a corresponding polymer of amino acids (a polypeptide), which in some cases will then fold into a functional protein without further help. But the cells in our bodies, and even bacteria, do these jobs reliably \emph{without}
those huge and expensive machines, despite the incessant nanoscale thermal motion!

Merely intoning that a wonderful molecular machine called the ribosome accomplishes this feat doesn't get us over the fundamental problem: At each step in translation, the triplet codon at the ribosome's active site fits one of the many available transfer RNA (tRNA) species \emph{somewhat} better than it fits the other 19 options.
But the binding energy difference, which quantifies ``somewhat better,'' only amounts to two or three hydrogen bonds. This translates into a fraction of time spent bound to the wrong tRNAs that is about $1/100$ times as great as the corresponding quantity for the correct amino acid \cite{Bphil08a}. If the fraction of incorrect amino acids incorporated into a polypeptide chain were that high, then \emph{every} protein copy longer than a few hundred amino acids would be defective!

In fact, the error rate of amino acid incorporation is more like $10^{-4}$. The fact that this figure is so much smaller than the one seemingly demanded by thermodynamics remained puzzling for decades. After all, the ribosome is rather complicated, but it is still a nanoscale machine. \emph{Which} of its features could confer this vast improvement in accuracy?

J.~Hopfield and J.~Ninio proposed an elegant physical mechanism, based on a  known but seemingly pointless feature of the ribosome \cite{Hopfield:1974p15095,Ninio:1975vv,Bphil08a}. To explore it, we begin by paraphrasing a metaphor due to U.~Alon \cite{Balon06a}. Imagine that you run an art museum and wish to find a mechanism that picks out Picasso lovers from among all your museum's visitors. You could open a door from the main hallway into a room with a Picasso painting. Visitors would wander in at random, but those who do not love Picasso would not remain as long as those who do. Thus, the concentration of Picasso lovers in the room would arrive at a steady value (with fluctuations, of course) that is enriched for the desired subpopulation.

To improve the enrichment factor further, you could hire an employee who occasionally closes the door to the main hallway, stopping the dilution of your enriched group by random visitors. Then open a new exit doorway onto an empty corridor. Some of the trapped visitors will gratefully escape, but die-hard Picasso lovers will still remain, leading to a second level of enrichment. After an appropriate time has elapsed, you can then reward everyone still in the room with, say, tickets to visit the Picasso museum in Paris.

The original authors realized that in the ribosome, the initial, reversible binding of a tRNA was followed by a transformation analogous to closing the door in the preceding metaphor. This transformation involved hydrolysis of a GTP (guanosine triphosphate) molecule complexed with the tRNA, and hence it was nearly \emph{irreversible,} due to the highly nonequilibrium concentration of GTP, compared to the hydrolysis products GDP and \ensuremath{\mathrm P_\mathrm i}{} (inorganic phosphate).  Such hydrolysis reactions were well known to supply the free energy needed to drive otherwise unfavorable reactions in cells, but here their role is more subtle.

Hopfield and Ninio were aware that after the hydrolysis, incorporation of the amino acid was delayed and could still be preempted by unbinding of the tRNA complex. The existence of this pathway had previously seemed wasteful: An energy-rich GTP had been ``spent'' without anything ``useful'' (protein synthesis) being done\cut{and indeed the hydrolysis actually created a delay and opportunity for reversing the step that had just been achieved!}. On the contrary, however, the authors argued that this second step implemented the mechanism in the art museum metaphor, giving the ribosome an independent second chance to dismiss a wrong tRNA that accidentally stayed bound long enough to progress to this stage. After all, spending some GTPs may be a modest price to pay compared to creating and then having to recycle an entire defective protein.

Hopfield coined the name ``kinetic proofreading'' for this mechanism, but we will refer to it as the ``classic Hopfield--Ninio'' (HN) mechanism because the original term is somewhat misleading. In chemical reaction contexts, a ``kinetic'' mechanism generally implies bias toward a product with lower activation barrier, even if it is less stable than another product with higher barrier. This preference is most pronounced at high, far-from-equilibrium catalytic rates \cite{Sartori:2013fv}. In contrast, the classic HN proofreading model involves two sequential thermodynamic (quasiequilibrium) discriminations\cut{, with the second acting on the biased output of the first}. Moreover, these discriminations take place prior to reading even the very next codon, in contrast to editorial proofreading, which generally happens after an entire manuscript is written. (Our choice of term also distinguishes the classic scheme from later models that are sometimes also called ``kinetic proofreading.'')

The qualitative word-model given earlier in this section may seem promising. But the corresponding kinetic equations make for difficult reading and understanding. Better intuition could emerge from a presentation that stays closer to the concrete ideas of discrete actors randomly arriving, binding, unbinding, and so on, visibly implementing the ideas behind the ``museum'' metaphor. The following sections will argue that stochastic simulation can realize that goal.
In a nutshell,
\begin{quote}\textsl{An effectively irreversible step, or at least a step far from equilibrium, gives rise to enhanced accuracy. The free energy of GTP hydrolysis is the price paid for this accuracy.}\end{quote}

It would also be valuable to confirm a key result of the analytic approach, which predicts that the enhancement of accuracy depends on GTP, GDP, and \ensuremath{\mathrm P_\mathrm i}{} being held far from chemical equilibrium, so that the hydrolysis step is nearly irreversible (the ``door shuts tightly'' in the museum metaphor). In fact, the model predicts \emph{no enhancement} of accuracy when this chemical driving force is low  \cite{Hopfield:1974p15095}. Far from equilibrium, however, the predicted error fraction can be as low as the \emph{square} of the equilibrium value (or even a higher power if multiple rounds of sequential testing are employed). \pnonly{CLINCH}

\subsection{A single ribosome in a bath of precursors\label{ss:srbp}}
This section's goal is to formulate the word-model of \ssref{wmkp} in the context of mRNA translation, then set up a stochastic simulation (see also \cite{Ezuck17a}). Later sections will show how students can explore the expectations raised at the end of the preceding section.

\begin{figure}\hbox to 6in{\includegraphics{ribocartoon.pdf}
\qquad\includegraphics{riborx.pdf}}
\caption{\label{f:f4}\small\arttitle{Two representations of the classic Hopfield--Ninio mechanism.}
\capitem aTraditional cartoon expressing the catalytic cycle of the ribosome (after \cite{Banerjee:2017en}).
\capitem bCorresponding kinetic diagram.
The large pale arrows indicate the net circulation in each cycle under cellular conditions, where \ensuremath{\mathrm{GTP}}{} is held out of equilibrium with \ensuremath{\mathrm{GDP}}{} and \ensuremath{\mathrm P_\mathrm i}. The symbol R denotes a ribosome complexed with mRNA\PNrevBfn{ with first binding site either empty or bound to a GTP complex}{}; R$^*$ is the corresponding complex ``activated'' by GTP hydrolysis. At far right, R indicates the ribosome with  one additional amino acid added to the nascent polypeptide chain. The classic Hopfield--Ninio proofreading model assumes that unbinding rates $k\ssr c$ and $\ell\ssr c$ are smaller than their mismatched counterparts $k\ssr w$ and $\ell\ssr w$, but that other constants are all equal for correct and wrong tRNA.
}\end{figure}
We will assume that a single ribosome is complexed with a single mRNA and has arrived at a particular codon. This complex sits in an infinite bath containing several free, dissolved species at fixed concentrations (\fref{f4}a): \begin{itemize}
\item \ensuremath{\mathrm{C}}{} denotes \underline correct tRNA (that is, the species that matches the codon currently being read), loaded with the corresponding amino acid. We will neglect the possibility of a tRNA being incorrectly loaded; accurate loading is the concern of another proofreading mechanism that we are not  studying now \cite{Hopfield:1976p14958,Yamane:1977p14957}.
\item \ensuremath{\mathrm W}{} is similar to \ensuremath{\mathrm{C}}, but refers to the \underline wrong tRNA for the codon under study.
\item Some reactions form complexes of tRNA with guanosine phosphates: \text{C$\cdot$GTP}, \text{C$\cdot$GDP}, \text{{W}$\cdot${GTP}}, and \text{{W}$\cdot${GDP}}.  (For simplicity, we suppress any mention of elongation factors, one of which, ``EF-Tu,'' is also included in these complexes but is only implicit in the classic HN mechanism.)
\end{itemize}

\fref{f4}b denotes the ribosome-mRNA complex by \ensuremath{\mathrm R}. In state \textsl0, this complex is not bound to any tRNA. (More precisely, no tRNA is bound at the ``A'' site of the ribosome; a previously bound tRNA, together with the nascent polypeptide chain, is bound at another site (\fref{f4}a), which we do not explicitly note.) Surrounding this state, \fref{f4}b shows four other states $\mathsl1$--$\mathsl4$ in which the ribosome is bound to the complexes introduced earlier. The upper part of the figure describes wrong tRNA binding and possible incorporation; the lower part corresponds to the  correct tRNA. Horizontal arrows at the top and bottom denote hydrolysis of GTP, which is coupled to a transformation of the ribosome into an activated state, $\ensuremath{\mathrm R}^*$.

Although any chemical reaction is fundamentally reversible, under cellular conditions the concentration ratio  $[\ensuremath{\mathrm P_\mathrm i}][\text{C$\cdot$GDP}]/[\text{C$\cdot$GTP}]$ is far below the equilibrium value, so that the reactions in \fref{f4}b are predominantly in the direction shown by the pale arrows.
This was one of the conditions in Hopfield's original proposal. (\ssref{rtdf} will explore relaxing it.)

Again, we are assuming that a \emph{single} ribosome bounces around this state diagram in the presence of fixed concentrations of feedstocks either imposed in vitro by the experimenter or supplied by a cellular milieu. There are two ways to ``exit the museum exhibit by the second door'': After hydrolysis, the ribosome can reject its tRNA-GDP complex with probability per unit time $\ell$. Or, with probability per unit time $k_{\rm add} $ it can {add} its amino acid to the nascent polypeptide, translocate the tRNA to the second binding site, and
eject any tRNA already bound there. Either way, the main binding site becomes vacant and, for the purposes of this state diagram, the ribosome returns to state \textsl0.

Supplementary computer code \ref{sc3} implements a Gillespie simulation on the five states of the ribosome (\fref{f4}b).
\pnonly{CLINCH}

\subsection{Visualization of the simulation results\label{ss:vsr}}
To keep the project modular, we constructed a simulation code that writes its state trajectory to a file.  A second code then reads that file and creates a visual output. The first of these codes operates similarly  to \ssref{bdp.s}, but with a four-way choice of what transition to make after each waiting interval.
The second code can be almost as simple as the one described in \ssref{wtde}. However, students with more time (perhaps in a capstone project) can make a more informative display with a reasonable additional effort, as follows.

The supplementary videos
not only show the state that is current at the end of each video frame; they also animate the pending arrivals of new complexes that are about to bind and the departures of old ones that have unbound without incorporation. By this means, the videos give a rough sense of the ``narrative'' in the trajectory being shown. These improvements are not difficult to add once the basic code is working. Alternatively, students can construct the basic version, then be shown these videos.

The exponential distribution of waiting times implies that there will be episodes with several events happening rapidly, interspersed with long pauses. For this reason, it is useful to view the simulation in two ways: Once with a shorter time step that resolves most individual events but covers only a limited time interval (supplementary video \ref{sv3}), and then with a coarser time step to see the entire synthesis trajectory (supplementary video \ref{sv4}).

\begin{figure}\begin{center}
	\includegraphics{expdist.pdf}
	\end{center}
\caption{\small\arttitle{Modified waiting times.} \textit{Solid line:} Exponential distribution of waiting times. \emph{Dashed line:} Shifted exponential distribution obtained by adding the constant $t_\mathrm{min}$ to each sample.
\label{f:alterwait}}\end{figure}
We also found it useful  (solely for visualization purposes) to alter the distribution of waiting times in a simple way that relieves visual congestion without, we think, too much damage to the realism of the simulation. Our modification, shown in the supplementary videos, was to add a small fixed delay, for example one half of one video frame, to every transition waiting time (\fref{alterwait}).

\subsection{Classic Hopfield--Ninio model\label{ss:ckp}}
Following Hopfield, we initially assume that the rate constant for incorporation, $k_{\rm add} $, is the same regardless of whether the tRNA is correct or incorrect. We also   suppose that the binding rates $k'\ssr c=k'\ssr w$ and $\ell'\ssr c=\ell'\ssr w$ also have this property; for example, all of them may be diffusion-limited \cite{Bphil08a}. Only the \emph{unbinding} rates differ in the classic  HN model:
$$k\ssr w=\phi_{-1}k\ssr c\text{ and }\ell\ssr w=\phi_3\ell\ssr c.$$
Here $\phi_{-1}=\phi_3\approx100$ is the preference factor for unbinding the wrong tRNA (relative to the correct one).
Again following Hopfield, we will also take the hydrolysis rate constants to be equal:   $m'\ssr w=m'\ssr c$ (and $m\ssr w=m\ssr c$).
\begin{table}\small\begin{tabular}{l|l|l||l|l|l}\textsl{Description}&\textsl{Symbol}&\textsl{Name in code}&\textsl{Classic HN}&\textsl{Realistic model}&\textsl{Equilibrium
	\\\hline
	binding GTP complex, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}\vphantom{1^{\strut}}$&$k'\ssr c$&\texttt{kc\_{}on}&$40$&40&
	40\\
	unbinding GTP complex, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$k\ssr c$&\texttt{kc\_{}off}&$0.50$&0.5&
	0.5\\
	binding GDP complex, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$\ell'\ssr c$&\texttt{lc\_{}on}&\PNrevAfn{$2.36\eem{12}$}{0.001}&\PNrevAfn{$1.18\eem{10}$}{0.001}&
	\PNrevAfn{13.04}{0.26}\\
	unbinding GDP complex, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$\ell\ssr c$&\texttt{lc\_{}off}&$0.085$&0.085&
	0.085\\
	hydrolysis and P$_\mathrm i$ release, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$m'\ssr c$&\texttt{mhc}&$0.01$&25&
	\PNrevAfn{25}{0.01}\\
	condensation/P$_\mathrm i$ binding, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$m\ssr c$&\texttt{msc}&\PNrevAfn{$2.36\eem{12}$}{0.001}&\PNrevAfn{$1.18\eem{10}$}{0.001}&
	\PNrevAfn{13.04}{0.26}\\
\hline
	binding GTP $\vphantom{1^{\strut}}$&$k'\ssr w=\phi_{1}k'\ssr c$&\texttt{kw\_{}on}&$\phi_{1}=1$&$0.68$&
	1\\
	unbinding GTP &$k\ssr w=\phi_{-1}k\ssr c$&\texttt{kw\_{}off}&$\phi_{-1}=5$&94&
	5\\
	binding GDP &$\ell'\ssr w=\phi_{-3}\ell'\ssr c$&\texttt{lw\_{}on}&$\phi_{-3}=1$&0.0027\pnonly{$^\star$}&
	1\\
	unbinding GDP &$\ell\ssr w=\phi_{3}\ell\ssr c$&\texttt{lw\_{}off}&$\phi_{3}=5\pnonly{^\dagger}$&7.9&
	5\\
	hydrolysis&$m'\ssr w=\phi_2m'\ssr c$&\texttt{mhw}&$\phi_2=1$&0.048&
	1\\
	condensation&$m\ssr w=\phi_{-2}m\ssr c$&\texttt{msw}&$\phi_{-2}=1$&1&
	1\\
\hline
	incorporation$\pnonly{^\#}\vphantom{1^{\strut}}$, $\mathrm s^{\raise.15ex\hbox{${\scriptscriptstyle -}$}\kern-.05em 1}$&$k_{\rm add,c} $&\texttt{kaddC}
	$0.01$&4.14&
	0.01\\
	incorporation&$k_{\rm add,w} =\phi_\mathrm{add}k_{\rm add,c}$&\texttt{kaddW}&$\phi_\mathrm{add}=1$ 	&$0.017$&
	1\\
\hline\end{tabular}
\caption
\small Illustrative values for the rates shown in \fref{f4}b. The third column gives variable names used in supplementary computer code \ref{sc3}. See the Appendix for discussion of the numerical values. The fifth column follows \cite{Banerjee:2017en,ISI:000398884800036}, who refer to our $\phi_i$ as $f_i$.  The last column uses rates from the HN model, but with hydrolysis and incorporation modified to satisfy equilibrium.
\hfil\break
\pnonly{Footnotes: $\star$:~Banerjee et al.\ chose $\phi_{-2}=1$, then got $\phi_{-3}$ by consistenc
: $\phi_1\phi_2\phi_3=\phi_{-1}\phi_{-2}\phi_{-3}$.
$\dagger$:~Hopfield chose $\phi_3=\phi_{-1}$. Here it's required by the consistency condition.
$\ddag$:~$m\ssr c$ and $\ell'\ssr c$ were chosen to be 0.001 to ensure irreversibility of the GTP hydrolysis step \cite{ISI:000398884800036}.
$\#$:~\textbf{\color{red}Here Phil disagrees with Banerjee and used the rate they quote for the irreversible next-to-last step.}}}
\end{table}
To visualize wrong incorporations within a reasonable time frame, we raised the probability of incorrect choices: The preference ratios $\phi_{-1}$ and $\phi_3$ were lowered from their realistic value of 100 to just 5. Other values in column 4 of Table I were loosely inspired by rate constants estimated from experimental data  in a simplified form  (see Appendix \ref{a:1}). (\ssref{mrm} will follow the experimental values more closely.) \PNrevC{These effective rate constants are either zeroth order (unbinding and hydrolysis), or else
pseudo-zeroth order (binding and condensation), with rates appropriate for the concentrations of reactants present in the experiment.}{These effective rate constants are either a constant probability per unit time (unbinding and hydrolysis) or else a probability per unit time with the substrate concentration already lumped in (binding and condensation). The values we chose were appropriate for the concentrations of reactants present in the experiment.}{*} \PNrevB{Some of the rates were modified to satisfy conditions for the classic Hopfield--Ninio kinetic proofreading model. For example, $m'\ssr c$ and $k_{\rm add,c}$ were lowered from 25$\,\ensuremath{\mathrm s}^{-1}$ and 4.14$\,\ensuremath{\mathrm s}^{-1}$, respectively, to 0.01$\ensuremath{\mathrm s}^{-1}$  (see supplementary section X.X for a full discussion of the Hopfield model rate constants).}{}{*}

Supplementary videos \ref{sv3}--\ref{sv4} show the resulting behavior.
Perhaps the most important impression we get from viewing these animations is that \emph{the cell is a busy place.} The riot of activity, the constant binding events that end with no ``progress'' (and often not even GTP hydrolysis), are hallmarks of chemical dynamics that are hard to appreciate in textbook discussions, yet vividly apparent in the simulation.
This is especially apparent in supplementary video \ref{sv4}, which shows a typical run of 25 amino acid incorporations. Because there are many unproductive binding and unbinding events in the simulation, not every event is shown in detail in video \ref{sv4}. However, focusing on the GDP-tRNA rejections shows that more correct tRNAs than incorrect tRNAs make it past GTP hydrolysis, and that the few incorrect tRNAs that do make it past are quickly rejected in the second proofreading step. In the instance shown, only one incorrect amino acid was incorporated out of 25 incorporations, much lower than the 1/5 error rate expected from single step equilibrium binding. Supplementary video \ref{sv3} provides a more detailed look at this process.
The videos also show clearly the jerky, nonuniform progress of synthesis, with some amino acid incorporations happening after much longer delays than others. That feature is by now well documented by single-molecule experiments.

A typical run created a chain of 100 amino acids, of which 6 were wrong. This error rate of $\approx(6/100)=0.06$ is \PNrevB{similar to the value $1/(\phi_3)^2 = 1/5^2 = 0.04$}{far smaller than the naive expectation of $1/\phi_3=0.20$}{*}. This is the essence of the classic HN mechanism; we see it taking shape in the animation, as many wrong tRNA complexes bind but are rejected, either prior to or after GTP hydrolysis.
\PNrevA{}{We also see many \emph{correct} complexes bind and get rejected, before or after GTP hydrolysis. This is the price paid for accuracy in the classic HN proofreading model.}{}
The error rate in the simulation is slightly larger than \PNrevB{1/25 = 0.04
Michaelis--Menten kinetics prediction for the error rate given the rate constants in Table 1 is 0.45}{$1/(\phi_3)^2 = 1/5^2 = 0.04$, however. The discrepancy is expected,}{*} because the limiting value $1/(\phi_3)^2$ is only achieved in the limit as the incorporation and hydrolysis catalytic rates are sent to zero \cite{ISI:000398884800036,Hopfield:1974p15095}. \PNrevA{Supplementary Figure 1 shows the predicted error, based on Michaelis--Menten kinetic analysis \cite{Banerjee:2017en}, with variation in incorporation or hydrolysis rate. The graph shows that the error approaches 0.04 as the incorporation rate (1a) or hydrolysis rate (1b) decrease.}{}{}

\subsection{More realistic model\label{ss:mrm}}
Much has been learned about ribosome dynamics after Hopfield's and Ninio's original insights \cite{Arodn11a,Bbaha17a}. We now know that each step in our model consists of substeps. For example, GTP hydrolysis is subdivided into GTPase activation followed by actual hydrolysis, the latter step probably depends on a rearrangement of ``monitoring bases'' in the ribosomal RNA, and so on \cite{Satpati:2014kz}.

The  model studied in \ssref{ckp} was designed to show the HN mechanism in its ``classic,'' or pure, form, and how it can enhance fidelity even without help from the effects just described. For example, we assumed that the only dependence on right versus wrong tRNA was via unbinding rates. Indeed, such dependence was later seen at the single-molecule level \cite{Blanchard:2004p7821}. But it now appears that some of the forward rates also depend on the identity of the tRNA \cite{Zaher:2010ga,ISI:000403920000010,ISI:000393403600005}, an effect sometimes called ``internal discrimination.'' In the limit that the ribosome  uses only internal discrimination (activation barrier heights of correct and incorrect tRNA binding differ and the equilibrium constants are the same), minimum error is obtained at fast catalytic rates \PNrevA{\cite{Sartori:2013fv}}{\cite{Bennett:1979tb}}{*}. This is in contrast to the HN scheme, which achieves minimum error as catalytic rate tends to zero. \PNrevA{However, a purely internal discrimination model can only discriminate at one step and is thus not as accurate as a pure HN model for similar bias factors in the tRNA binding rates.}{}{*}

In our stochastic simulation model, it is straightforward to add internal discrimination effects by altering rate constants (Table I column 5). See Appendix \ref{a:2} for discussion of the values.

Supplemental video \ref{sv5} shows that the ribosome with experimentally measured rates can be faster and more efficient than a ribosome with only classic HN proofreading. \PNrevC{}{We see both a bias for correct tRNA binding/hydrolysis and a bias for rejection of wrong tRNAs before GTP hydrolysis.}{*} Of the 26 correct tRNA binding events in this run, 25 resulted in successful incorporation. This is compared to the  fraction $24/10245=0.002$ of productive correct tRNA binding events in a typical run of the classic HN ribosome simulation. In addition, of the 30 incorrect tRNA binding events on the realistic ribosome, all 30 resulted in rejection. The error fraction of the ribosome with realistic rates is also more accurate than the classic HN ribosome. Of 10\bn000 amino acids simulated, 18 wrong amino acids were incorporated (error fraction of 0.0018), compared to \PNrevA{X wrong amino acids}{6/100}{*} for the classic HN ribosome. \PNrevC{This matches the Michaelis--Menten kinetics predicted error fraction 0.0017 for the realistic ribosome.}{This simulated error fraction of 0.0018 is consistent with the analytic prediction of 0.0017  from first-passage times
\cite{ISI:000398884800036}.}{*}

However, the simulated ribosome with in-vitro measured rates is still not as fast or accurate as the real  \emph{E.~coli} ribosome in vivo, which translates at 15--20 amino acids per second with an error rate of 1/10\bn000 \cite{Bmilo15a}. Thus, the realistic ribosome likely evolved to combine HN proofreading (quasiequilibrium, energetic proofreading) with internal discrimination (unequal forward rates) to optimize speed, efficiency, and accuracy \PNrevA{\cite{Sartori:2013fv}}{\cite{Rao:2015vy}}{}. Despite this, simulating the classic HN model of proofreading is still a valuable exercise for students. By visualizing discrimination via only a difference in unbinding rates, students see the minimal components necessary to attain high accuracy in a broad class of biological reactions. Also, the classic HN mechanism illustrates an essential part of biological proofreading which fundamentally relies on non-equilibrium physics.

There is also recent evidence pointing to \emph{two} kinetic proofreading steps, that is, two sequential, nearly irreversible steps each of which can be followed by unbinding of tRNA \cite{ISI:000388835700066,Chen:2016bx}. Our simulation could be extended to include such effects\PNrevC{}{, whereas analytic methods would quickly become intractable}{*}.
Finally, additional interesting steps arise during ``translocation,'' in which the previous tRNA, from which the nascent peptide chain has been released, and the current tRNA, now carrying that chain with an additional amino acid, are both shifted one step inside the ribosome, freeing the binding site so that the entire cycle can begin again (\fref{f4}a). Because this step is not related to accuracy, we have simplified by omitting it from our model.

\subsection{Role of thermodynamic driving force\label{ss:rtdf}}
For comparison, we return to the classic Hopfield--Ninio model, this time operating at nearly equilibrium concentrations of GTP, GDP, and $\ensuremath{\mathrm P_\mathrm i}$ to demonstrate the importance of the ``one-way door''  (the GTP hydrolysis step).  Table I column 6 summarizes the rates for this undriven model (see Appendix \ref{a:3}).

With \PNrevA{equilibrium}{these}{*} rates, the reaction still creates a chain,  because we assumed a fixed probability per time to irreversibly add an amino acid whenever the ribosome visits its activated state. But this time a typical run gave 17 errors in a chain of length 100, illustrating the significance of the thermodynamic driving force in reducing the error rate. \PNrevA{This matches the Michaelis--Menten predicted error of 0.49 given the rate constants in Table I column 6. This error rate is consistent with the fact that, for a HN model with only difference in the unbinding rates, the error can never fall below $1/\phi_3 = 1/5$ in equilibrium conditions \cite{Hopfield:1974p15095}.}{This error rate of {about 0.17} is consistent with the Michaelis-Menten predicted error of $1/\phi_3=0.20$, which is the lowest the error can be for a classic HN model in equilibrium conditions \cite{Hopfield:1974p15095}.}{*}

Analysis of the events in the simulation showed that of the 17 wrong amino acids incorporated, 10 were through direct binding of GDP$\cdot$tRNA. Thus, for many amino acids, the first discrimination step was bypassed\PNrevC{This results in the {\color{red}0.20 [[PREV PARAGRAPH SAID 0.17]]} error rate observed in the simulation. Thus, two independent discriminations are not possible in a simulation with GTP hydrolysis at equilibrium}{, resulting in the high error rate observed  in the simulation.}{*}

\PNrevA{}{To gain more insight into the role of the irreversible GTP hydrolysis step, some students may wish to rerun the simulation with different incorporation and hydrolysis rates. For example, a simulation with $m'\ssr c = 25$ and $k_{\rm add} = 4.14$ results in a simulation with many tRNAs flipping between GDP and GTP states, another way in which the two discrimination steps become coupled into one.}{*}

\section{Conclusion}
The models described here show fairly elementary physical principles that lie at the heart of cell biology.
Specifically,  gene expression and kinetic proofreading are two important, fundamental topics that are well within reach of undergraduates.

A module that introduces stochastic simulation need not dominate a semester course: One class week is enough for the first exposure. Indeed, the entire simulation plus visualization in supplementary computer code \ref{sc1} consists of just \emph{seven short lines} of code, and yet it creates a valuable educational experience not available in a static textbook. Moreover, the opening material is not specifically biological in character; it can serve as
a stepping stone to more complex simulations relevant for a variety of courses.

\section*{Acknowledgments}
We are grateful to Ned Wingreen and Anatoly Kolomeisky for correspondence and to Bruce Sherwood for help with software.
This work was partially supported by the United States
National Science Foundation under Grants PHY--1601894 and MCB--1715823. Some of the work was done at
the Aspen Center for Physics, which is supported by NSF grant PHY--1607611, and at the Physical Biology of the Cell summer school
(Marine Biological Laboratory).