Consider an octagon $O$ with opposite edges identified. Lemma 3.2.4 of (1) ([subscription link](http://plms.oxfordjournals.org/content/102/2/291.full.pdf)) claims that the affine automorphism group of $O$ is generated by $D\_8$ and the shear $\sigma$ such that, if $O$ is drawn in the plane with a pair of edges parallel to each axis, $\sigma$ acts by Dehn twists on the horizontal cylinders. However, I don't see any way for these elements to generate the shear in the $\pi/8$ direction which acts by Dehn twists in the corresponding cylinders.


Note that the map $f\to Df$ which extracts the linear part is an isomorphism for $\mathrm{Aff}(O)$. The linear parts of the elements of $D\_8$ are standard, while the linear part of $\sigma$ is
$$\begin{pmatrix} 1 & 2+2\sqrt2\\ 0 & 1\end{pmatrix}$$
and that of the other shear is
$$\begin{pmatrix} \cos\frac{\pi}{8} & \sin\frac{\pi}{8}\\ -\sin\frac{\pi}{8} & \cos\frac{\pi}{8}\end{pmatrix}^{-1} \begin{pmatrix} 1 & 2+2\sqrt2\\ 0 & 1\end{pmatrix} \begin{pmatrix} \cos\frac{\pi}{8} & \sin\frac{\pi}{8}\\ -\sin\frac{\pi}{8} & \cos\frac{\pi}{8}\end{pmatrix}$$
Note that the rotations by $\pi/8$ are not in $D\_8$. Experimentally I haven't been able to express this in terms of the linear parts of $D\_8$ and $\sigma$, but I'm not familiar with any algorithm for deciding the word problem in f.g. subgroups of $GL(2,\mathbb R)$.


The proof of Lemma 3.2.4 references pages 185-6 of (2) ([Google books link](http://books.google.com/books?hl=en&lr=&id=iMsVm_mdDEYC&oi=fnd&pg=PA165&dq=Teichm%C2%A8uller%20disks%20and%20Veech%E2%80%99s%20F-structures&ots=TIA4ixqYJg&sig=Tm5GZir8xRZFCTHC4bWtco0YkAs#v=onepage&q=Teichm%C2%A8uller%20disks%20and%20Veech%E2%80%99s%20F-structures&f=false)). This states that if $O'$ is the surface obtained by gluing two octagons along opposite parallel edges, the image of $\mathrm{Aff}^+(O')\to PGL(2,\mathbb R)$ is generated by a rotation by $\pi/4$ and the image of $\sigma$. Exploiting the double cover $O'\to O$ we see that this implies the same result for $O$. However, (2) refrences section 7 of (3) ([subscription link)](http://download.springer.com/static/pdf/121/art%253A10.1007%252FBF01388890.pdf?auth66=1398217625_e29ed898d3dff5284904e5a89355bb16&ext=.pdf), which appears to use a rotation by $\pi/8$ rather than $\pi/4$ (specifically, the rotation by $\frac{2\pi}{\epsilon n}$ where here $\epsilon=2,n=8$).


Is this indeed an error, or do $D\_8$ and $\sigma$ actually generate $\mathrm{Aff}(0)$? If they do, what is an expression for the shear in the $\pi/8$ direction?


1. John Smillie, Corinna Ulcigrai: Beyond Sturmian sequences: coding linear
trajectories in the regular octagon, Proceedings of the London Mathematical
Society, 102(2) (2011), pp. 291-340.
2. C. J. Earle and F. P. Gardiner, Teichmuller disks and Veech’s F-structures, Contemporary Mathematics
(American Mathematical Society, Providence, RI, 1997) 165–189.
3. W. A. Veech, ‘Teichmuller curves in moduli space, Eisenstein series and an application to triangular
billiards’, Invent. Math. 97 (1989) 553–583.



Generators for the affine automorphism group of the octagon
It turns out the shear is given by:


$$\begin{pmatrix}\sqrt2/2 & \sqrt2/2\\ \sqrt2/2 & -\sqrt2/2\end{pmatrix}
\begin{pmatrix} 1 & 2+2\sqrt2\\ 0 & 1\end{pmatrix}
\begin{pmatrix} -1 & 0\\ 0 & 1\end{pmatrix}$$


This expression was found by mapping to matrices over $(\mathbb Z/p\mathbb Z)[\sqrt2]$ for odd primes $p$ such that $2$ is not a quadratic residue and using Magma to derive an expression for the desired matrix in terms of generators for $D\_8$ and the linear part of $\sigma$. The smallest prime for which resulting expression was also valid in the original group was $p=11$.


