This question is a continuation of the following two questions:


[Discriminants of indefinite integral binary quadratic forms admitting 3 or 6 torsion.](https://mathoverflow.net/questions/133445/discriminants-of-indefinite-integral-binary-quadratic-forms-admitting-3-or-6-tor)


[On certain solutions of a quadratic form equation](https://mathoverflow.net/questions/245094/on-certain-solutions-of-a-quadratic-form-equation)


Let $f$ be a binary quadratic form with integer coefficients and non-zero discriminant. For $T = \begin{pmatrix}t\_1 & t\_2 \\ t\_3 & t\_4 \end{pmatrix} \in \operatorname{GL}\_2(\mathbb{R})$, define $f\_T(x,y) = f(t\_1 x + t\_2 y, t\_3 x + t\_4 y)$. Put


$$\displaystyle \operatorname{Aut}(f) = \{T \in \operatorname{GL}\_2(\mathbb{Z}) : f\_T = f\}.$$


When $f$ is positive definite, then $\# \operatorname{Aut}(f)$ is easy to determine. In particular, if $f(x,y)$ is reduced, so that it is written as


$$\displaystyle f(x,y) = ax^2 + bxy + cy^2$$


with $|b| \leq a \leq c$ with $ac > 0$, then $\# \operatorname{Aut}(f) = 12$ when $a = b = c = 1$, $\# \operatorname{Aut}(f) = 8$ when $a = c = 1, b = 0$, $\# \operatorname{Aut}(f) = 4$ when $a = \pm b$, and $\#\operatorname{Aut}(f) = 2$ otherwise. When $f$ is reducible, the situation is also easy to account for. The most difficult case is when $f$ is indefinite and irreducible. 


One part of the problem is easy. If we consider only positive determinant elements, then they are given by


$$\displaystyle \begin{pmatrix} \dfrac{s+bt}{2} & & -ct \\ \\ at & & \dfrac{s - bt}{2} \end{pmatrix},$$


where $(s,t)$ is a solution to the Pell equation $x^2 - (b^2 - 4ac)y^2 = 4$ (note that this equation is always soluble since $(2,0)$ is a solution).


The problem is the elements of negative determinant. I can show (see the second question linked above) that such elements take the form


$$\displaystyle \begin{pmatrix} m & (bm + cn)/a \\ n & - m\end{pmatrix},$$


where $(m,n)$ is a solution to $am^2 + bmn + cn^2 = a$, which is again always soluble. But there is no guarantee that $(bm+cn)/a$ is an integer. This is guaranteed if $n$ is co-prime to $a$, since then looking at the equation above we see that $a | n(bm+cn)$, whence $a | bm + cn$. There are however examples where this doesn't happen; I believe $5x^2 + 3xy - 11y^2$ is such an example. All of the solutions to $5x^2 + 3xy - 11y^2 = 5$ have $5 | y$, and since $5 \nmid 3$ and $\gcd(m,n) = 1$ by necessity, it is impossible for $3m-11n$ to be divisible by 5 and thus no way for the above matrix to be integral. 


However, such examples seem extremely rare. I don't have a lot of computational evidence except to say that there is no such example with $D = b^2 - 4ac < 100$.


The question is, when is the set


$$\displaystyle G^-(f) = \{T \in \operatorname{Aut}(f) : \det T = -1\}$$


non-empty? There are several sufficient conditions for this to be non-empty: if $f$ is equivalent to a diagonal form, if $f$ is equivalent to a form such that $a | b$, , if $f$ is equivalent to a symmetric form, or $f$ is equivalent to a form such that there exists a solution $(m,n)$ to $am^2 + bmn + cn^2 = a$ with $\gcd(n,a) = 1$. Moreover, if the class number is one, then all forms with discriminant $d$ are equivalent and therefore must be equivalent to either $x^2 - (d/4)y^2$ if $d$ is even or $x^2 + xy - (d-1)y^2/4$ if $d$ is odd. Both cases have non-empty $G^-$. 


A related question is, suppose that $f\_1(x,y) = a\_1 x^2 - b\_1 y^2$ and $f\_2(x,y) = a\_2 x^2 - b\_2 y^2$ are two non-proportional binary quadratic forms such that $a\_1 b\_1 = a\_2 b\_2 > 0$. Then when can $f\_1, f\_2$ be $\operatorname{GL}\_2(\mathbb{Z})$-equivalent?



On the automorphism group of binary quadratic forms
Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:



> 
> 
> > 
> > Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:
> > 
> > 
> > 
> 
> 
> $(1)$ There exists $A \in \text{GL}\_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.
> 
> 
> $(2)$ The matrix $S$ is $\text{SL}\_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.
> 
> 
> $(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}\_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)
> 
> 
> 


**Hint:** If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$.
Then reduce the case when $A$ is upper triangular.


If the above conditions hold, the $\text{SL}\_2(\mathbb{Z})$-class to which such an $S$ belongs is called an *ambig class* (sic). This is also what John Robertson calls an *ambiguous class* in [2]. From this source, we get



> 
> 
> > 
> > Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+}(D) \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.
> > 
> > 
> > 
> 
> 
> 




---


[1] T. Ibukiyama, M. Kaneko, "[Quadratic Forms and Ideal Theory of Quadratic Fields](https://link.springer.com/content/pdf/10.1007/978-4-431-54919-2_6.pdf)" in *Bernoulli Numbers and Zeta Functions*, pp 75-93, 2014.  

[2] J. Robertson, "[Computing in Quadratic Orders](http://www.jpr2718.org/quadr.pdf)", 2009.


