There are rare algebraic varieties such that the number of points over finite fields $\mathbb{F}\_p$ is given by a polynomial in $p$. One notable series of examples is the commuting variety: $[A,B]=0$ of $n\times n$ matrices $A,B$ over finite field. The computation was obtained by [Feit and Fine in 1960](https://projecteuclid.org/journals/duke-mathematical-journal/volume-27/issue-1/Pairs-of-commuting-matrices-over-a-finite-field/10.1215/S0012-7094-60-02709-5.short) and many generalizations have been obtained recently. But it seems results in natural generality are not yet achieved (see below).


**Question 1:** Consider pairs of $3\times 3$ anticommuting $AB+BA = 0 $ over finite fields $\mathbb{F}\_p$, is true that their number is polynomial in $p$, for $p>2$ ? May be one needs to exclude some other primes, not only $p=2$ or $p=2$ is the only exception ?


**Question 2:** If the number of points is indeed given by a polynomial ($p>2$), then it is given by the polynomial found by Roland Bacher and Peter Taylor in [MO 404760](https://mathoverflow.net/q/404760/10446):
$$2p^{10}+7p^9-3p^8-6p^7-4p^6+3p^5+4p^4-2p^3$$
([My direct calculation](https://www.kaggle.com/alexandervc/count-anticommuting-matrices-over-f-p-attempt-3) yields 221157 and 31511625 matrices for $p=3,5$ respectively,
and colleagues found that it is the only polynomial of degree 10 which satisfies these conditions and has minimal possible coefficients. Heuristic to search for polynomials with the smallest possible coefficients works quite fine in my experience for such questions.)


**Question 3:** Bonus question. It might be count is polynomial for any $n$ and $n$x$n$ anticommuting $[A,B]=0$, and there is nice generating function over $n$ for such polynomials - similar to Feit,Fine result for commuting matrices ? (Well, it might be better to leave it for separate question).


======================


PS


For 2x2 case the polynomial seems to be given by $+p^5+3p^4-2p^3-2p^2+p$ for $p>2$, checked till $p=19$.


Anticommuting variety has been studied recently e.g.: [Anti-commuting varieties](https://arxiv.org/abs/1805.00378).


======================


**Context**
We might expect similar results not only for pairs, but triples, n-tuples of commuting/anticommuting matrices: [MO271752](https://mathoverflow.net/q/271752/10446), but commuting/anticommuting is just an example, it should be true for much wider class of algebras - "categorical exponential formula" might be the right context for such questions [MO272045](https://mathoverflow.net/q/272045/10446), [MO275524](https://mathoverflow.net/q/275524/10446), however presently known forms seems does not cover even Feit,Fine case. See also very nice results and connections with Hasse-Weil zeta function in  [Yifeng Huang 2021](https://yifeng-huang-math.github.io/files/talk_gocc.pdf). The general question about varieties which are polynomial count seems also not so simple as [disproof of Kontsevich conjecture](https://arxiv.org/abs/math/0012198) indicates.
Such varieties thought to be defined over the mysterious ["field with one element"](https://en.wikipedia.org/wiki/Field_with_one_element).



Number of $3\times 3$ anticommuting matrices over finite fields $\mathbb{F}_p$ is (or is not?) polynomial in $p$?
Let's work over a field $K$, which when finite is supposed to have $q$ elements. I'll assume the characteristic to be $\neq 2$, since in characteristic 2 we get the commuting variety which is well-known. Let $Q=Q(K)$ be the set of anticommuting pairs $(A,B)$.


If $A$ has eigenvalues $(x,y,z)$ (possibly on an extension), the eigenvalues of $B\mapsto AB+BA$ are, $2x,2y,2z$, and, with multiplicity 2, $x+y$, $x+z$, $y+z$. Write $W\_A$ as the kernel of this operator, i.e., the set of $B$ such that $(A,B)\in Q$.


We essentially need to discuss in terms of the conjugacy class of $A$, and more precisely in terms of the cardinal of $W\_A$ and of the conjugacy class of $A$.


Write $Q=Q\_1\sqcup Q'\sqcup Q''$, where $Q\_1$ is the set of pairs $(A,0)$ with $A$ invertible; $Q'$ is the set of pairs $(A,B)$ with $A$ invertible, $B\neq 0$, $Q''$ is the set of pairs $(A,B)$ with $A$ not invertible. Define $V'$ and $V''$ as the first projection of $Q'$ and $Q''$.


Write $u\_q=(q^3-1)(q^3-q)(q^3-q^2)$ for the cardinal of $\mathrm{GL}\_3(\mathbf{F}\_q)$. Then $V\_1$ has cardinal $u\_q$.


From the eigenvalues fact above, the elements of $V'$ have eigenvalues $(x,-x,y)$ for some nonzero $x,y$. Write $V'=V\_2\sqcup V\_3\sqcup V\_4\sqcup V\_5$ with


* $V\_2$ (resp. $V\_3$) the set of $K$-diagonalizable (resp. non-$K$-diagonalizable) invertible matrices with eigenvalues of the form $(x,-x,y)$ with $y\neq\pm x$; (note that $y$ is the trace).
* $V\_4$ (resp. $V\_5$) the set of diagonalizable (resp. non-diagonalizable) invertible matrices with eigenvalues $(x,x,-x)$ (note that $x$ is the trace)


Write $V''=V\_6\sqcup \dots\sqcup V\_{16}$ with


* $V\_6$ (resp. $V\_7$) the set of $K$-diagonalizable (resp. non-$K$-diagonalizable) matrices with eigenvalues of the form $(0,x,y)$, $x,y$ nonzero and $x\neq\pm y$
* $V\_8$ (resp. $V\_9$) the set of diagonalizable (resp. non-diagonalizable) matrices with eigenvalues of the form $(0,x,x)$, $x\neq 0$ (note that $x$ is half the trace)
* $V\_{10}$ (resp. $V\_{11}$) the set of $K$-diagonalizable (resp. non-$K$-diagonalizable) matrices with eigenvalues of the form $(0,x,-x)$, $x$ nonzero
* $V\_{12}$ (resp. $V\_{13}$) the set of diagonalizable (resp. non-diagonalizable) matrices with eigenvalues of the form $(0,0,x)$, $x\neq 0$
* $V\_{14}$, $V\_{15}$, $V\_{16}$ the set of nilpotent matrices of rank 2, 1, 0 respectively.


For each $i\in\{2,\dots,16\}$, all matrices $A$ in $V\_i$ have a centralizer in $\mathrm{GL}\_3(\mathbf{F}\_q)$ of the same size, say $c\_i$ (and hence a conjugacy class of cardinal $u\_q/c\_i$), and have $W\_A$ of the same size, say $q^{d\_i}$ (we will compute them below). Also denote by $e\_i$ the number of conjugacy classes in $V\_i$.


Let $Q\_i$ be the inverse image of $V\_i$ in $Q'$ (for $2\le i\le 5$) and in $Q''$ for $i\ge 6$.


Then for $i\ge 6$ the cardinal of $Q\_i$ is $u\_qe\_iq^{d\_i}/c\_i$, and for $2\le i\le 5$ it is $u\_qe\_i(q^{d\_i}-1)/c\_i$, the difference being only because we excluded $B=0$ in those cases. We now compute case by case.


* $d\_2=d\_3=2$, $d\_4=4$, $d\_5=2$, $d\_6=d\_7=d\_8=d\_9=1$, $d\_{10}=d\_{11}=3$, $d\_{12}=4$, $d\_{13}=2$, $d\_{14}=3$, $d\_{15}=5$, $d\_{16}=9$.
* $c\_2=c\_6=c\_{10}=(q-1)^3$, $c\_3=c\_7=c\_{11}=(q+1)(q-1)^2$, $c\_4=c\_8=c\_{12}=(q-1)(q^2-1)(q^2-q)$, $c\_5=c\_9=c\_{13}=q(q-1)^2$, $c\_{14}=(q-1)q^2$, $c\_{15}=(q-1)^2q^3$, $c\_{16}=u\_q$.
* $e\_2=(q-1)(q-3)/2$ (choose $y\neq 0$, then $x\notin\{0,y,-y\}$, up to $x\to -x$), $e\_3=(q-1)^2/2$ (choose $y\neq 0$ and a non-square $x^2\neq 0$), $e\_4=e\_5=e\_8=e\_9=e\_{12}=e\_{13}=q-1$, $e\_6=(q-1)(q-3)/2$ (choose $y\neq 0$, then then $x\notin\{0,y,-y\}$, up to $x\leftrightarrow y$), $e\_7=(q-1)^2/2$ (choose nonzero $y$ and non-square $x^2$), $e\_{10}=e\_{11}=(q-1)/2$, $e\_{14}=e\_{15}=e\_{16}=1$.


So the desired cardinal is $|Q\_1|+\sum\_{i=2}^5|Q\_i|+\sum\_{i=6}^{16}|Q\_i|$, namely
$$|Q(\mathbf{F}\_q)|=u\_q+u\_q\sum\_{i=2}^5 e\_i(q^{d\_i}-1)/c\_i+u\_q\sum\_{i=6}^{16} e\_iq^{d\_i}/c\_i.$$


Computation yields
$$|Q(\mathbf{F}\_q)|=2q^{10} + 7q^9 - 3q^8 - 6q^7 - 4q^6 + 3q^5 + 4q^4 - 2q^3,$$
which confirms your expectation.


