Suppose we are given some polynomial with integer coefficients, which we regard as carving out an affine variety $E$, for example:


$$ 3x^2y - 12 x^3y^5 + 27y^9 - 2 = 0 \tag{$\*$} $$


(We might consider a bunch of equations, we might work over projective space, but let's keep it simple for now).


We are interested in the number of points on $E$ when we reduce modulo $p$, i.e. over the finite field $\mathbb{F}\_p$ as the prime $p$ varies. For our single equation in two variables, as a rough approximation, we would expect $p$ points in general. So for each prime $p$, we define numbers $a\_p$ which measure the deviation from this, 


$$a\_p := p - \text{number of solutions to $(\*)$ over $\mathbb{F}\_p$}$$


Question: When is it true that the numbers $\{a\_p\}$ are "modular", in the sense that there exists a modular form 


$$f = \sum\_{n=1}^\infty b\_n q^n$$


such that $b\_p = a\_p$ for almost all primes $p$? (The "almost all" is to avoid problems with bad primes. Note that $f$ is still uniquely determined by the above requirement.)


The Modularity Theorem of Breuil, Conrad, Taylor and Diamond says that this is true when $E$ is an elliptic curve, i.e. takes the form $y^2 = 4x^3 - g\_2 x - g\_3$ for some integers $g\_2, g\_3$. In that case, $f$ is a weight 2 modular form of level $N$ where $N$ is the "conductor" of $E$. 


But is it true for more general varieties? 


(Note: I am aware of a generalized "Modularity Theorem" for certain Abelian varieties, but it's not clear to me that what people mean by "Modular" in that context is the same as the simple-minded notion I'm using --- that an adjusted count of points mod $p$ gives the Fourier coefficients of a modular form.) 



Modular forms from counting points on algebraic varieties over a finite field
The correct setting for this construction turns out to be *projective* varieties, so let me suppose we have a smooth variety $X$ inside $\mathbf{P}^N$, for some $N \ge 1$, defined by the vanishing of some homogenous polynomials $F\_1, \dots, F\_r$ in variables $x\_0, \dots, x\_N$, with the $F\_i$ having coefficients in $\mathbf{Q}$. Actually, let me assume the $F\_i$ have coefficients in $\mathbf{Z}$, which is no loss since we can just multiply up. Then we can make sense of the reduction of $X$ modulo $p$; and we want to study the point counts $\#X(\mathbf{F}\_p)$ as a function of $p$, possibly neglecting some finite set $\Sigma$ containing all primes such that the reduction of $X$ mod $p$ is singular.


Thanks to Grothendieck, Deligne, and others, we have a very powerful bunch of tools for analysing this problem. The setup is as follows. Choose your favourite prime $\ell$. Then the theory of **etale cohomology** attaches to $X$ a bunch of finite-dimensional $\mathbf{Q}\_\ell$-vector spaces $H^i\_{\mathrm{et}}(X\_{\overline{\mathbf{Q}}}, \mathbf{Q}\_\ell)$ (let me abbreviate this by $H^i\_\ell(X)$ to save typing). The dimension of $H^i\_\ell$ is the same as the $i$-th Betti number of the manifold $X(\mathbb{C})$; but they encode much more data, because each $H^i\_\ell(X)$ is a representation of the Galois group $\operatorname{Gal}(\overline{\mathbf{Q}} / \mathbf{Q})$, unramified outside $\Sigma \cup \{\ell\}$; so for every prime not in this set, and every $i$, we have a number $t\_i(p)$, the trace of Frobenius at $p$ on $H^i\_\ell$, which turns out to be independent of $\ell$.


**Theorem**: $\#X(\mathbf{F}\_p)$ is the alternating sum $\sum\_{i=0}^{2 dim(X)} (-1)^i t\_i(p)$.


Now let's analyse $H^i\_\ell$ as a Galois representation. Representations of Galois groups needn't be direct sums of irreducibles, but we can replace $H^i\_\ell$ by its semisimplification, which does have this property and has the same trace as the original $H^i\_\ell$. This semisimplification will look like $V\_{i, 1} + \dots + V\_{i, r\_i}$ where $V\_{i, j}$ are irreducible; and the $V\_{i, j}$ all have motivic weight $i$, so the same $V$ can't appear for two different $i$'s. So we get a slightly finer decomposition


$\#X(\mathbf{F}\_p) = \sum\_{i=0}^{2 \mathrm{dim} X} (-1)^i \sum\_{j=1}^{k\_i} t\_{i, j}(p)$


where $t\_{i,j}(p)$ is the trace of $Frob\_p$ on $V\_{i,j}$.


Let me distinguish now between several different types of irreducible pieces:


* $V\_{i, j}$ is a one-dimensional representation. Then $i$ must be even, and the trace of Frobenius on $V\_{i, j}$ is $p^{i/2} \chi(p)$ where $\chi$ is a Dirichlet character.
* $V\_{i, j}$ is two-dimensional and comes from a modular form. Then $t\_{i,j}(p) = a\_p(f)$, and $f$ must have weight $i+1$.
* $V\_{i,j}$ is two-dimensional and doesn't come from a modular form. This can happen, but it's rare, and it's expected that all examples come from another kind of analytic object called a Maass wave form; in particular this forces $i$ to be even.
* $V\_{i, j}$ has dimension $> 2$. Then $V\_{i, j}$ cannot be the Galois representation coming from a modular form, because these always have dimension 2.


You seem to want your varieties to have $X(\mathbf{F}\_p)$ = (polynomial in $p$) + (coefficient of a modular form). From the above formulae, it's clear that this can only happen if all the $V\_{i,j}$ have dimension 1 or 2; there is exactly one with dimension 2 and it comes from a modular form; and the one-dimensional pieces all come from the trivial Dirichlet character. This always happens for genus 1 curves, because the $H^0$ and $H^2$ are always 1-dimensional for a curve, and the genus condition forces the $H^1$ to be two-dimensional. 


However, once you step away from genus 1 curves, this is totally not the generic behaviour, and it will only occur for unusual and highly symmetric examples, such as the rigid Calabi-Yaus and extremal $K\_3$ surfaces in the links you've posted. 


