It has bug me for a while that I don't have a good understanding of the theory of Hecke operators. For elliptic modular forms, it was explained in Koblitz's book that they arose from viewing the modular forms as function on modular points (lattices in $\mathbb{C}$, possibly with additional structures) but I feel this is very particular to elliptic modular forms as there doesn't seem to be a similar interpretation for other kind of modular forms such as Siegel modular forms. For Siegel modular forms of general level $\Gamma = \Gamma^{(2)}(N)$, it was defined by Andrianov in his book *Modular Forms and Hecke Operators*: He proved that the commeasurator of $\Gamma^{(2)}(N)$ in $G = GSp\_4(\mathbb{Q})^+$ is the whole group $G$ (Lemma 3.1) and commented that



> 
> Using Lemma 3.1 as a point of departure, one could determine the Hecke ring
>  of the pair $(\Gamma, G)$ and then consider its representations on spaces of modular forms for the group $\Gamma$. However, the structure of the Hecke rings that arise is in general unknown, and one does not yet have a concrete general theory of Hecke operators.
> 
> 
> Because our constructions are not meant as an end in themselves, but rather as a means for studying Diophantine problems in number theory, we shall simplify the situation by, in the first place, limiting ourselves to the types of congruence subgroups that arise in arithmetic, and, in the second place, considering certain subrings of the Hecke ring of the pair $(\Gamma, G)$, rather than the entire Hecke ring.
> 
> 
> 


(I changed Andrianov's $K$ to $\Gamma$ and $S$ to $G$.)


The subring Andrianov mentioned is defined on page 124, namely the ring of Hecke operator associated to the Shimura pair $(\Gamma, S(\Gamma))$ for certain subgroup $S(\Gamma)$ of $G$ defined in equation (3.5).


Now, I have no idea how Andrianov came up with that $S(\Gamma)$. Is there a systematic explanation of the ring? I am aware that for level 1, the theory seems to be explainable from the representation theoretic perspective and Satake isomorphism.


I would imagine that a similar theory could be developed for Hermitian modular forms (as in Ikeda's paper <https://www.math.kyoto-u.ac.jp/~ikeda/hermitian>) but I am not so sure what one should change from the Siegel case. What should be "the" Hecke algebra for Hermitian modular forms of level $N$? (The group $U(2,2)$ I think does not split so I don't know how Satake isomorphism will work here.)



Hecke operators for hermitian modular forms of general level
It's best to think of these things adelically. Suppose you have a reductive group $G$ over a number field $F$, and you set 
$$\mathcal{G} = G(\mathbf{A}\_{F, f}) = \sideset{}{'}\prod\_{\text{$v$ finite place of $F$}} G(F\_v).$$
For any open compact $U \subset \mathcal{G}$ you can form the Hecke algebra $\mathbf{C}[U \backslash \mathcal{G} / U]$ of compactly supported, bi-$U$-invariant functions on $\mathcal{G}$, and this will act on the $U$-invariants of any $\mathcal{G}$-representation (in particular, on spaces of automorphic forms for $G$ of level $U$).


Now, if $U$ is of the form $\prod\_v U\_v$, you get a formula
$$\mathbf{C}[U \backslash \mathcal{G} / U] = \sideset{}{'}\bigotimes\_v \mathbf{C}[U\_v \backslash G(F\_v) / U\_v],$$
so it suffices to understand each of the local Hecke algebras -- this is the advantage of the adelic approach.


For all but finitely many $v$, the group $G(F\_v)$ is unramified (= quasi-split and splits over an unramified extension) and $U\_v$ is a hyperspecial maximal compact. In this case, the local Hecke algebra is commutative, and its structure is described by the Satake isomorphism (which works fine in this degree of generality -- it's not necessary to assume that $G$ is split). In particular, in your $U(2, 2)$ setting, for places $v$ split in the quadratic field defining the unitary group, we have $G(F\_v) \cong \operatorname{GL}\_4(F\_v)$ and the spherical Hecke algebra is the same as that of $GL\_4$, which is a polynomial ring $\mathbf{C}[X\_1, X\_2, X\_3, X\_4^{\pm 1}]$.


The problem is what to do at the bad places. There is, essentially, no hope of describing $\mathbf{C}[U\_v \backslash G(F\_v) / U\_v]$ in any concrete way for a totally general $U\_v$. Ideally, one would want to define some collection of "nice" open subgroups of $G(F\_v)$, and some "nice" (ideally commutative) subalgebras of the Hecke algebras of these subgroups, which would still be rich enough that the actions of these subalgebras on smooth representations encode the "interesting" information. For $G = \mathrm{GL}\_2$ we understand this very well, thanks to Atkin, Lehner, and Casselman. However, it's not clear what the "right" approach is for general $G$.


For $G = \operatorname{GSp}(4)$ several approaches have been tried. Andrianov's approach is to use subgroups $U\_v$ that look like "block upper-triangular" matrices, with the bottom right 2x2 block congruent to 0 modulo powers of $v$. Then you can look for a subalgebra of the Hecke algebra that somehow "comes from" the $GL\_2$ blocks along the diagonal, and that is Andrianov's $S(\Gamma)$. However, there are other approaches; Roberts and Schmidt have worked out a theory of newforms, oldforms, and Hecke operators using the "paramodular subgroups", which has essentially no overlap with Andrianov's setup.


There are generalisations for generic representations of $GL(n)$, and a few other groups of small rank, e.g. $GSp(4)$ and $U(2, 1)$; see [this question](https://mathoverflow.net/questions/89110/new-vectors-for-p-adic-groups). However, there's nothing like a general theory for arbitrary $G$.


