In the following, let $G=\operatorname{GL}\_n(\mathbb{C})$ or $G=\operatorname{\mathbb PGL}\_n(\mathbb{C})$, depending on whichever has a better chance of yielding an affirmative answer. One could more generally ask the question for a complex, (reductive, affine,) algebraic group - but since this generalization is not of importance to me, I leave it as a bonus objective.


**Question:** I am asking whether there is a scheme that parametrizes the closed, algebraic subgroups of $G$ in the same way that the Hilbert scheme classifies all closed subvarieties.


Now, I am quite certain that $G$ has a Hilbert scheme $\operatorname{\mathfrak Hilb}\_G$. Then, I might refine the question by asking whether the closed subgroups of $G$ form a (possibly closed) subscheme of $\operatorname{\mathfrak Hilb}\_G$. 


Finally, of course I would like to know as much as possible about how that scheme, if it exists, looks like (as in, geometric properties of any kind are welcome).


**My thoughts:** Denoting by $\mu:G\times G\to G$ the group operation and by $\iota:G\to G$ inversion, it seems plausible to me that the $H\in\operatorname{\mathfrak Hilb}\_G$ which satisfy $\mu(H\times H)=H$ and $\iota(H)=H$ should form a closed subscheme, which would then be exactly what I am looking for. However, I also feel that I might only have a naive and vague idea of what is going on.



Is there a scheme parametrizing the closed subgroups of an algebraic group?
Here is an affirmative answer in the sense of algebraic spaces under a reductivity hypothesis on the subgroups, using some hard input from SGA3. (The representability by a scheme for the functor classifying parabolic subgroups is well-documented in the literature in various settings; see section 3 of Exp. XXVI in SGA3 for the case of a general base scheme.) Let $G \rightarrow S$ be a smooth affine group scheme, and let $F$ be the functor on $S$-schemes
$$F(T) = \{H \subset G\_T \mbox{ a reductive closed subgroup scheme}\}$$
where "reductive $T$-group" means "smooth affine $T$-group with connected reductive geometric fibers".


For any $T$ and $H \in F(T)$, the isomorphism type of the fiber of $H \rightarrow T$ over a geometric point $t$ of $T$ is determined by the root datum for $H\_t$, and it is a result in the relative theory of reductive groups that this root datum is "locally constant" in $t$. More precisely, if $\Sigma$ is the set of isomorphism classes of root data and for $\sigma \in \Sigma$ we define $T\_{\sigma}$ to be the set of points of $T$ over which the geometric fiber of $H$ has root datum isomorphic to $\sigma$ then the $T\_{\sigma}$'s are pairwise disjoint and *open* in $T$. Thus, if we define the subfunctor $F\_{\sigma} \subset F$ to consist of those "points" of $F$ whose associated geometric fibers $H\_t$ all have root datum $\sigma$ then the $F\_{\sigma}$'s are open subfunctors which cover $F$ disjointly. More specifically, the representability of $F$ is equivalent to that of the $F\_{\sigma}$'s, and in particular, if $F\_{\sigma}$ is represented by some $M\_{\sigma}$ then $F$ is represented by the disjoint union of the $M\_{\sigma}$'s. This is the same as the game whereby the construction of Hilbert schemes is reduced to the case of a fixed Hilbert polynomial.


Now we fix $\sigma$ and focus on $F\_{\sigma}$. Consider the associated Chevalley group $G\_{\sigma}$ over $\mathbf{Z}$ (i.e., the unique "split" reductive $\mathbf{Z}$-group whose geometric fibers have root datum $\sigma$). By a hard fact in the general theory of reductive group schemes, a reductive group $H \rightarrow T$ has all geometric fibers with root datum $\sigma$ if and only if $H$ and $(G\_{\sigma})\_T$ become isomorphic over an etale cover of $T$. So, informally, $F\_{\sigma}$ is the functor of closed subgroup schemes which are etale forms of $(G\_{\sigma})\_T$. This motivates us to look at the smooth separated Hom-scheme $$\mathscr{H} := \underline{\rm{Hom}}((G\_{\sigma})\_S,G)$$
whose set of $T$-points is the set of $T$-group scheme homomorphisms $(G\_{\sigma})\_T \rightarrow G\_T$ (see SGA3 XXIV Cor. 7.2.3 for existence) naturally in $T$. Over $\mathscr{H}$ there is a "universal homomorphism" 
$$f:(G\_{\sigma})\_{\mathscr{H}} \rightarrow G\_{\mathscr{H}}.$$


I claim that the condition of being a closed immersion is represented by an open subscheme $X$ of $\mathscr{H}$, so then $X$ represents the functor assigning to any $S$-scheme $T$ the set of reductive closed $T$-subgroup schemes $H \subseteq G\_T$ *equipped with* a $T$-group isomorphism $\varphi:H \simeq (G\_{\sigma})\_T$. Let's grant this. There is an evident map $X \rightarrow F\_{\sigma}$ that forgets $\varphi$, and this is a surjection for the etale topology since all $H \in F\_{\sigma}(T)$ become isomorphic to $(G\_{\sigma})\_T$ over an etale cover of $T$ (as noted above). Thus, $X \rightarrow F\_{\sigma}$ is a torsor for the etale topology relative to the natural action of the automorphism scheme of $G\_{\sigma}$. That is, $F\_{\sigma}$ is the quotient sheaf for $X$ modulo the faithful action of the smooth separated $S$-group $({\rm{Aut}}\_{G\_{\sigma}/\mathbf{Z}})\_S$ that is an extension of the constant etale $S$-group ${\rm{Aut}}(\sigma)\_S$ by the adjoint semisimple $S$-group $(G\_{\sigma}^{\rm{ad}})\_S := (G\_{\sigma}/Z\_{G\_{\sigma}})\_S$, so by Artin's work on algebraic spaces it follows that this quotient sheaf $F\_{\sigma}$ is an algebraic space, even smooth (with the algebraic space $X/(G\_{\sigma}^{\rm{ad}})\_S$ as an $S$-smooth etale cover). 


If the root datum $\sigma$ is semisimple then $(G\_{\sigma})\_S$ is semisimple and so its the automorphism scheme is $S$-affine, hence *quasi-compact* over $S$, so $F\_{\sigma}$ is quasi-separated over $S$. Thus, in such cases the valuative criterion for separatedness is applicable, giving that $F\_{\sigma}$ is also separated. For general $\sigma$ separatedness might still hold but one would have to look more closely at the contributions from the derived group and maximal central torus of $G\_{\sigma}$; probably nobody cares, so I won't delve into that. 


It remains to build the open subscheme $X$ as above, so we show more generally:


**Theorem**. *Let $f:G' \rightarrow G$ be a homomorphism between affine fppf group schemes over a scheme $S$, with $G'$ reductive. Then the set $U$ of $s \in S$ such that $f\_s$ is a closed immersion is open in $S$ and $f\_U:G'\_U \rightarrow G\_U$ is a closed immersion.*


Maybe this is handled somewhere in SGA3; I don't remember offhand. The proof of this turned out to be a bit more involved than I expected, but I may be overlooking something simpler.


*Proof*: We may and do assume $S$ is noetherian. By the very difficult SGA3, XVI, 1.5(a), $f$ is a closed immersion if and only if it is a monomorphism. Hence, once we show that $U$ is open it will suffice to show that the $U$-group $\ker(f\_U)$ is trivial. The $S$-group $\ker(f)$ is relatively affine and finite type over $S$, so it is trivial if and only if all of its fibers are trivial (by Nakayama's Lemma), so it remains to prove that $U$ is open. 


By "spreading out" considerations and passing to local rings on $S$ at such points $s$, we may assume $S$ is local noetherian and just need to check that $\ker f = 1$ if $\ker(f\_s) = 1$ for the closed point $s$ of $S$. Since it is enough to check fiberwise triviality, by picking a dvr whose special point dominates $s$ and whose generic point dominates another chosen point of $S$ we may assume $S = {\rm{Spec}}(A)$ for a dvr $A$, say with fraction field $K$. Thus, the schematic closure of $f\_K(G'\_K)$ inside $G$ is a *flat* affine $A$-group of finite type that we shall call $\mathbf{G}$. Clearly $f$ factors through an $A$-homomorphism
$G' \rightarrow \mathbf{G}$ that is a closed immersion between special fibers and surjective between generic fibers. Thus, comparison of fiber dimensions implies that $f$ has finite kernel between generic fibers, so $\ker f$ is quasi-finite over $S$. 


The generic fiber $\mathbf{G}\_K = f\_K(G'\_K)$ is a quotient of $G'\_K$ and so is $K$-smooth. Thus, we can form the group smoothening $\mathbf{G}' \rightarrow \mathbf{G}$ (see Theorem 5 in 7.1 of "Neron Models"); by construction this is a smooth $A$-group with the same $K$-fiber as $\mathbf{G}$ and it is moreover affine since $\mathbf{G}$ is affine. By its universal property, we get a factorization $G' \rightarrow \mathbf{G}'$ whose special fiber has trivial kernel and hence is a closed immersion. It suffices to show that this is a closed immersion between $K$-fibers, so we may replace $G$ with $\mathbf{G}'$ to reduce to the case of a smooth target $G$ with $f\_K$ is surjective (so $G\_K$ is connected). Since $f$ induces an isogeny between the smooth identity components on fibers over $S$, the identity components of the fibers of $G'$ are *reductive*. 


Let $G^0$ be the open complement of the non-identity component of the special fiber, so $G^0 \rightarrow S$ is smooth, separated, and finite type with connected reductive fibers. By a clever translation argument in SGA3, XXV, section 4, a quasi-finite homomorphism between separated flat group schemes of finite type over a Dedekind base is an affine morphism, so $G^0$ is *affine* (as $G$ is). Thus, $G^0$ is a reductive group scheme, so the open immersion $j:G^0 \hookrightarrow G$ between smooth affine $A$-groups is also a *closed immersion* (SGA3, XVI, 1.5(a)). The defining ideal for this closed immersion vanishes on the $K$-fibers and hence vanishes; i.e., $G^0 = G$. We conclude that $f:G' \rightarrow G$ is a fiberwise isogeny for dimension, quasi-finiteness, and connectedness reasons. But it is a general (hard!) fact that a fiberwise isogeny between reductive group schemes is always *finite* flat. Since $f$ is a closed immersion between special fibers, it must be an isomorphism there and hence as a finite flat map its degree is 1. Thus, $f$ is an isomorphism, so $\ker f = 1$ as desired.


QED 


