Let $G\_1$ and $G\_2$ be connected reductive algebraic groups defined over $\mathbb{C}$ and let $V\_1$ and $V\_2$ be irreducible representations of $G\_1$ and $G\_2$ respectively. I'm interested in general techniques for computing the (generators of the) invariant polynomial functions on the representation $V\_1 \otimes V\_2$ of the group $G\_1 \times G\_2$. 


In particular, I'm interested in computing a generating set for $\mathbb{C}[V\_1 \otimes V\_2]^{SO\_n \otimes SO\_m}$, where $V\_1 = \mathbb{C}^n$ and $V\_2 = \mathbb{C}^m$ are the natural modules these special orthogonal groups ($n \geq m >2$). My first guess is to treat $V\_1 \otimes V\_2$ as a direct sum of $m$ copies of $V\_{1}$ and use the first formulation of the Fundamental Theorem for $SO\_n$ which gives me a generating set for $\mathbb{C}[V\_{1}^{m}]^{SO\_n}$ (which all have degree $2$, being the orthogonal contractions $(v,w)$?). But now I'm stuck trying to figure out what structure this algebra has as an $SO\_m$-module, and what the degrees of the generators of the invariant polynomial algebra should thus be. 


According to a table in "Some Remarks on Nilpotent Orbits" by V. Kac, $\mathbb{C}[V\_1 \otimes V\_2]^{SO\_n \otimes SO\_m}$ should be generated by $m$ invariant polynomials of degrees $2,4,6,...$ 


This seems like it should be a fairly simple exercise, but I'm just going about it the wrong way. 



Invariant polynomials for a product of algebraic groups
There is a fruitful way of thinking about the algebra $\mathbb{C}[M\_{n,m}]^{{O\_n}\times O\_m}$ that originates from the $(GL\_n, GL\_m)$-duality. Namely, 



> 
> $$
> \mathbb{C}[M\_{n,m}]=\bigoplus\_{\lambda}V\_n^{\lambda}\otimes V\_m^{\lambda},
> $$
> 
> 
> 


where the sum is over all partitions $\lambda$ with at most $\min(n,m)$ parts and $V\_n^\lambda$ is the polynomial representation of $GL\_n$ with highest weight $\lambda=(\lambda\_1,\lambda\_2,\ldots)$ with zeros appended at the end to make it length $n,$ and similarly for $m.$ Now, since $O\_n$ is a spherical subgroup of $GL\_n$, the space of $O\_n$-invariants $V\_n^{\lambda}$ is at most one-dimensional; it is non-zero precisely when $\lambda$ is *even* (i.e. all parts are even). A good source for these results is Roger Howe's *Schur Lectures*.


It follows that as a vector space, 



> 
> $$
> \mathbb{C}[M\_{n,m}]^{O\_n\times O\_m}=\bigoplus\_{\lambda}(V\_n^{\lambda})^{O\_n}\otimes (V\_m^{\lambda})^{O\_m},
> $$
> 
> 
> 


and now the sum is over even partitions with the same restriction as before, and every summand is one-dimensional and is spanned by an explicitly described polynomial on $M\_{n,m}$. Using standard techniques (described in the Schur lectures), this description can be amplified to yield the algebra structure as well. Namely, the algebra is graded by an affine semigroup of rank $\min(n,m)$ and is freely generated by explicit elements in degrees $2k$ for $1\leq k\leq \min(n,m).$ With a bit of extra work, one can also handle the more general case of $SO\_n\times SO\_m$ invariants.




---


Let me connect this description of the invariants with Abdelmalek's description following the path that Neil originally had in mind. Assume for concreteness that $n\geq m.$ The First Fundamental Theorem for $O\_n$ in geometric form states that the map


$$
q:M\_{n,m}\to Sym\_{m,m} \qquad X\mapsto X^{T}X
$$
is the geometric quotient, i.e. it is $O\_n$-invariant and gives rise to the isomorphism


$$
\mathbb{C}[M\_{n,m}]^{O\_n}\simeq \mathbb{C}[Sym\_{m,m}].
$$


(The restriction $n\geq m$ assures that the image of $q$, which consists of symmetric matrices of rank at most $\min(n,m),$ is all of $Sym\_{m,m}.$)


It is also clearly $GL\_m$-equivariant, where $GL\_m$ acts on the symmetric matrices of order $m$ by 


$$
Y\to g^T Y g\qquad Y\in Sym\_{m,m}, g\in GL\_m,
$$


and, *a fortiori*, $O\_m$-invariant. Now pass to the $O\_m$-invariants! Under this identification, 



> 
> $$
> \mathbb{C}[M\_{n,m}]^{O\_n\times O\_m} \simeq \mathbb{C}[Sym\_{m,m}]^{O\_m}.
> $$
> 
> 
> 


Thus the problem is reduced to determination of the orthogonal invariants of a symmetric matrix $Y$. It is well known that this invariant ring is freely generated by the coefficients of the characteristic polynomial of $Y.$ The coefficient of $\lambda^{m-k}$ has degree $k$ in the entries of $Y$ and, up to a sign, it is equal to the sum of the principal minors of $Y=X^T X$ of order $k, 1\leq k\leq m.$ Thus the fundamental invariants have degrees $2k, 1\leq k\leq m,$ in the entries of $X.$ Additionally, it can be verified that the $k$th fundamental invariant corresponds to the summand with $\lambda=(1^k)$ in the direct decomposition of $\mathbb{C}[M\_{n,m}]^{O\_n\times O\_m}$ coming from the $(GL\_n,GL\_m)$-duality (i.e. the second highlighted decomposition above).


