Let $\pi\_1, \pi\_2$ be two $k$-dimensional subspaces of $\mathbb R^n$. Using elements of the orthogonal group $O(n)$, how much can we simplify $\pi\_1, \pi\_2$? Certainly there always exists $A \in O(n)$ such that $A \cdot \pi\_1$ is the span of the first $k$ canonical bases, but this doesn't say anything about $A \cdot \pi\_2$.


More precisely, I'd like to know if there is a standard, well-known fundamental domain of the $O(n)$-action on $Gr(k, n) \times Gr(k, n)$, given by $A \cdot (\pi\_1, \pi\_2) = (A \pi\_1, A \pi\_2)$. Since $\dim Gr(k, n) = k(n-k)$ and $\dim O(n) = n(n-1)/2$, the quotient space should have the following dimension:
$$\dim \frac{Gr(k, n) \times Gr(k, n)}{O(n)} = 2k(n-k) - \frac12 n(n-1) = -2 \left( k^2 -nk + \frac 14 (n^2 - n) \right)$$
This is a quadratic expression in $k$, and it is a positive number when $n - \sqrt n < 2k < n + \sqrt n$. (For example [see this](https://www.wolframalpha.com/input?i=plot+2x%28n-x%29+-+n%28n-1%29%2F2%2C+with+n+%3D+100)) Outside this range, the quotient space should be a discrete collection of points. Within the range, $O(n)$ shouldn't be able to exhaust the degrees of freedom of $Gr(k, n) \times Gr(k, n)$, and thus the quotient space should have extra degrees of freedom. As a simple example, $(k, n) = (2, 4)$ will produce 2 degrees of freedom and $(k, n) = (1, 4)$ will produce 0 degrees of freedom.



Fundamental domain for two Grassmannians
Your argument about the dimension of the quotient doesn't take into account that there may be elements of the orthogonal group that don't do anything to the pair $(\pi\_1,\pi\_2)$. For example, if $2k<n-1$, then the span of the two planes has codimension at least $2$ in $\mathbb{R}^n$, so there will be rotations that are the identity on both $\pi\_1$ and $\pi\_2$, and you have to subtract that stabilizer as well.


More generally, here is what you can say: Define a quadratic form $Q$ on $\pi\_1$ by letting $Q(v) = |v'|^2$ where $v'$ is the orthogonal projection of $v\in \pi\_1\subset\mathbb{R}^n$ onto $\pi\_2$. Since $|v'|^2\le |v|^2$ with equality if and only if $v$ lies in $\pi\_2$, we see that the eigenvalues of $Q(v)$ with respect to the 'natural' norm $|v|^2$ are all less than or equal to $1$. Let those eigenvalues be $1\ge \cos(\theta\_1)^2\ge \cos(\theta\_2)^2\ge\cdots\ge\cos(\theta\_k)^2\ge 0$, where $\theta\_i\in[0,\pi/2]$. Then there will exist an orthonormal basis $e\_1,\ldots,e\_k$ of $\pi\_1$ and an orthonormal basis $f\_1,\ldots,f\_k$ of $\pi\_2$ such that $e\_i' = \cos\theta\_i\, f\_i$.


In fact, it's not hard to see that the quantity $\theta = (\theta\_1,\theta\_2,\ldots,\theta\_k)$, where $0\le\theta\_1\le\theta\_2\le\cdots\le\theta\_k\le \pi/2$, completely determines the pair $(\pi\_1,\pi\_2)$ up to $\mathrm{O}(n)$ equivalence. Moreover, the sequence $\theta$ has to start with at least $2k{-}n$ zeroes (since the intersection of $\pi\_1$ and $\pi\_2$ has to have dimension at least $2k{-}n$). Beyond this, there is no restriction on the $\theta\_i$, so the moduli space is a $j$-simplex where $j= k - \max(0,2k{-}n) = \min(k,n{-}k)$.


**N.B.:** I feel that I would be remiss in not mentioning that this is a very special case of the 'two-point' problem for Riemannian symmetric spaces. The Grassmannian of $k$-planes in $\mathbb{R}^n$ is, of course, a Riemannian symmetric space of rank $\min(k,n{-}k)$, and the above description generalizes to Riemannian symmetric spaces of rank $r$, i.e., that the moduli space of pairs of points in a Riemannian symmetric space $M=U/K$ of rank $r$ is naturally an $r$-dimensional polyhedron. If you want to know more about the general case, I suggest consulting a good text on symmetric spaces (cf. Helgason) that describes the action of the reduced Weyl group.


