I am interested in the generating function of $SO(N)$ random matrix, that is, I want to compute
$$
Z\_N[J]=\int dM e^{{\rm Tr} (J^T M)},
$$
where $dM$ is the $SO(N)$ Haar measure, and $J$ is an arbitrary $N\times N$ matrix. From this generating function, I can generate all correlations $\langle M\_{ij}M\_{kl}\cdots\rangle$ by taking derivatives with respect to the elements of $J$.


Due to the invariance of the measure, one sees that $Z[J]=Z[U^TJV]$ with $U,V\in SO(N)$, and thus $Z$ only depends on the singular values of $J$. (Stated otherwise, $Z$ only depends on the $N$ invariants ${\rm Tr}((J^TJ)^n)$, $n=1,...,N$).


Finally, at least for $N=2$ and $N=3$, one can show that $Z$ is also invariant under permutations of the singular values of $J$ (maybe it can be generalized for all $N$ ?).


It is not too hard to compute explicitly $Z\_2[J]$, which is given in terms of a Bessel function of the sum of the two singular values of $J$.


Is there a way (or has it been done in the literature) to compute $Z\_N$ for any $N$ ? I would already be happy with $Z\_3$, which I cannot manage to compute explicitly.




---


EDIT : Here is an attempt, which kind of works for $N=2$, but for which I am stuck for $N=3$.
If we define a Laplacian $\Delta=\sum\_{ij}\frac{\partial^2}{\partial J\_{ij}^2}$ (with $J\_{ij}$ the elements of $J$), one shows easily that
$$
\Delta Z[J]=N Z[J]. \tag{1}
$$
If we call $\lambda\_i$ the singular values of $J$ (with $\lambda\_1>\lambda\_2>\ldots$), using the fact that $Z[J]=Z[\lambda\_1,\lambda\_2,\ldots]$, one shows (at least for $N=2$ and $N=3$, but it might be generalizable to $N\geq4$) that
$$
\Delta Z=\frac{1}{D}\sum\_{i}\frac{\partial}{\partial \lambda\_i}\left(D\frac{\partial}{\partial \lambda\_i}Z\right),
$$
where $D=\prod\_{i< j}(\lambda\_i^2-\lambda\_j^2)$ is related to the Jacobian to go from $J\_{ij}$ to $\lambda\_i$. This equation looks nice enough, so my hope is that a solution exists, I am not quite sure how to find it for $N=3$.


In the case $N=2$, we can compute $Z\_2$ exactly via its definition, and it reads $Z\_2[\lambda\_1,\lambda\_2]=I\_0(\lambda\_1+\lambda\_2)$, with $I\_\nu$ the modified Bessel function of the first kind. One checks that this is indeed a solution of Eq. (1). 


Unfortunately, even in that case, it is not clear to me how to find this solution starting from Eq. (1) only. Given that $D=\lambda\_1^2-\lambda\_2^2$, it is tempting to defined $u=(\lambda\_1+\lambda\_2)/2$ and $v=(\lambda\_1-\lambda\_2)/2$. Then Eq. (1) is solved by separation of variables and we find a family of solution $Z\_{2,\mu}$ (where I have already use the fact that $Z\_N[0]=1$) :
$$
Z\_{2,\mu}[u,v]=I\_0(\sqrt{\mu}u)I\_0(\sqrt{4-\mu}v).
$$
Clearly the solution to my problem corresponds to $\mu=4$, but it is not clear to me what is the rigorous argument to pick this value of $\mu$ (since $Z\_N>0$ $\forall J$, we must have $\mu\geq 4$ as $I\_0$ can be negative for imaginary variables; but how to select $\mu=4$ as the only viable solution ?).


One way to solve this issue of $\mu$ is to use the fact that $Z\_2[J=\lambda Id\_2]$ can be computed explicitly: $Z\_2[J=\lambda Id\_2]=I\_0(2\lambda)$, which unambiguously selects $\mu=4$. 


Since we can always compute the expansion of $Z\_N[\lambda Id\_N]$ explicitly at least for small $\lambda$, this kind of argument might be enough to fix the constant also for $N>2$. For $N=3$, a few special cases can be computed explicitly, which might help too.




---


Any insight for the solution of Eq. (1) would be greatly appreciated.



Generating function of $SO(N)$ random matrix
**Singular value decomposition:** The real $N\times N$ matrix $J$ has singular values $\sigma\_1,\sigma\_2\ldots\sigma\_N\geq 0$ in the decomposition $J=U\,{\rm diag}\,(\sigma\_1,\sigma\_2\ldots\sigma\_N)V$ with $U,V\in{\rm O}(N)$. We need to restrict $U,V$ to ${\rm SO}(N)$, which we can do by allowing for one of the singular values to become negative:
$$J=U\,{\rm diag}\,(\lambda\_1,\lambda\_2\ldots\lambda\_N)V,\;\;U,V\in{\rm SO}(N),$$
$$\lambda\_n=\sigma\_n\geq 0,\;\;{\rm for}\;\;n=1,2,\ldots N-1,$$
$$\lambda\_N=\begin{cases}
\sigma\_N\;{\rm if}\;{\rm det}\,M\geq 0,\\
-\sigma\_N\;{\rm if}\;{\rm det}\,M<0.
\end{cases}$$
Note that this need to consider the $\lambda\_n$'s implies that the integral $Z\_N$ is not only a function of the invariants ${\rm Tr}((J^TJ)^n)$. 


Without loss of generality we may now take a diagonal $J={\rm diag}\,(\lambda\_1,\lambda\_2,\ldots \lambda\_N)$ and then we need the integral
$$Z\_N=\int\_{{\rm O}(N)} dM \exp\left(\sum\_{n=1}^N \lambda\_n M\_{nn}\right).$$  

**Large-$N$ limit:** I do not think there is a closed form expression for arbitrary $N$, but for $N\gg 1$ the diagonal matrix elements of $M$ are independent Gaussians (mean zero, variance $1/N$), resulting in
$$Z\_N\rightarrow \exp\left(\frac{1}{2N}\sum\_{n=1}^N \lambda\_n^2\right),\;\;N\gg 1.$$
This result for $Z\_N$ in the large-$N$ limit depends only on the singular values (because $\lambda\_n$ enters only quadratically), but for small-$N$ this is no longer true. 


**Small-$N$ results:**
For small $N$ we can use the parametrization of the orthogonal group given in Appendix B of [arXiv:1405.3115.](https://arxiv.org/abs/1405.3115) 


• $N=2$
$$Z\_2=\frac{1}{\pi}\int\_0^{\pi} \exp\bigl[(\lambda\_1+\lambda\_2)\cos\theta\bigr]\,d\theta=I\_0(\lambda\_1+\lambda\_2)$$
Note that in terms of the singular values considered in the OP, this would read $Z\_2=I\_0(\sigma\_1\pm\sigma\_2)$, where the $\pm$ sign is the sign of ${\rm det}\,J$.


• $N=3$
$$Z\_3=\frac{1}{8\pi^2}\int\_0^{2\pi} d\alpha\int\_0^{2\pi} d\alpha'\int\_0^\pi \sin\theta\, d\theta$$
$$\qquad\qquad\times\exp\left[\cos \alpha \cos \alpha' (\lambda\_1+\lambda\_2 \cos \theta)-\sin \alpha \sin \alpha' (\lambda\_1 \cos \theta+\lambda\_2)+\lambda\_3\cos \theta\right]$$
The two integrals over $\alpha,\alpha'$ can be carried out (by rewriting them as integrals over $\alpha\pm\alpha'$), with the result
$$Z\_3=\frac{1}{2}\int\_{-1}^1 I\_0\left[\tfrac{1}{2}(\lambda\_1-\lambda\_2) (1-x)\right] I\_0\left[\tfrac{1}{2} (\lambda\_1+\lambda\_2)(1+x)\right]\,e^{\lambda\_3 x}\,dx.$$
As shown by [Paul Enta](https://mathoverflow.net/a/302646/11260), this integral can be written as the inverse Laplace transform
${\cal L}^{-1}[F(s)](t)$ for $t=1$ of the function 
$$F\_3(s)=\frac{1}{\sqrt{(s-\lambda\_1-\lambda\_2-\lambda\_3) (s+\lambda\_1+\lambda\_2-\lambda\_3) (s+\lambda\_1-\lambda\_2+\lambda\_3) (s-\lambda\_1+\lambda\_2+\lambda\_3)}},$$
to demonstrate that it is invariant under permutation of the $\lambda\_n$'s.  

Notice also that $F\_3(s)$ is invariant under a sign change of two of the $\lambda\_n$'s, but not under a sign change of one single $\lambda\_n$. This is consistent with the definition of the $\lambda\_n$'s in terms of the singular values, given above. 


A closed-form expression of either the Bessel-function integral or the inverse Laplace transform seems not forthcoming, except for some special cases (see the comments by Adam).


**Average over the full orthogonal group:** If we integrate over the full group ${\rm O}(N)$, instead of only over ${\rm SO}(N)$, we should combine the results for $\pm\lambda\_N$,
$$Z\_{{\rm O}(N)}(\lambda\_1,\lambda\_2,\ldots \lambda\_{N-1},\lambda\_N)=
\tfrac{1}{2}Z\_{{\rm SO}\,(N)}(\lambda\_1,\lambda\_2,\ldots \lambda\_{N-1},\lambda\_N)+
\tfrac{1}{2}Z\_{{\rm SO}(N)}(\lambda\_1,\lambda\_2,\ldots \lambda\_{N-1},-\lambda\_N).$$
There is now no need to distinguish the $\lambda\_n$'s from the singular values $\sigma\_n$. In particular, for $N=2$ one has
$$Z\_{{\rm O}(2)}=\tfrac{1}{2}I\_0(\sigma\_1+\sigma\_2)+\tfrac{1}{2}I\_0(\sigma\_1-\sigma\_2).$$


