Given that the random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}\_{M},\sigma\_{x}^{2}\textbf{I}\_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}\_{M},\sigma\_{y}^{2}\textbf{I}\_{M})$ are independent, what would be the p.d.f. of


$$Z = \left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2, $$
where $\mathcal{CN}(.,.)$ is the complex normal random variable.


Which kind of variable change could be applied here?



p.d.f. of $\left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2$, where $\textbf{x}$ and $\textbf{y}$ are complex Gaussians?
You can use the rotational invariance of the Gaussian distribution to take $\mathbf{y}=2^{-1/2}\sigma\_y(\sqrt \xi\_{2M},0,0,\ldots,0)$, with $\xi\_{2M}$ distributed independently of $\mathbf{x}$ according to a chi-squared distribution with $2M$ degrees of freedom. Then 
$$Z=\tfrac{1}{2}\sigma\_y^2 \xi\_{2M}|z|^2\;\;\text{with}\;\; z=\frac{x\_1}{\sum\_{n=1}^M |x\_n|^2}.$$
This can be further reduced to
$$Z=\frac{\sigma\_y^2}{\sigma\_x^2} \frac{\xi\_{2M}\xi\_{2}}{(\xi\_{2}+\xi\_{2M-2})^2},$$
with $\xi\_2$ and $\xi\_{2M-2}$ independently chi-squared distributed with $2$, respectively, $2M-2$ degrees of freedom.


I don't think the distribution of this rational function of three independent chi-squared distributions has a closed form expression. For $M\gg 1$ one has simply 
$$Z\rightarrow \frac{\sigma\_y^2}{2M\sigma\_x^2}\xi\_2,$$
so $Z$ has for large $M$ a chi-squared distribution with 2 degrees of freedom.


