Suppose $x\_i$ come from 2-d standard Normal centered at 0. What is the range of $a$ for which the following iteration converges almost surely?


$$w\_{i+1} = w\_i-a x\_i \langle w\_i, x\_i \rangle$$


For 1-d, one can write $w\_{i+1}^2$ as $w\_i^2=w\_0^2\prod\_i (1-\alpha x\_i^2)$, take log and apply central limit theorem. For 2-D, one could use a [similar approach](https://stats.stackexchange.com/a/586427/511), but using matrix logarithm instead of regular logarithm. However, this side-steps some technical details (we have a product of rank-1 matrices, how can you take matrix logarithm of that?) so I'm wondering if the resulting estimate is valid.


In particular, it predicts that iteration converges iff $\alpha \in (0.12131, 0.91861)$, while in [simulations](https://www.wolframcloud.com/obj/yaroslavvb/newton/forum-largest-a.nb), I'm seeing convergence for $\alpha=0.93$ (but divergence for $\alpha=0.94$)


The question is equivalent to finding $\alpha$ such that $C\_i$ converges almost surely where


$$C\_i=w\_i w\_i^T$$
$$C\_{i+1}=(1-\alpha x\_i x\_i^T)C\_i(1-\alpha x\_i x\_i^T)$$



Range of $a$ such that $w \leftarrow w-a x \langle w, x \rangle$ converges almost surely?
As you say, the threshold is around $a=0.937087$.


The exact condition for convergence is
$$
\int\_{0}^{2\pi}\int\_0^\infty\log(1-2ar^2\cos^2\theta+a^2r^4\cos^2\theta)re^{-r^2/2}\,dr\,d\theta<0.
$$
I don't think there will be a clean formula for $a$.


The derivation of the expression is as follows:
Let $r\_j=\|w\_j\|$. The entire problem is rotationally symmetric. Additionally, the operation is linear. Combining these two facts, we have $r\_{j+1}=Z\_jr\_j$ where $(Z\_j)$ is an i.i.d sequence of random variables. Taking logarithms, we have that $r\_j\to 0$ if $\mathbb E \log Z<0$ and $r\_j\to\infty$ if $\mathbb E \log Z>0$.


To compute $\mathbb E\log Z$, suppose without loss of generality that $w\_j=(1,0)$. Write $x\_{j+1}=(r\cos\theta,r\sin\theta)$ where $r$ is distributed as a Rayleigh random variable and $\theta$ is uniform on $[0,2\pi]$. Then
$$
w\_{j+1}=(1,0)-a(r\cos\theta,r\sin\theta)r\cos\theta,
$$
so that $Z\_{j+1}=\|w\_{j+1}\|$. The expression for $\mathbb E\log Z\_{j+1}$ is then as displayed in the first display equation.


