Let $f: \mathbb{R}^d\rightarrow\mathbb{R}\_{\geq 0}$ be a function which is convex and smooth (*i.e.*, in $C^{\infty}$). If $x^\* \in \mathbb{R}^d$ is the (global) minimum of $f$, it is [well known](https://rkganti.wordpress.com/2015/08/21/convergence-rate-of-gradient-descent-algorithm/) that gradient descent with a (sufficiently small) fixed step size $\eta$,
$$ x\_{n+1} = x\_{n} - \eta \nabla f(x\_n), $$
converges to the global minimum $x\_n \rightarrow x^\*$.


**Question**: What is the asymptotic behavior of gradient descent when a finite minimizer $x^\*$ does not exist (*i.e.*, when $\forall x \in \mathbb{R}^d : \nabla f(x)\neq 0$)? 


My guess is that $x\_n/||x\_n||$ should converge to some finite solution, but I could not find a proof. Or perhaps there is some counter-example to this claim?


Thanks in advance!



Asymptotic behavior of gradient descent on a smooth, convex, non-negative function with no finite minimum
I think there are simple counterexamples of the form $f(x,y)=\phi(x)+\psi(y)$ (here $x$ and $y$ denote real variables), where $\phi$ and $\psi$ are smooth, positive, decreasing, strictly convex functions.


The idea is that, $-\psi'(x)$ and $-\phi'(x)$ being positive decreasing functions, is not an obstruction for their ratio to oscillate between arbitrarily large and small values, on arbitrarily large intervals. This produces zig-zag orbits $z\_n:=(x\_n,y\_n)$ for which $z\_n/\|z\_n\|$ may accumulate to both $(1,0)$ and $(0,1)$.


**Construction.** Define some smooth, positive, decreasing functions $u(t)$ and $v(t)$ such that for all $n\in\mathbb{N}$
$$u(x):={1\over (n!)^2}\quad\text{for $n$ even, and}\; n!+1\le x\le (n+2)!$$
and
$$v(x):={1\over (n!)^2}\quad\text{for $n$ odd, and}\; n!+1\le x\le (n+2)!$$


(so one has only to extend them smoothly for $x\le2$ and between $n!$ and $n!+1$ to have them defined everywhere). As a consequence we have
$${v(x)\over u(x)}=\begin{cases}
n^{-2} &\text{for $n$ odd, and }\; n!+1\le x\le (n+1)! \\
n^2 &\text{for $n>0$ even, and }\; n!+1\le x\le (n+1)! \\
\end{cases}$$
while
$$n^{-2}\le {v(x)\over u(x)}\le n^2 \quad \text{for any $n>0$ and }\; n! \le x\le n!+1 \ .$$


Note also that $u$ (respectively $v$) has finite integral on any right-unbounded interval, because the contribute to the integral on any interval $[n!+1, (n+2)!+1]$ (for $n$ even, resp. odd) is bounded by $(n+2)!/(n!)^2=(n+2)(n+1)/n!$. Therefore we can consider the smooth, decreasing, strictly convex functions
$$\phi(x):=\int\_x^\infty u(t)dt\quad\text{and }\quad \psi(x):=\int\_x^\infty v(t)dt\; .$$


The negative gradient iteration $u\_{n+1}=u\_n-\eta\nabla f(u\_n)$ writes, with $z\_n:=(x\_n,y\_n)$
$$
\begin{cases}
x\_{n+1}=x\_n+\eta \,u(x\_n) \\
y\_{n+1}=y\_n+\eta \,v(y\_n)\\
\end{cases}
$$
and it follows from the above identities and inequalities on $v/u$ that it produces sequences $(x\_n,y\_n)$ with $\liminf\_{n\to+\infty}y\_n/x\_n=0$ and $\limsup\_{n\to+\infty}y\_n/x\_n=+\infty$,
corresponding to a sequence $z\_n/\|z\_n\|$ that clusters both to $(1,0)$ and to $(0,1)$.


**Sketch of the computation.** Both sequences $x\_m=x\_0+\eta\sum\_{k=0}^{m-1}u(x\_j)$ and $y\_m=x\_0+\eta\sum\_{k=0}^{m-1}v(y\_j)$ are increasing and diverging. 
Let's define $\mu(t)$ as the largest integer $j$ such that $x\_j\le n!$ and 
$\nu(t)$ as the largest integer $j$ such that $y\_j\le n!$. The point is to use the above identities and inequalities on $u$ and $v$ to show that, for even $n$ and $m=\mu(n!)$, the principal part in both the above sum is given by the indices $j$ such that $\mu(n!)\le j\le \mu((n+1)!)$, so that $y\_m/x\_m=O(1/n^2)$. Analogously, for odd $n$ and $m=\nu(n!)$, one should get $x\_m/y\_m=O(1/n^2)$.


