Let $M$ be a non-compact matrix Lie group and $T\_e M$ its lie algebra. 


Consider a point $x \in M $ and $ \triangle \in T\_e M$. 


To move from $x$ to a point $y \in M$ along $\triangle$, below group operation seems to be commonly used in iterative optimization on Lie groups.


$y = x \exp(\triangle)$


Is this straight forward? How can this operation be explained?



Explanation of $y = x \exp(\triangle)$ for a Lie Group
The answer of Theo basically says it all what the exponential is concerned, but I maybe can shed some light regarding the optimization perspective. 


Let $M$ be your Lie group and suppose it is a subgroup of $\textrm{GL}\_n$. Now, if you want to solve
\[
 \min f(x) \quad \textrm{s.t. $x \in M$}
\]
by an iterative method, one usually uses a so-called smooth retraction map
\[
 R\_x: T\_x M \to M. 
\]
to replace your current approximation $x$ to $x\_+ := R\_x(\Delta)$. Think for example of doing a line-search $R\_x(-t \textrm{ grad}\_x f)$ for $t>0$ where $\textrm{ grad}\_x f$ is the Riemannian gradient of $f$.


(The retraction map has to fulfill some properties, in order for this to work, likes smoothness and being a first-order approximation of the geodesics (see below).)


Due to the left (or right) action of a Lie group on itself by multiplication, the exponential mapping at the identity $\exp$ can be transported to get a retraction at $x$ as $x\exp$. As Theo already explained, $\exp$ does not need to be a global diffeomorphism, but that is not needed for optimization, since we perform updates locally.


Another typical choice for $R\_x$ are the geodesics in $x$. For some metrics, the $\exp$ coincides with the geodesics (for instance, bi-invariant metrics), but not always. One can also use cheaper alternatives for $R\_x$ if your are only concerned with optimization. A nice reference for this retraction-based optimization on manifolds (and so, also Lie groups) is <http://press.princeton.edu/titles/8586.html> .


