The normal gradient descent is additive: $w\_{t+1}=w\_t-\lambda\_t\nabla f(w\_t)$, but is there a multiplicative gradient descent that looks something like $w\_{t+1}=w\_t[-\lambda\_t\nabla f(w\_t)]$?


I know there is a well-known exponentiated gradient descent (EG) algorithm, which gives $w\_{t+1}\propto w\_t\exp[-\lambda\_t\nabla f(w\_t)]$, I'm just wondering if a more general multiplicative gradient descent like the one given above exists?


Thanks!



Multiplicative gradient descent?
The most general form of such algorithms are named Mirror-Descent. This algorithm is an extension of gradient descent for non-Euclidean geometries.


For a formal explanation on how multiplicative weights (or exponentiated gradient descent) is a particular setup for Mirror-Descent see Appendix A.2 from <http://arxiv.org/abs/1407.1537> 


