Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  study accelerated descent dynamics for constrained convex optimization. This dynamics can be described naturally as a coupling of a dual variable accumulating gradients at a given rate $\eta(t)$, and a primal variable obtained as the ighted average of the mirrored dual trajectory, with ights $w(t)$. Using a Lyapunov argument,  give sufficient conditions on $\eta$ and $w$ to achieve a desired convergence rate. As an example,  show that the replicator dynamics (an example of mirror descent on the simplex) can be accelerated using a simple averaging scheme.  then propose an adaptive averaging heuristic which adaptively computes the ights to speed up the decrease of the Lyapunov function.  provide guarantees on adaptive averaging in continuous-time, prove that it preserves the quadratic convergence rate of accelerated first-order methods in discrete-time, and give numerical experiments to compare it with existing heuristics, such as adaptive restarting. The experiments indicate that adaptive averaging performs at least as ll as adaptive restarting, with significant improvements in some cases.
['Walid Krichene', 'Alexandre M. Bayen', 'Peter L. Bartlett']
I am writing a research paper on a idea Adaptive Averaging in Accelerated Descent Dynamics
[{"from": "human", "value": "I am writing a research paper on a idea Adaptive Averaging in Accelerated Descent Dynamics"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  study accelerated descent dynamics for constrained convex optimization. This dynamics can be described naturally as a coupling of a dual variable accumulating gradients at a given rate $\\eta(t)$, and a primal variable obtained as the ighted average of the mirrored dual trajectory, with ights $w(t)$. Using a Lyapunov argument,  give sufficient conditions on $\\eta$ and $w$ to achieve a desired convergence rate. As an example,  show that the replicator dynamics (an example of mirror descent on the simplex) can be accelerated using a simple averaging scheme.  then propose an adaptive averaging heuristic which adaptively computes the ights to speed up the decrease of the Lyapunov function.  provide guarantees on adaptive averaging in continuous-time, prove that it preserves the quadratic convergence rate of accelerated first-order methods in discrete-time, and give numerical experiments to compare it with existing heuristics, such as adaptive restarting. The experiments indicate that adaptive averaging performs at least as ll as adaptive restarting, with significant improvements in some cases."}]