Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  consider the problem of efficiently computing the maximum likelihood estimator in Generalized Linear Models (GLMs) when the number of observations is much larger than the number of coefficients (n ≫ p ≫ 1). In this regime, optimization algorithms can immensely benefit from approximate second order information.  propose an alternative way of constructing the curvature information by formulating it as an estimation problem and applying a Stein-type lemma, which allows further improvements through sub-sampling and eigenvalue thresholding. Our algorithm enjoys fast convergence rates, resembling that of second order methods, with modest per-iteration cost.  provide its convergence analysis for the case where the rows of the design matrix are i.i.d. samples with bounded support.  show that the convergence has two phases, a quadratic phase follod by a linear phase. Finally,  empirically demonstrate that our algorithm achieves the highest performance compared to various algorithms on several datasets.
['Murat A. Erdogdu']
I am writing a research paper on a idea Newton-Stein method: a second order method for GLMs via Stein's Lemma
[{"from": "human", "value": "I am writing a research paper on a idea Newton-Stein method: a second order method for GLMs via Stein's Lemma"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  consider the problem of efficiently computing the maximum likelihood estimator in Generalized Linear Models (GLMs) when the number of observations is much larger than the number of coefficients (n \u226b p \u226b 1). In this regime, optimization algorithms can immensely benefit from approximate second order information.  propose an alternative way of constructing the curvature information by formulating it as an estimation problem and applying a Stein-type lemma, which allows further improvements through sub-sampling and eigenvalue thresholding. Our algorithm enjoys fast convergence rates, resembling that of second order methods, with modest per-iteration cost.  provide its convergence analysis for the case where the rows of the design matrix are i.i.d. samples with bounded support.  show that the convergence has two phases, a quadratic phase follod by a linear phase. Finally,  empirically demonstrate that our algorithm achieves the highest performance compared to various algorithms on several datasets."}]