Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  calculate analytically the rate of convergence at long times in the backpropagation learning algorithm for networks with and without hidden units. For networks without hidden units using the standard quadratic error function and a sigmoidal transfer function,  find that the error decreases as 1/t for large t, and the output states approach their target values as 1/t. It is possible to obtain a different convergence rate for certain error and transfer functions, but the convergence can never be faster than 1/t. These results are unaffected by a momentum term in the learning algorithm, but convergence can be substantially improved by an adaptive learning rate scheme.#R##N##R##N#For networks with hidden units,  generally expect the same rate of convergence to be obtained as in the single-layer case; hover, under certain circumstances one can obtain a polynomial speed-up for non sigmoidal units, or a logarithmic speed-up for sigmoidal units. Our analytic results are confirmed by empirical measurements of the convergence rate in numerical simulations.
['Gerald Tesauro', 'Yu He', 'Subutai Ahmad']
I am writing a research paper on a idea Asymptotic convergence of backpropagation
[{"from": "human", "value": "I am writing a research paper on a idea Asymptotic convergence of backpropagation"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  calculate analytically the rate of convergence at long times in the backpropagation learning algorithm for networks with and without hidden units. For networks without hidden units using the standard quadratic error function and a sigmoidal transfer function,  find that the error decreases as 1/t for large t, and the output states approach their target values as 1/t. It is possible to obtain a different convergence rate for certain error and transfer functions, but the convergence can never be faster than 1/t. These results are unaffected by a momentum term in the learning algorithm, but convergence can be substantially improved by an adaptive learning rate scheme.#R##N##R##N#For networks with hidden units,  generally expect the same rate of convergence to be obtained as in the single-layer case; hover, under certain circumstances one can obtain a polynomial speed-up for non sigmoidal units, or a logarithmic speed-up for sigmoidal units. Our analytic results are confirmed by empirical measurements of the convergence rate in numerical simulations."}]