Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Error back propagation (EBP) is now the most used training algorithm for feedforward artificial neural networks (FFANNs). Hover, it is generally believed that it is very slow if it does converge, especially if the network size is not too large compared to the problem at hand. The main problem with the EBP algorithm is that it has a constant learning rate coefficient, and different regions of the error surface may have different characteristic gradients that may require a dynamic change of learning rate coefficient based on the nature of the surface. Also, the characteristic of the error surface may be unique in every dimension, which may require one learning rate coefficient for each ight. To overcome these problems several modifications have been suggested. This survey is an attempt to present them together and to compare them. The first modification was momentum strategy where a fraction of the last ight correction is added to the currently suggested ight correction. It has both an accelerating and a decelerating effect where they are necessary. Hover, this method can give only a relatively small dynamic range for the learning rate coefficient. To increase the dynamic range of the learning rate coefficient, such methods as the bold driver and SAB (self-adaptive back propagation) re proposed. A modification to the SAB that eliminates the requirement of selection of a good learning rate coefficient by the user gave the SuperSAB. A slight modification to the momentum strategy produced a new method that controls the oscillation of ights to speed up learning. Modification to the EBP algorithm in which the gradients are rescaled at every layer helped to improve the performance. Use of expected output of a neuron instead of actual output for correcting ights improved performance of the momentum strategy. The conjugate gradient method and self-determination of adaptive learning rate require no learning rate coefficient from the user. Use of energy functions other than the sum of the squared error has shown improved convergence rate. An effective learning rate coefficient selection needs to consider the size of the training set. All these methods to improve the performance of the EBP algorithm are presented here.
['Dilip Sarkar']
I am writing a research paper on a idea Methods to speed up error back-propagation learning algorithm
[{"from": "human", "value": "I am writing a research paper on a idea Methods to speed up error back-propagation learning algorithm"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Error back propagation (EBP) is now the most used training algorithm for feedforward artificial neural networks (FFANNs). Hover, it is generally believed that it is very slow if it does converge, especially if the network size is not too large compared to the problem at hand. The main problem with the EBP algorithm is that it has a constant learning rate coefficient, and different regions of the error surface may have different characteristic gradients that may require a dynamic change of learning rate coefficient based on the nature of the surface. Also, the characteristic of the error surface may be unique in every dimension, which may require one learning rate coefficient for each ight. To overcome these problems several modifications have been suggested. This survey is an attempt to present them together and to compare them. The first modification was momentum strategy where a fraction of the last ight correction is added to the currently suggested ight correction. It has both an accelerating and a decelerating effect where they are necessary. Hover, this method can give only a relatively small dynamic range for the learning rate coefficient. To increase the dynamic range of the learning rate coefficient, such methods as the bold driver and SAB (self-adaptive back propagation) re proposed. A modification to the SAB that eliminates the requirement of selection of a good learning rate coefficient by the user gave the SuperSAB. A slight modification to the momentum strategy produced a new method that controls the oscillation of ights to speed up learning. Modification to the EBP algorithm in which the gradients are rescaled at every layer helped to improve the performance. Use of expected output of a neuron instead of actual output for correcting ights improved performance of the momentum strategy. The conjugate gradient method and self-determination of adaptive learning rate require no learning rate coefficient from the user. Use of energy functions other than the sum of the squared error has shown improved convergence rate. An effective learning rate coefficient selection needs to consider the size of the training set. All these methods to improve the performance of the EBP algorithm are presented here."}]