Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  shows that if a large neural network is used for a pattern classification problem, and the learning algorithm finds a network with small ights that has small squared error on the training patterns, then the generalization performance depends on the size of the ights rather than the number of ights. More specifically, consider an l-layer feed-forward network of sigmoid units, in which the sum of the magnitudes of the ights associated with each unit is bounded by A. The misclassification probability converges to an error estimate (that is closely related to squared error on the training set) at rate O((cA)l(l+1)/2 √(log n)/m) ignoring log factors, where m is the number of training patterns, n is the input dimension, and c is a constant. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of ights. It also supports heuristics (such as ight decay and early stopping) that attempt to keep the ights small during training.
['Peter L. Bartlett']
I am writing a research paper on a idea For Valid Generalization the Size of the Weights is More Important than the Size of the Network
[{"from": "human", "value": "I am writing a research paper on a idea For Valid Generalization the Size of the Weights is More Important than the Size of the Network"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  shows that if a large neural network is used for a pattern classification problem, and the learning algorithm finds a network with small ights that has small squared error on the training patterns, then the generalization performance depends on the size of the ights rather than the number of ights. More specifically, consider an l-layer feed-forward network of sigmoid units, in which the sum of the magnitudes of the ights associated with each unit is bounded by A. The misclassification probability converges to an error estimate (that is closely related to squared error on the training set) at rate O((cA)l(l+1)/2 \u221a(log n)/m) ignoring log factors, where m is the number of training patterns, n is the input dimension, and c is a constant. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of ights. It also supports heuristics (such as ight decay and early stopping) that attempt to keep the ights small during training."}]