Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The past year saw the introduction of new architectures such as Highway networks and Residual networks which, for the first time, enabled the training of feedforward networks with dozens to hundreds of layers using simple gradient descent. While depth of representation has been posited as a primary reason for their success, there are indications that these architectures defy a popular view of deep learning as a hierarchical computation of increasingly abstract features at each layer. #R##N#In this report,  argue that this view is incomplete and does not adequately explain several recent findings.  propose an alternative viewpoint based on unrolled iterative estimation---a group of successive layers iteratively refine their estimates of the same features instead of computing an entirely new representation.  demonstrate that this viewpoint directly leads to the construction of Highway and Residual networks. Finally  provide preliminary experiments to discuss the similarities and differences beten the two architectures.
['Klaus Greff', 'Rupesh Kumar Srivastava', 'Jürgen Schmidhuber']
I am writing a research paper on a idea Highway and Residual Networks learn Unrolled Iterative Estimation
[{"from": "human", "value": "I am writing a research paper on a idea Highway and Residual Networks learn Unrolled Iterative Estimation"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The past year saw the introduction of new architectures such as Highway networks and Residual networks which, for the first time, enabled the training of feedforward networks with dozens to hundreds of layers using simple gradient descent. While depth of representation has been posited as a primary reason for their success, there are indications that these architectures defy a popular view of deep learning as a hierarchical computation of increasingly abstract features at each layer. #R##N#In this report,  argue that this view is incomplete and does not adequately explain several recent findings.  propose an alternative viewpoint based on unrolled iterative estimation---a group of successive layers iteratively refine their estimates of the same features instead of computing an entirely new representation.  demonstrate that this viewpoint directly leads to the construction of Highway and Residual networks. Finally  provide preliminary experiments to discuss the similarities and differences beten the two architectures."}]