Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  deals with learnability of concept classes defined by neural networks, showing the hardness of PAC-learning (in the complexity, not merely information-theoretic sense) for networks with a particular class of activation. The obstruction lies not with the VC dimension, which is known to grow slowly; instead, the result follows the fact that the  loading  problem is NP-complete. (The complexity scales badly with input dimension; the loading problem is polynomial-time if the input dimension is constant.) Similar and ll-known theorems had already been proved by Megiddo and by Blum and Rivest, for  binary-threshold  networks. It turns out the general problem for continuous sigmoidal-type functions, as used in practical applications involving steepest descent, is  not  NP-hard—there are “sigmoidals” for which the problem is in fact trivial—so it is an open question to determine what properties of the activation function cause difficulties. Ours is the first result on the hardness of loading networks which do not consist of binary neurons;  employ a piecewise-linear activation function that has been used in the neural network literature. Our theoretical results lend further justification to the use of incremental (architecture-changing) techniques for training networks.
['Bhaskar DasGupta', 'Hava T. Siegelmann', 'Eduardo D. Sontag']
I am writing a research paper on a idea On a learnability question associated to neural networks with continuous activations (extended abstract)
[{"from": "human", "value": "I am writing a research paper on a idea On a learnability question associated to neural networks with continuous activations (extended abstract)"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely:  deals with learnability of concept classes defined by neural networks, showing the hardness of PAC-learning (in the complexity, not merely information-theoretic sense) for networks with a particular class of activation. The obstruction lies not with the VC dimension, which is known to grow slowly; instead, the result follows the fact that the  loading  problem is NP-complete. (The complexity scales badly with input dimension; the loading problem is polynomial-time if the input dimension is constant.) Similar and ll-known theorems had already been proved by Megiddo and by Blum and Rivest, for  binary-threshold  networks. It turns out the general problem for continuous sigmoidal-type functions, as used in practical applications involving steepest descent, is  not  NP-hard\u2014there are \u201csigmoidals\u201d for which the problem is in fact trivial\u2014so it is an open question to determine what properties of the activation function cause difficulties. Ours is the first result on the hardness of loading networks which do not consist of binary neurons;  employ a piecewise-linear activation function that has been used in the neural network literature. Our theoretical results lend further justification to the use of incremental (architecture-changing) techniques for training networks."}]