Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: While Deep Neural Networks (DNNs) have achieved tremendous success for LVCSR tasks, training these networks is slow. To date, the most common approach to train DNNs is via stochastic gradient descent (SGD), serially on a single GPU machine. Serial training, coupled with the large number of training parameters and speech data set sizes, makes DNN training very slow for LVCSR tasks. While 2nd order, data-parallel methods have also been explored, these methods are not always faster on CPU clusters due to the large communication cost beten processors. In this work,  explore using a specialized hardware/software approach, utilizing a Blue Gene/Q (BG/Q) system, which has thousands of processors and excellent interprocessor communication.  explore using the 2nd order Hessian-free (HF) algorithm for DNN training with BG/Q, for both cross-entropy and sequence training of DNNs. Results on three LVCSR tasks indicate that using HF with BG/Q offers up to an 11x speedup, as ll as an improved word error rate (WER), compared to SGD on a GPU.
['Tara N. Sainath', 'I-Hsin Chung', 'Bhuvana Ramabhadran', 'Michael Picheny', 'John A. Gunnels', 'Brian Kingsbury', 'George Saon', 'Vernon Austel', 'Upendra V. Chaudhari']
I am writing a research paper on a idea Parallel Deep Neural Network Training for LVCSR Tasks using Blue Gene/Q
[{"from": "human", "value": "I am writing a research paper on a idea Parallel Deep Neural Network Training for LVCSR Tasks using Blue Gene/Q"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: While Deep Neural Networks (DNNs) have achieved tremendous success for LVCSR tasks, training these networks is slow. To date, the most common approach to train DNNs is via stochastic gradient descent (SGD), serially on a single GPU machine. Serial training, coupled with the large number of training parameters and speech data set sizes, makes DNN training very slow for LVCSR tasks. While 2nd order, data-parallel methods have also been explored, these methods are not always faster on CPU clusters due to the large communication cost beten processors. In this work,  explore using a specialized hardware/software approach, utilizing a Blue Gene/Q (BG/Q) system, which has thousands of processors and excellent interprocessor communication.  explore using the 2nd order Hessian-free (HF) algorithm for DNN training with BG/Q, for both cross-entropy and sequence training of DNNs. Results on three LVCSR tasks indicate that using HF with BG/Q offers up to an 11x speedup, as ll as an improved word error rate (WER), compared to SGD on a GPU."}]