Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Although today most language models treat language purely as word sequences, there is recurring interest in tapping new sources of information, such as disfluencies, prosody, the interlocutor’s dialog act, and the interlocutor’s recent words. In order to estimate the potential value of such sources of information,  extend Shannon’s guessing-game method for estimating entropy to work for spoken dialog. Four teams of two subjects each predicted the next word in a dialog using various amounts of context: one word, two words, all the words spoken so far, or the full dialog audio so far. The entropy benefit in the full-audio condition over the full text condition was substantial, .64 bits per word, greater than the .54 bit benefit of full text context over trigrams. This suggests that language models may be improved by use of the prosody of the speaker and context from the interlocutor. Index Terms: entropy, perplexity, Shannon’s guessing game, prediction, context, prosody
['Nigel Ward', 'Benjamin H. Walker']
I am writing a research paper on a idea Estimating the Potential of Signal and Interlocutor-Track Information for Language Modeling
[{"from": "human", "value": "I am writing a research paper on a idea Estimating the Potential of Signal and Interlocutor-Track Information for Language Modeling"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Although today most language models treat language purely as word sequences, there is recurring interest in tapping new sources of information, such as disfluencies, prosody, the interlocutor\u2019s dialog act, and the interlocutor\u2019s recent words. In order to estimate the potential value of such sources of information,  extend Shannon\u2019s guessing-game method for estimating entropy to work for spoken dialog. Four teams of two subjects each predicted the next word in a dialog using various amounts of context: one word, two words, all the words spoken so far, or the full dialog audio so far. The entropy benefit in the full-audio condition over the full text condition was substantial, .64 bits per word, greater than the .54 bit benefit of full text context over trigrams. This suggests that language models may be improved by use of the prosody of the speaker and context from the interlocutor. Index Terms: entropy, perplexity, Shannon\u2019s guessing game, prediction, context, prosody"}]