Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The Shannon information measures are ll known to be continuous functions of the probability distribution for a given finite alphabet. , hover,  show that these measures are discontinuous with respect to almost all commonly used "distance" measures when the alphabet is countably infinite. Such "distance" measures include the Kullback-Leibler divergence and the variational distance. Specifically,  show that all the Shannon information measures are in fact discontinuous at all probability distributions. The proofs are based on a probability distribution which can be realized by a discrete-time Markov chain with countably infinite number of states. Our findings reveal that the limiting probability distribution may not fully characterize the asymptotic behavior of a Markov chain. These results explain why certain existing information-theoretical tools are restricted to finite alphabets, and provide hints on how these tools can be extended to countably infinite alphabet.
['Siu-Wai Ho', 'Raymond W. Yeung']
I am writing a research paper on a idea On the Discontinuity of the Shannon Information Measures
[{"from": "human", "value": "I am writing a research paper on a idea On the Discontinuity of the Shannon Information Measures"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The Shannon information measures are ll known to be continuous functions of the probability distribution for a given finite alphabet. , hover,  show that these measures are discontinuous with respect to almost all commonly used \"distance\" measures when the alphabet is countably infinite. Such \"distance\" measures include the Kullback-Leibler divergence and the variational distance. Specifically,  show that all the Shannon information measures are in fact discontinuous at all probability distributions. The proofs are based on a probability distribution which can be realized by a discrete-time Markov chain with countably infinite number of states. Our findings reveal that the limiting probability distribution may not fully characterize the asymptotic behavior of a Markov chain. These results explain why certain existing information-theoretical tools are restricted to finite alphabets, and provide hints on how these tools can be extended to countably infinite alphabet."}]