Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it to you precisely.


Let a discrete source X and a capacitance channel C, if we define the entropy of the source (in other words the density of the information, its "richness") by H(X), so if we have H(X)>C, Shannon's conditions are not verified and it's not certain that the noisy channel can capture information without error. So in this case, this curious source starts to weight, to give a notion of importance, all the information it emits, so that information of too low weight is "lost" and thus to decrease its entropy H(X) until it returns to the conditions of Shannon and to obtain H(X) < C and thus to transmit the information.


Here is a detailed description of this source:


1. Source S transmits the information (A1, A2, A3, A4, A5, ..., An) through a noisy channel of capacitance C.
2. The source S is variable, that is to say that it transmits (A1, A2, A3, A4, A5, ... Aj) with j which varies randomly from 1 to n.
3. There exists a p such that the entropy H of S is < C when it transmits (A1, A2, A3, A4, A5, ..., Aj) with j < or=p but > C with j>p.
4. Since the information A1 and An are less important, they no longer transmit it when j>p so that its entropy H is again lower than C and the information passes. (If you have to fire n-p informations to reduce the entropy H below C, we can say that there are n-p informations less important than the source S can potentially transfer).


I tried to represent this system by associating to each Ai a probability pi etc., I tried to tweak the formula of Shannon's entropy to express that I I'm moving out the information A1 and An but in vain, I didn't succeed in writing the "variable" aspect of the source S, I also failed to write mathematically the notion of importance for A1 and An... in short I struggle completely and I bug on almost all the points of my problem to write it correctly. 


I need to write down the properties of this source properly to do my calculations, if ever someone can help me, I really thank you. If Shannon's probabilities are not enough to write this source, and we need Kolmogorov's formalism, I am open :)


PS : sorry for my english I am french


Best regards, 


Lulu2612



Shannon problem
The concept of weighted entropy with weight function $\varphi$ defined as


$$
H\_\varphi = -\sum\_i \phi(A\_i) p(A\_i) \log p(A\_i)
$$


is not so new. However, this recent reference seems to give a good discussion.


<https://arxiv.org/abs/1710.10798>


If I understand your problem correctly you need to find weights that give you a weighted entropy below the channel capacity and correspond to your evaluation of the relative importance of the different symbols. This is an optimisation problem.


