If we wish to encode a gaussian source, $X\sim\mathcal{N}(0,\sigma^2)$ at rate $R$, then decode it to create an estimate $\hat{X}$, rate-distortion theory tells us that the lowest mean-squared-error we can achieve through the encoder-decoder is $\sigma^2 2^{-2R}.$ That is, using an optimal encoder and decoder, we can achieve $E(X-\hat{X})^2=\sigma^2 2^{-2R}.$ (possibly asymptotically as we let code block lengths increase)


What might the distribution of $\hat{X}|\_{X=x}$ be, for such encoders and decoders? That is, how is our estimate distributed around the right answer? 


For illustration, we know obviously that no choice of enc/dec could yield a deterministic $\hat{X}|\_{X=x}=x+\sqrt{\sigma^22^{-2R}},$ even though this distribution would give us the correct mean-squared-error. (Otherwise we could design a lossless decoder!)




---


I believe that it should be possible to design some random coding scheme where, as the code block lengths get large, eventually each $i$th message in our block, $\hat{X\_i}|\_{X\_i=x\_i}$, becomes close in distribution to $\mathcal{N}(x\_i,\sigma^22^{-2R}).$


Equivalently, I am trying to find a sequence of encoders and decoders where:


$$D(p^n||q^n)\rightarrow 0, \text{ as }n \uparrow \infty$$


where $x^n$ denotes a block of $n$ messages, $\hat{X}^n$ is the decoded estimate of all these $n$ messages, $p^n$ is the distribution of ${\hat{X}^n|\_ {X^n=x^n}}$ and $q^n$ is the distribution of $\mathcal{N}(x^n,\sigma^22^{-2R}I\_{n\times n})$.


Finding such an encoder/decoder pair would essentially allow us to properly say that quantization of a gaussian source can be modeled as gaussian noise.



Rate-Distortion theory: What is the distribution of distortion on an optimal Gaussian encoder?
Yes, you can design a countable-alphabet quantizer for Gaussian RVs where quantization noise approaches Gaussian noise in relative entropy, as codeword length increases.


'On Lattice Quantization Noise' by Zamir and Feder in Transactions on Information Theory Vol. 42 No. 4, July 1996: 


<http://www.eng.tau.ac.il/~zamir/papers/lqn.pdf>


