For two pmf $p=\lbrace p\_i\rbrace$ and $q=\lbrace q\_i\rbrace$ on the same finite alphabet, we know that relateive entropy
$D(p\|q)=\sum p\_i\log\frac{p\_i}{q\_i}$
and 1-norm
$\|p-q\|\_1=\sum |p\_i-q\_i|$
are both measures of their distance. But it is unfortunate that relative entropy is not a norm. My question is: even so, do we still have equivalence between these two measure of distance? To be specific, assume $\|p-q\|\_1\le C$ for some positive constant $C$, do we have $D(p\|q)\le MC$ for some positive $M$? If have, how to prove? Thanks a lot!



equivalence of 1-norm and relative entropy?
No, that is not true. Let $p^{(n)}\to q$ in $L^1$ such that $p^{(n)}$ lies in the (relative) interior of the probability simplex whereas $q$ is on the boundary (of the simplex), i.e., $q\_i=0$ for some $i$. Then $D(P^{(n)}\|q)=\infty$ for every $n$.


But the other direction is true because of the Pinsker's inequality $\|p-q\|\_1\le \sqrt{2D(p\|q)}$ (you may already know this fact!). 


