Consider a random variable $F$ with a distribution parameterized by $\theta$ and another random variable $G$ with a distribution parameterized by a variate of $F$, denoted $f$. Note that $F$ is dependent on $\theta$, but $G$ is dependent on $\theta$ only through $f$.


Consider the likelihood for $\theta$ given observations for $F$ and $G$. $g$ contains no more information about $\theta$ than $f$ (as $G$ is parameterized solely by $f$, not $\theta$), so one would expect


$Likelihood(\theta|f,g) = Likelihood(\theta|f)$.


However, given the usual definition of likelihood, $L(\phi|x)=Pr[X=x|\phi]$, the equality does not hold:


$L(\theta|f,g) = Pr[F=f,G=g|\theta]$,


$L(\theta|f) = Pr[F=f|\theta]$,


$Pr[F=f,G=g|\theta] \neq Pr[F=f|\theta]$.


Is there a notion of likelihood that captures the intuition that the likelihood of a parameter value should not change upon the addition of observations that do not hold additional information about the parameter? I'm not sure what terms to search for or what books and literature to peruse.



Is there a notion of likelihood that incorporates information content?
What you are missing is normalization: your likelihood functions $L(\theta|f,g)$ and $L(\theta|f)$ shouldn't be comparable, because adding up the likelihood for all the possible $\theta$s gives different totals. If you instead define  

$\displaystyle L(\phi|x) = \frac{Pr[X=x|\phi]}{\sum\_{\phi'} P[X=x|\phi']}$,  

then you obtain equalities: 


$\displaystyle L(\theta|f,g) = \frac{Pr[F=f, G=g|\theta]}{\sum\_{\theta'} P[F=f, G=g|\theta']}
= \frac{Pr[G=g|F=f]Pr[F=f|\theta]}{\sum\_{\theta'} P[G=g|F=f]P[F=f|\theta']}$ 


$\displaystyle = \frac{Pr[F=f|\theta]}{\sum\_{\theta'} P[F=f|\theta']} = L(\theta|f) $ 


To learn more, try searching for [Bayesian Networks](http://en.wikipedia.org/wiki/Bayesian_network), especially "parameter learning."


