Suppose I have some random variable X with probability distribution P(.;theta). Suppose I have a single sample x from this distribution.


Does it make any sense to ask for the derivative of x with respect to theta?


I believe it does not, but am having trouble convincing some of my colleagues.



Derivatives through random variables?
It does not because $x$ is just some element of your state space. You could conceivably choose $x$ as your sample point for any or all of the $\theta$'s. So what it would make sense to differentiate with respect to $\theta$ is $$\mathbb{P}\_{\theta}(X=x)$$ as long as you have a discrete distribution. For a continuous distribution you could substitute $x$ with some interval.


