Finding UMVUE (Minimum-variance unbiased estimator) looks like an optimisation problem (minimise the variance of $\hat{\theta}$ given the constraint $\text{E}\_\theta\hat{\theta}=\theta.$


I tried to apply here a standard method from optimisation but failed.
The setting: $(X\_1,\dots, X\_n)$ are i.i.d. random variables on the space $(\Omega,F, \mu)$ and $\rho\_\theta \mu$ is the measure corresponding to the parameter $\theta.$
Let’s consider a functional
$$
L:= \int\_{\Omega}\hat{\theta}^2(X\_1,\dots,X\_n)\rho\_\theta\mu - \left(\int\_{\Omega}\hat{\theta}(X\_1,\dots,X\_n)\rho\_\theta\mu \right)^2 - 
\lambda \left( \int\_{\Omega}\hat{\theta}(X\_1,\dots,X\_n)\rho\_\theta\mu - \theta
\right )
$$
where $\lambda$ is a Lagrange multiplier corresponding to the constraint $\int\_{\Omega}\hat{\theta}(X\_1,\dots,X\_n)\rho\_\theta\mu = \theta.$ Notice that the second term is simple $\theta^2.$


Next we take a functional derivative with respect to $\hat{\theta}.$ The result (with simplified notation) is


$$
\frac{\delta L}{\delta \hat{\theta}} = 2 \int\_{\Omega}\hat{\theta} \delta \hat{\theta}\rho\_\theta\mu - 0 - \lambda \int\_{\Omega}\delta \hat{\theta}\rho\_\theta\mu = \int\_{\Omega}(2\hat{\theta}-\lambda) \delta \hat{\theta}\rho\_\theta\mu.
$$
So $\frac{\delta L}{\delta \hat{\theta}}=0$ is equivalent to $2\hat{\theta}=\lambda.$ That implies (as I understand) that $\hat{\theta}$ is a constant estimator. This answer doesn’t make any sense to me.


Is it possible to modify this computation somehow in order to get something meaningful?



UMVUE as an optimization problem
$\newcommand\th\theta$


1. It is not true in general that the Cramér--Rao lower bound is a solution to a meaningful optimisation problem: ["Under some regularity conditions, the Cramér-Rao lower bound is
attained iff $f\_\th$ is in an exponential family" (p.14)](https://pages.stat.wisc.edu/%7Eshao/stat709/stat709-17.pdf), where $(f\_\th)\_{\th\in I}$ is a parametric family defining the statistical model and $I$ is an open interval on the real line. In fact, an important condition is missing in this quote: if the Cramér--Rao lower bound is attained for an unbiased estimator $T(X\_1,\dots,X\_n)$ of $\th$, then, under certain regularity conditions, the joint density of the sample $(X\_1,\dots,X\_n)$ must be of the following specific form
$$f\_\th(x\_1,\dots,x\_n)
=\exp[w(\th)T(x\_1,\dots,x\_n)]
c(\th)h(x\_1,\dots,x\_n),$$
with the same function $T$ in the exponent. So, in most cases the Cramér--Rao lower bound will not be attained.
2. You cannot define an estimator $\hat\theta$ by the formula $\hat\theta(X\_1,\dots,X\_n):=\th$ -- because an estimator of $\th$ cannot depend on $\th$; it may depend only on the sample $X\_1,\dots,X\_n$.


