{"word_count": 258, "character_count": 2187}
The maximum likelihood estimator is invariant in the sense that for all bijective function  , if   is the maximum likelihood estimator of   then  . Let  , then   is equal to  , and the likelihood function in   is  . And since   is the maximum likelihood estimator of  ,hence,   is the maximum likelihood estimator of  .For instance, the Bernoulli distribution is   with   and Given sample  , the likelihood is The log-likelihood is then with ICI Thus, the first order condition is satisfied when  . In order to illustrate, consider the following data&#62; set.seed(1)&#62; X=sample(0:1,size=15,replace=TRUE)&#62; X[1] 0 0 1 1 0 1 1 1 1 0 0 0 1 0 1The (negative) log-likelihood is here&#62; loglik=function(p){+ -sum(log(dbinom(X,size=1,prob=p)))+ }that we can visualize below&#62; u=seq(0,1,by=.025)&#62; v=-Vectorize(loglik)(u)&#62; plot(u,v,type="l",xlab="",ylab="")From calculations above, we know that the maximum likelihood estimator for  is&#62; mean(X)[1] 0.5333333The numerical version is&#62; (opt=optim(.5,loglik))$par[1] 0.5333008$value[1] 10.36385$countsfunction gradient20 NA$convergence[1] 0$messageNULLSomehow, we were lucky here, because we did not say that the optimization was on the interval . Nevertheless, our estimator for the probability belongs to . In order to insure that the optimal value is in , we can consider some constrained optimization routine&#62; constrOptim(.5, loglik, grad=NULL,ui=matrix(c(1,-1),2,1), ci=c(0,-1))$par[1] 0.5333008$value[1] 10.36385$countsfunction gradient20 NA$convergence[1] 0$messageNULL$outer.iterations[1] 2$barrier.value[1] 6.909277e-05On the previous graph, we did - indeed - reach that maximum of the log-likelihood&#62; abline(v=opt$par,col="red")An alternative is to consider   (as in the exponential family). The log-likelihood is then since Here Thus, the first order condition is satisfied wheni.e. From a numerical perspective, we have the same optimal value&#62; loglik=function(theta){+ -sum(log(dbinom(X,size=1,prob=exp(theta)/(1+exp(theta)))))+ }&#62; (opt=optim(0,loglik))$par[1] 0.1335938$value[1] 10.36385$countsfunction gradient20 NA$convergence[1] 0$messageNULL&#62; exp(opt$par)/(1+exp(opt$par))[1] 0.5333489