{"word_count": 757, "character_count": 6508}
This week, in the MAT8595 course, we will start the section on inference for extreme values. To start with something simple, we will use maximum likelihood techniques on a Generalized Pareto Distribution (we've seen Monday Pickands-Balkema-de Hann theorem).Maximum Likelihood EstimationIn the context of parametric models, the standard technique is to consider the maximum of the likelihood (or the log-likelihod).Let  denote the parameter (with ). Given some - stnardard - technical assumptions, such as , or  on some neighbourhood of , thenwhere  denotes Fisher information matrix (see any textbook for mathematical statistics courses). Consider here some i.i.d. sample, from a Generalized Pareto Distribution, with parameter , so thatIf we solve (numerically) the first order condition of the maximum likelihood, we get an estimator   which satisfiesThe idea of this asymptotic normality is the following : if the true distribution of the sample is a GPD with parameter , then, if  is large enough, then  will have a joint normal distribution. So if we generate a lot of sample (sufficently large, say 200 observations), then the scatterplot of the estimator should the same as the scatterplot of a Gaussian distribution,&#62; library(evir)&#62; n=200&#62; param=matrix(NA,1000,2)&#62; for(s in 1:1000){+ x=rgpd(n,xi=1/1.5,beta=1)+ param[s,]=gpd(x,0)$par.ests+ }&#62; m=apply(param,2,mean)&#62; S=var(param)&#62; library(mnormt)&#62; x=seq(min(param[,1])-.05,max(param[,1])+.05,length=101)&#62; y=seq(min(param[,2])-.05,max(param[,2])+.05,length=101)&#62; vx=rep(x,each=length(y))&#62; vy=rep(y,length(x))&#62; vz=dmnorm(cbind(vx,vy),m,S)&#62; z=matrix(vz,length(y),length(x))&#62; COL=rev(heat.colors(100))&#62; image(x,y,z,col=COL)&#62; points(param)and to get a 3d representation&#62; x=seq(min(param[,1])-.05,max(param[,1])+.05,length=31)&#62; y=seq(min(param[,2])-.05,max(param[,2])+.05,length=31)&#62; vx=rep(x,each=length(y))&#62; vy=rep(y,length(x))&#62; vz=dmnorm(cbind(vx,vy),m,S)&#62; z=matrix(vz,length(y),length(x))&#62; persp(x,y,t(z),shade=TRUE,col="green",theta=-30,phi=20,ticktype="detailed",+ xlab="xi",ylab="sigma")With 200 observations, if the true underlying distribution is a GPD, then, indeed, the joint distribution of  seems to be normal. That would be interesting to generate some confidence intervals for instance, or define some tests.To go further, see any standard textbook on statistical mathematics, e.g. Casella &#38; Berger (2002).Delta MethodAnother important property is the so called delta-method (we've seen Monday in class that it was obtained easily using a first order Taylor expansion). The idea is that if  is asymptotically normal, and if  is sufficently smooth, then  will also be asymptotically Gaussian. More precicely (see also the header of this blog)From this property, we can get the normality of  (which is another parametrization used in extreme value models), or on any quantile, . Let us run some simulation, one more time to check that we actually have a joint normality.&#62; library(evir)&#62; n=200&#62; param=riskm=matrix(NA,1000,2)&#62; for(s in 1:1000){+ x=rgpd(n,xi=1/1.5,beta=1)+ param[s,]=gpd(x,0)$par.ests+ xihat=param[s,1]+ shat=param[s,2]+ q=shat * (.01^(-xihat) - 1)/xihat+ tvar=q+(shat + xihat * q)/(1 - xihat)+ riskm[s,]=c(1/xihat,q)+ }&#62; m=apply(riskm,2,mean)&#62; S=var(riskm)&#62; library(mnormt)&#62; x=seq(min(riskm[,1])-.05,max(riskm[,1])+.05,length=101)&#62; y=seq(min(riskm[,2])-.05,max(riskm[,2])+.05,length=101)&#62; vx=rep(x,each=length(y))&#62; vy=rep(y,length(x))&#62; vz=dmnorm(cbind(vx,vy),m,S)&#62; z=matrix(vz,length(y),length(x))&#62; image(x,y,t(z),col=COL)&#62; points(riskm)As we can see bellow, with samples of size 200, we cannot use this asymptotical result: it looks like we do not have enought data. But if we run the same code with&#62; n=5000We get the joint normality of  and . This is what we can get from this result, called delta-method in statistical textbooks. See again Casella &#38; Berger (2002) for more details.Profile LikelihoodAnother interesting tool is the concept of profile likelihood. This would be interesting here since the main interest is the tail index ,  being here some kind of auxilary parameter. See Venzon &#38; Moolgavkar (1988) for more details. Here, we will plotBut more generally, it is possible to considerwhere  is the set of interesting parameters. Then (under standard suitable conditions) we can prove thatwhich can be used to derive confidence intervals. In the GPD case, for each , we have to find an optimal . We compute the (profile) likelihood i.e. . And we can compute the maximum of this profile likelihood. This two-stage optimization is, in general, not equivalent with the (global) maximization of the likelihood, as computed below&#62;  n=500&#62;  set.seed(1)&#62;  x=rgpd(n,xi=1/1.5,beta=1)&#62;  loglikelihood=function(xi,beta){+  sum(log(dgpd(x,xi,mu=0,beta))) }&#62;  XIV=(1:300)/100;L=rep(NA,300)&#62;  for(i in 1:300){+  XI=XIV[i]+  profilelikelihood=function(beta){+  -loglikelihood(XI,beta) }+  L[i]=-optim(par=1,fn=profilelikelihood)$value }&#62;  plot(XIV,L,type="l")&#62;  XIV[which.max(L)][1] 0.67&#62;  gpd(x,0)$par.ests       xi      beta 0.6730145 0.9725483We are not far away. Actually, if we want to compute the maximum of the profile likelihood (and not only compute the values of the profile likelihood on a grid, as before), we use&#62;  PL=function(XI){+  profilelikelihood=function(beta){+  -loglikelihood(XI,beta) }+  return(optim(par=1,fn=profilelikelihood)$value)}&#62;  (OPT=optimize(f=PL,interval=c(0,3)))$minimum[1] 0.6731025$objective[1] 822.5574Observe that, indeed, we are not far away from the maximum likelihood estimator of  (I believe that it's mainly a computational issue here, and theat the two are similar, here... actually, I'd be glad to hear about cases where maximum of the profile likelihood is not the same as the maximum of the likelihood). The interesting point is that we can use this technique to compute a confidence interval, and even visualize it on a graph&#62;  up=OPT$objective&#62;  abline(h=-up)&#62;  abline(h=-up-qchisq(p=.95,df=1),col="red")&#62;  I=which(L&#62;=-up-qchisq(p=.95,df=1))&#62;  lines(XIV[I],rep(-up-qchisq(p=.95,df=1),length(I)),+  lwd=5,col="red")&#62;  abline(v=range(XIV[I]),lty=2,col="red")The vertical lines are the lower and the upper bound of a 95% confidence interval for parameter .To go further, see Murphy, S.A &#38; van der Vaart, A.W. (2000). On Profile Likelihood.