{"word_count": 1146, "character_count": 8316}
This week, we started to discuss overdispersion when modeling claims frequency. In my previous post, I discussed computations of empirical variances with different exposure. But I did use only one factor to compute classes. Of course, it is possible to use much more factors. For instance, using cartesian products of factors,&#62; X=as.factor(paste(sinistres$carburant,sinistres$zone,+ cut(sinistres$ageconducteur,breaks=c(17,24,40,65,101))))&#62; E=sinistres$exposition&#62; Y=sinistres$nbre&#62; vm=vv=ve=rep(NA,length(levels(X)))&#62;   for(i in 1:length(levels(X))){+     ve[i]=Ei=E[X==levels(X)[i]]+     Yi=Y[X==levels(X)[i]]+   vm[i]=meani=weighted.mean(Yi/Ei,Ei)    # moyenne +   vv[i]=variancei=sum((Yi-meani*Ei)^2)/sum(Ei)    # variance+  cat("Class ",levels(X)[i],"average =",meani," variance =",variancei,"\n")+ }Class D A (17,24]  average = 0.06274415  variance = 0.06174966 Class D A (24,40]  average = 0.07271905  variance = 0.07675049 Class D A (40,65]  average = 0.05432262  variance = 0.06556844 Class D A (65,101] average = 0.03026999  variance = 0.02960885 Class D B (17,24]  average = 0.2383109   variance = 0.2442396 Class D B (24,40]  average = 0.06662015  variance = 0.07121064 Class D B (40,65]  average = 0.05551854  variance = 0.05543831 Class D B (65,101] average = 0.0556386   variance = 0.0540786 Class D C (17,24]  average = 0.1524552   variance = 0.1592623 Class D C (24,40]  average = 0.0795852   variance = 0.09091435 Class D C (40,65]  average = 0.07554481  variance = 0.08263404 Class D C (65,101] average = 0.06936605  variance = 0.06684982 Class D D (17,24]  average = 0.1584052   variance = 0.1552583 Class D D (24,40]  average = 0.1079038   variance = 0.121747 Class D D (40,65]  average = 0.06989518  variance = 0.07780811 Class D D (65,101] average = 0.0470501   variance = 0.04575461 Class D E (17,24]  average = 0.2007164   variance = 0.2647663 Class D E (24,40]  average = 0.1121569   variance = 0.1172205 Class D E (40,65]  average = 0.106563    variance = 0.1068348 Class D E (65,101] average = 0.1572701   variance = 0.2126338 Class D F (17,24]  average = 0.2314815   variance = 0.1616788 Class D F (24,40]  average = 0.1690485   variance = 0.1443094 Class D F (40,65]  average = 0.08496827  variance = 0.07914423 Class D F (65,101] average = 0.1547769   variance = 0.1442915 Class E A (17,24]  average = 0.1275345   variance = 0.1171678 Class E A (24,40]  average = 0.04523504  variance = 0.04741449 Class E A (40,65]  average = 0.05402834  variance = 0.05427582 Class E A (65,101] average = 0.04176129  variance = 0.04539265 Class E B (17,24]  average = 0.1114712   variance = 0.1059153 Class E B (24,40]  average = 0.04211314  variance = 0.04068724 Class E B (40,65]  average = 0.04987117  variance = 0.05096601 Class E B (65,101] average = 0.03123003  variance = 0.03041192 Class E C (17,24]  average = 0.1256302   variance = 0.1310862 Class E C (24,40]  average = 0.05118006  variance = 0.05122782 Class E C (40,65]  average = 0.05394576  variance = 0.05594004 Class E C (65,101] average = 0.04570239  variance = 0.04422991 Class E D (17,24]  average = 0.1777142   variance = 0.1917696 Class E D (24,40]  average = 0.06293331  variance = 0.06738658 Class E D (40,65]  average = 0.08532688  variance = 0.2378571 Class E D (65,101] average = 0.05442916  variance = 0.05724951 Class E E (17,24]  average = 0.1826558   variance = 0.2085505 Class E E (24,40]  average = 0.07804062  variance = 0.09637156 Class E E (40,65]  average = 0.08191469  variance = 0.08791804 Class E E (65,101] average = 0.1017367   variance = 0.1141004 Class E F (17,24]  average = 0           variance = 0 Class E F (24,40]  average = 0.07731177  variance = 0.07415932 Class E F (40,65]  average = 0.1081142   variance = 0.1074324 Class E F (65,101] average = 0.09071118  variance = 0.1170159Again, one can plot the variance against the average,&#62; plot(vm,vv,cex=sqrt(ve),col="grey",pch=19,+ xlab="Empirical average",ylab="Empirical variance")&#62; points(vm,vv,cex=sqrt(ve))&#62; abline(a=0,b=1,lty=2)An alternative is to use a tree. The tree can be obtained from another variable (the insured had, or had not, a claim, during the period considered) but it should be rather close to the one we would like to model (the number of claims over the period considered). Here, I did use the whole database (with more that 600,000 lines)&#62; library(tree)&#62; T=tree((nombre&#62;0)~as.factor(zone)+as.factor(puissance)++ as.factor(marque)+as.factor(carburant)+as.factor(region)++ agevehicule+ageconducteur,data=baseFREQ,+ split =  "gini",minsize =25000)The tree is the following&#62; plot(T)&#62; text(T)Now, each knot defines a class, and it is possible to use it to define a class. Which is supposed to be homogeneous.&#62; X=as.factor(T$where)&#62; E=sinistres$exposition&#62; Y=sinistres$nbre&#62; vm=vv=ve=rep(NA,length(levels(X)))&#62;   for(i in 1:length(levels(X))){+     ve[i]=Ei=E[X==levels(X)[i]]+     Yi=Y[X==levels(X)[i]]+   vm[i]=meani=weighted.mean(Yi/Ei,Ei)    # moyenne +   vv[i]=variancei=sum((Yi-meani*Ei)^2)/sum(Ei)    # variance+  cat("Class ",levels(X)[i],"average =",meani," variance =",variancei,"\n")+  }Class  6 average =   0.04010406  variance = 0.04424163 Class  8 average =   0.05191127  variance = 0.05948133 Class  9 average =   0.07442635  variance = 0.08694552 Class  10 average =  0.4143646   variance = 0.4494002 Class  11 average =  0.1917445   variance = 0.1744355 Class  15 average =  0.04754595  variance = 0.05389675 Class  20 average =  0.08129577  variance = 0.0906322 Class  22 average =  0.05813419  variance = 0.07089811 Class  23 average =  0.06123807  variance = 0.07010473 Class  24 average =  0.06707301  variance = 0.07270995 Class  25 average =  0.3164557   variance = 0.2026906 Class  26 average =  0.08705041  variance = 0.108456 Class  27 average =  0.06705214  variance = 0.07174673 Class  30 average =  0.05292652  variance = 0.06127301 Class  31 average =  0.07195285  variance = 0.08620593 Class  32 average =  0.08133722  variance = 0.08960552 Class  34 average =  0.1831559   variance = 0.2010849 Class  39 average =  0.06173885  variance = 0.06573939 Class  41 average =  0.07089419  variance = 0.07102932 Class  44 average =  0.09426152  variance = 0.1032255 Class  47 average =  0.03641669  variance = 0.03869702 Class  49 average =  0.0506601   variance = 0.05089276 Class  50 average =  0.06373107  variance = 0.06536792 Class  51 average =  0.06762947  variance = 0.06926191 Class  56 average =  0.06771764  variance = 0.07122379 Class  57 average =  0.04949142  variance = 0.05086885 Class  58 average =  0.2459016   variance = 0.2451116 Class  59 average =  0.05996851  variance = 0.0615773 Class  61 average =  0.07458053  variance = 0.0818608 Class  63 average =  0.06203737  variance = 0.06249892 Class  64 average =  0.07321618  variance = 0.07603106 Class  66 average =  0.07332127  variance = 0.07262425 Class  68 average =  0.07478147  variance = 0.07884597 Class  70 average =  0.06566728  variance = 0.06749411 Class  71 average =  0.09159605  variance = 0.09434413 Class  75 average =  0.03228927  variance = 0.03403198 Class  76 average =  0.04630848  variance = 0.04861813 Class  78 average =  0.05342351  variance = 0.05626653 Class  79 average =  0.05778622  variance = 0.05987139 Class  80 average =  0.0374993   variance = 0.0385351 Class  83 average =  0.06721729  variance = 0.07295168 Class  86 average =  0.09888492  variance = 0.1131409 Class  87 average =  0.1019186   variance = 0.2051122 Class  88 average =  0.05281703  variance = 0.0635244 Class  91 average =  0.08332136  variance = 0.09067632 Class  96 average =  0.07682093  variance = 0.08144446 Class  97 average =  0.0792268   variance = 0.08092019 Class  99 average =  0.1019089   variance = 0.1072126 Class  100 average = 0.1018262   variance = 0.1081117 Class  101 average = 0.1106647   variance = 0.1151819 Class  103 average = 0.08147644  variance = 0.08411685 Class  104 average = 0.06456508  variance = 0.06801061 Class  107 average = 0.1197225   variance = 0.1250056 Class  108 average = 0.0924619   variance = 0.09845582 Class  109 average = 0.1198932   variance = 0.1209162Here, when ploting the empirical variance (per knot) against the empirial average of claims, we getHere, we can identify classes where remaining heterogeneity.