{"word_count": 358, "character_count": 3994}
In the introduction of Computational Actuarial Science with R, there was a short paragraph on how could we import only some parts of a large database, by selecting specific variables. The trick was to use the following&#62; read.table.select.columns=function(datatablename,I,sep=";"){+ datanc=read.table(datatablename,header=TRUE,sep=sep,skip=0,nrows=1)+ mycols=rep("NULL",ncol(datanc))+ names(mycols)=names(datanc)+ mycols[I]=NA+ datat=read.table(datatablename,header=TRUE,sep=sep,colClasses=mycols)+ return(datat)}For instance, if we use the same dataset as in the introduction, we can import only two variables of interest,&#62; loc="http://myweb.fsu.edu/jelsner/extspace/extremedatasince1899.csv"&#62; dt1=read.table.select.columns(loc,c("Region","Wmax"),sep=",")&#62; head(dt1,10)    Region      Wmax1    Basin 105.563422    Basin  40.000003    Basin  35.418224    Basin  51.067435  Florida  87.343286    Basin  96.641387     Gulf  35.418228       US  35.418229       US  87.3432810      US 106.35318&#62; dim(dt1)[1] 2100    2In other cases, it might be interesting to select some raws, or to avoid some of them (e.g. because of some typos in the original dataset). If we want to drop some specific raws, we can use&#62; read.table.drop.rows=function(datatablename,I,sep=";"){+ I=sort(I)+ if(min(I)&#62;1) minI=1+ if(min(I)==1) minI=NULL+ index1=c(minI,I[c(which(diff(I)&#62;1),length(I))]+1)+ index2=c(I[c(minI,which(diff(I)&#62;1)+1)],max(index1)-1)+ datat=read.table(datatablename,header=TRUE,sep,skip=0,nrows=1)+ datat=datat[-1,]+ for(i in 1:length(index1)){+ datat0=read.table(datatablename,header=FALSE,sep=sep,skip=index1[i],nrows=index2[i]-index1[i])+ names(datat0)=names(datat)+ datat=rbind(datat,datat0)+ }+ return(datat)}On the same dataset, assume that we have some troubles reading some lines, or we know that values are not valid,&#62; dt2=read.table.drop.rows(loc,c(3,6:8),sep=",")&#62; head(dt2[,1:5],10)     Yr  Region      Wmax      sst sun1  1899   Basin 105.56342 0.046596 8.42  1899   Basin  40.00000 0.046596 8.43  1899   Basin  51.06743 0.046596 8.44  1899 Florida  87.34328 0.046596 8.45  1899      US  87.34328 0.046596 8.46  1899      US 106.35318 0.046596 8.47  1899      US  51.06743 0.046596 8.48  1899      US  90.19791 0.046596 8.49  1899   Basin  56.48593 0.046596 8.410 1899   Basin 131.26902 0.046596 8.4&#62; dim(dt2)[1] 2096   11Now, we can try to do both at the same time (more or less) : for some specific variables, we want to import a subpart of the database, but we also want to avoid some specific lines,&#62; read.table.select.columns.rows=function(datatablename,Ic,Ir,sep=";"){+ datanc=read.table(datatablename,header=TRUE,sep=sep,skip=0,nrows=1)+ mycols=rep("NULL",ncol(datanc))+ names(mycols)=names(datanc)+ mycols[Ic]=NA+ I=sort(Ir)+ if(min(I)&#62;1) minI=1+ if(min(I)==1) minI=NULL+ index1=c(minI,I[c(which(diff(I)&#62;1),length(I))]+1)+ index2=c(I[c(minI,which(diff(I)&#62;1)+1)],max(index1)-1)+ datat=read.table(datatablename,header=TRUE,sep=sep,skip=0,nrows=1,colClasses=mycols)+ datat=datat[-1,]+ for(i in 1:length(index1)){+ datat0=read.table(datatablename,header=FALSE,sep=sep,skip=index1[i],nrows=index2[i]-index1[i],colClasses=mycols)+ names(datat0)=names(datat)+ datat=rbind(datat,datat0)+ }+ return(datat)}&#62; dt3=read.table.select.columns.rows(loc,c("Wmax","Region"),c(3,6:8),sep=",")&#62; head(dt3)   Region      Wmax1   Basin 105.563422   Basin  40.000003   Basin  51.067434 Florida  87.343285      US  87.343286      US 106.35318&#62; dim(dt3)[1] 2096    2One can observe that it is the same, here, as&#62; df=read.table(loc,header=TRUE,sep=",")&#62; sdf=df[-c(3,6:8),c("Wmax","Region")]&#62; head(sdf)        Wmax  Region1  105.56342   Basin2   40.00000   Basin4   51.06743   Basin5   87.34328 Florida9   87.34328      US10 106.35318      US&#62; dim(sdf)[1] 2096    2But if the dataset was much larger (with thousands of variables) with also some problems on specific lines, we now have a nice code to import our database.