<!DOCTYPE html>
<!-- Generated by pkgdown: do not edit by hand --><html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Using Cross Validation with maxcovr • maxcovr</title>
<!-- jquery --><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script><!-- Bootstrap --><link href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.3.7/flatly/bootstrap.min.css" rel="stylesheet" crossorigin="anonymous">
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha256-U5ZEeKfGNOja007MMD3YBI0A3OSZOQbeG6z2f2Y0hu8=" crossorigin="anonymous"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css" integrity="sha256-eZrrJcwDc/3uDhsdt61sL2oOBY362qM3lon1gyExkL0=" crossorigin="anonymous">
<!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.4/clipboard.min.js" integrity="sha256-FiZwavyI2V6+EXO1U+xzLG3IKldpiTFf3153ea9zikQ=" crossorigin="anonymous"></script><!-- sticky kit --><script src="https://cdnjs.cloudflare.com/ajax/libs/sticky-kit/1.1.3/sticky-kit.min.js" integrity="sha256-c4Rlo1ZozqTPE2RLuvbusY3+SU1pQaJC0TjuhygMipw=" crossorigin="anonymous"></script><!-- pkgdown --><link href="../pkgdown.css" rel="stylesheet">
<script src="../pkgdown.js"></script><meta property="og:title" content="Using Cross Validation with maxcovr">
<meta property="og:description" content="">
<meta name="twitter:card" content="summary">
<!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>
    <div class="container template-article">
      <header><div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <span class="navbar-brand">
        <a class="navbar-link" href="../index.html">maxcovr</a>
        <span class="version label label-default" data-toggle="tooltip" data-placement="bottom" title="Released version">0.1.3.9200</span>
      </span>
    </div>

    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
<li>
  <a href="../index.html">
    <span class="fa fa-home fa-lg"></span>
     
  </a>
</li>
<li>
  <a href="../reference/index.html">Reference</a>
</li>
<li class="dropdown">
  <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
    Articles
     
    <span class="caret"></span>
  </a>
  <ul class="dropdown-menu" role="menu">
<li>
      <a href="../articles/cross_validation_with_maxcovr.html">Using Cross Validation with maxcovr</a>
    </li>
    <li>
      <a href="../articles/intro_to_maxcovr.html">Using maxcovr</a>
    </li>
    <li>
      <a href="../articles/using_max_coverage_relocation.html">Using max_coverage_relocation</a>
    </li>
  </ul>
</li>
<li>
  <a href="../news/index.html">Changelog</a>
</li>
      </ul>
<ul class="nav navbar-nav navbar-right">
<li>
  <a href="https://github.com/njtierney/maxcovr">
    <span class="fa fa-github fa-lg"></span>
     
  </a>
</li>
      </ul>
</div>
<!--/.nav-collapse -->
  </div>
<!--/.container -->
</div>
<!--/.navbar -->

      
      </header><div class="row">
  <div class="col-md-9 contents">
    <div class="page-header toc-ignore">
      <h1>Using Cross Validation with maxcovr</h1>
                        <h4 class="author">Nicholas Tierney</h4>
            
            <h4 class="date">2019-06-13</h4>
      
      <small class="dont-index">Source: <a href="https://github.com/njtierney/maxcovr/blob/master/vignettes/cross_validation_with_maxcovr.Rmd"><code>vignettes/cross_validation_with_maxcovr.Rmd</code></a></small>
      <div class="hidden name"><code>cross_validation_with_maxcovr.Rmd</code></div>

    </div>

    
    
<p>This vignette provides a guide to performing cross validation with modelr and maxcovr. Cross validation is a bit of a strange concept in this context - we aren’t trying to make predictions here, but what we are trying to do is appropriately evaluate the performance of maxcovr in a fairer way.</p>
<p>In the future if this is a feature that is under demand, I will incorporate cross validation into maxcovr. In the mean time, here’s a vignette.</p>
<div id="performing-cross-validation-on-max_coverage" class="section level1">
<h1 class="hasAnchor">
<a href="#performing-cross-validation-on-max_coverage" class="anchor"></a>Performing cross validation on max_coverage</h1>
<p>We will stick with the previous example using york and york_crime</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb1-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(maxcovr)</a>
<a class="sourceLine" id="cb1-2" title="2"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(dplyr)</a></code></pre></div>
<pre><code>## 
## Attaching package: 'dplyr'</code></pre>
<pre><code>## The following objects are masked from 'package:stats':
## 
##     filter, lag</code></pre>
<pre><code>## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union</code></pre>
<div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb5-1" title="1"><span class="co"># subset to be the places with towers built on them.</span></a>
<a class="sourceLine" id="cb5-2" title="2">york_selected &lt;-<span class="st"> </span>york <span class="op">%&gt;%</span><span class="st"> </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/filter.html">filter</a></span>(grade <span class="op">==</span><span class="st"> "I"</span>)</a>
<a class="sourceLine" id="cb5-3" title="3"></a>
<a class="sourceLine" id="cb5-4" title="4">york_unselected &lt;-<span class="st"> </span>york <span class="op">%&gt;%</span><span class="st"> </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/filter.html">filter</a></span>(grade <span class="op">!=</span><span class="st"> "I"</span>)</a></code></pre></div>
<p>Thanks to the <code>modelr</code> package, it is relatively straightforward to perform cross validation.</p>
<div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb6-1" title="1"><span class="co"># first we partition the data into 5 folds</span></a>
<a class="sourceLine" id="cb6-2" title="2"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(modelr)</a>
<a class="sourceLine" id="cb6-3" title="3"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(purrr)</a>
<a class="sourceLine" id="cb6-4" title="4"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(tibble)</a>
<a class="sourceLine" id="cb6-5" title="5">mc_cv &lt;-<span class="st"> </span>modelr<span class="op">::</span><span class="kw"><a href="https://www.rdocumentation.org/packages/modelr/topics/crossv_mc">crossv_kfold</a></span>(york_crime, <span class="dv">5</span>) <span class="op">%&gt;%</span><span class="st"> </span></a>
<a class="sourceLine" id="cb6-6" title="6"><span class="st">    </span><span class="co"># we change the test and train sets from the `resample`</span></a>
<a class="sourceLine" id="cb6-7" title="7"><span class="st">    </span><span class="co"># to tibbles</span></a>
<a class="sourceLine" id="cb6-8" title="8"><span class="st">    </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate</a></span>(<span class="dt">test =</span> <span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map</a></span>(test,as_tibble),</a>
<a class="sourceLine" id="cb6-9" title="9">           <span class="dt">train =</span> <span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map</a></span>(train,as_tibble))</a></code></pre></div>
<p>This creates a dataframe with test and training sets</p>
<div class="sourceCode" id="cb7"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb7-1" title="1">mc_cv</a></code></pre></div>
<pre><code>## # A tibble: 5 x 3
##   train                 test                .id  
##   &lt;list&gt;                &lt;list&gt;              &lt;chr&gt;
## 1 &lt;tibble [1,451 × 12]&gt; &lt;tibble [363 × 12]&gt; 1    
## 2 &lt;tibble [1,451 × 12]&gt; &lt;tibble [363 × 12]&gt; 2    
## 3 &lt;tibble [1,451 × 12]&gt; &lt;tibble [363 × 12]&gt; 3    
## 4 &lt;tibble [1,451 × 12]&gt; &lt;tibble [363 × 12]&gt; 4    
## 5 &lt;tibble [1,452 × 12]&gt; &lt;tibble [362 × 12]&gt; 5</code></pre>
<p>We then fit the model on the training set using <code>map_df</code>, from the <code>purrr</code> package.</p>
<div class="sourceCode" id="cb9"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb9-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(purrr)</a>
<a class="sourceLine" id="cb9-2" title="2"><span class="co"># then we fit the model</span></a>
<a class="sourceLine" id="cb9-3" title="3"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb9-4" title="4">    mc_cv_fit &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb9-5" title="5">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb9-6" title="6">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb9-7" title="7">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb9-8" title="8">                                   <span class="dt">n_added =</span> <span class="dv">20</span>,</a>
<a class="sourceLine" id="cb9-9" title="9">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb9-10" title="10">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##   3.536   0.423   3.978</code></pre>
<p>Then we can use the <code>summary_mc_cv</code> function to extract out the summaries from each fold. This summary takes the facilities placed using the training set of users, and then takes the test set of users and counts what percent of these are being covered by the training model.</p>
<div class="sourceCode" id="cb11"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb11-1" title="1">summarised_cv &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit, mc_cv)</a>
<a class="sourceLine" id="cb11-2" title="2"></a>
<a class="sourceLine" id="cb11-3" title="3">summarised_cv <span class="op">%&gt;%</span><span class="st"> </span>knitr<span class="op">::</span><span class="kw"><a href="https://www.rdocumentation.org/packages/knitr/topics/kable">kable</a></span>()</a></code></pre></div>
<table class="table">
<thead><tr class="header">
<th align="right">n_added</th>
<th align="left">n_fold</th>
<th align="right">distance_within</th>
<th align="right">n_cov</th>
<th align="right">pct_cov</th>
<th align="right">n_not_cov</th>
<th align="right">pct_not_cov</th>
<th align="right">dist_avg</th>
<th align="right">dist_sd</th>
</tr></thead>
<tbody>
<tr class="odd">
<td align="right">20</td>
<td align="left">1</td>
<td align="right">100</td>
<td align="right">58</td>
<td align="right">0.1597796</td>
<td align="right">305</td>
<td align="right">0.8402204</td>
<td align="right">1065.714</td>
<td align="right">1382.361</td>
</tr>
<tr class="even">
<td align="right">20</td>
<td align="left">2</td>
<td align="right">100</td>
<td align="right">45</td>
<td align="right">0.1239669</td>
<td align="right">318</td>
<td align="right">0.8760331</td>
<td align="right">1175.605</td>
<td align="right">1604.221</td>
</tr>
<tr class="odd">
<td align="right">20</td>
<td align="left">3</td>
<td align="right">100</td>
<td align="right">37</td>
<td align="right">0.1019284</td>
<td align="right">326</td>
<td align="right">0.8980716</td>
<td align="right">1325.302</td>
<td align="right">1703.502</td>
</tr>
<tr class="even">
<td align="right">20</td>
<td align="left">4</td>
<td align="right">100</td>
<td align="right">53</td>
<td align="right">0.1460055</td>
<td align="right">310</td>
<td align="right">0.8539945</td>
<td align="right">1024.408</td>
<td align="right">1341.573</td>
</tr>
<tr class="odd">
<td align="right">20</td>
<td align="left">5</td>
<td align="right">100</td>
<td align="right">49</td>
<td align="right">0.1353591</td>
<td align="right">313</td>
<td align="right">0.8646409</td>
<td align="right">1104.269</td>
<td align="right">1310.461</td>
</tr>
</tbody>
</table>
<p>Eyeballing the values, it looks like the pct coverage stays around 10%, but we can plot it to get a better idea. We can overlay the coverage obtained using the full dataset to get an idea of how we are performing.</p>
<div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb12-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/library">library</a></span>(ggplot2)</a>
<a class="sourceLine" id="cb12-2" title="2">summarised_cv <span class="op">%&gt;%</span></a>
<a class="sourceLine" id="cb12-3" title="3"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggplot">ggplot</a></span>(<span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/aes">aes</a></span>(<span class="dt">x =</span> n_fold,</a>
<a class="sourceLine" id="cb12-4" title="4">               <span class="dt">y =</span> pct_cov)) <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb12-5" title="5"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_point">geom_point</a></span>() <span class="op">+</span></a>
<a class="sourceLine" id="cb12-6" title="6"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_path">geom_line</a></span>(<span class="dt">group =</span> <span class="dv">1</span>) <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb12-7" title="7"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggtheme">theme_minimal</a></span>()</a></code></pre></div>
<p><img src="cross_validation_with_maxcovr_files/figure-html/plot-summarised-cv-1.png" width="700"></p>
<p>Here we see that the pct_coverage doesn’t seem to change much across the folds.</p>
<p>Coming up next, we will explore how to perform cross validation as we increase the number of facilities added.</p>
<p>Ideally, there should be a way to do this using purrr, so we don’t have to fic 5 separate models, but perhaps this will change when we enable n_added to take a vector of values.</p>
<div class="sourceCode" id="cb13"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb13-1" title="1"><span class="co"># then we fit the model</span></a>
<a class="sourceLine" id="cb13-2" title="2"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb13-3" title="3">    mc_cv_fit_n20 &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb13-4" title="4">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb13-5" title="5">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb13-6" title="6">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb13-7" title="7">                                   <span class="dt">n_added =</span> <span class="dv">20</span>,</a>
<a class="sourceLine" id="cb13-8" title="8">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb13-9" title="9">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##   3.329   0.412   3.745</code></pre>
<div class="sourceCode" id="cb15"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb15-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb15-2" title="2">    mc_cv_fit_n40 &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb15-3" title="3">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb15-4" title="4">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb15-5" title="5">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb15-6" title="6">                                   <span class="dt">n_added =</span> <span class="dv">40</span>,</a>
<a class="sourceLine" id="cb15-7" title="7">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb15-8" title="8">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##   3.400   0.332   3.739</code></pre>
<div class="sourceCode" id="cb17"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb17-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb17-2" title="2">    mc_cv_fit_n60 &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb17-3" title="3">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb17-4" title="4">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb17-5" title="5">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb17-6" title="6">                                   <span class="dt">n_added =</span> <span class="dv">60</span>,</a>
<a class="sourceLine" id="cb17-7" title="7">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb17-8" title="8">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##   3.737   0.344   4.089</code></pre>
<div class="sourceCode" id="cb19"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb19-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb19-2" title="2">    mc_cv_fit_n80 &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb19-3" title="3">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb19-4" title="4">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb19-5" title="5">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb19-6" title="6">                                   <span class="dt">n_added =</span> <span class="dv">80</span>,</a>
<a class="sourceLine" id="cb19-7" title="7">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb19-8" title="8">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##   3.665   0.453   4.124</code></pre>
<div class="sourceCode" id="cb21"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb21-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/system.time">system.time</a></span>(</a>
<a class="sourceLine" id="cb21-2" title="2">    mc_cv_fit_n100 &lt;-<span class="st"> </span><span class="kw"><a href="https://purrr.tidyverse.org/reference/map.html">map_df</a></span>(mc_cv<span class="op">$</span>train,</a>
<a class="sourceLine" id="cb21-3" title="3">                     <span class="op">~</span><span class="kw"><a href="../reference/max_coverage.html">max_coverage</a></span>(<span class="dt">existing_facility =</span> york_selected,</a>
<a class="sourceLine" id="cb21-4" title="4">                                   <span class="dt">proposed_facility =</span> york_unselected,</a>
<a class="sourceLine" id="cb21-5" title="5">                                   <span class="dt">user =</span> ., <span class="co"># training set goes here</span></a>
<a class="sourceLine" id="cb21-6" title="6">                                   <span class="dt">n_added =</span> <span class="dv">100</span>,</a>
<a class="sourceLine" id="cb21-7" title="7">                                   <span class="dt">distance_cutoff =</span> <span class="dv">100</span>))</a>
<a class="sourceLine" id="cb21-8" title="8">)</a></code></pre></div>
<pre><code>##    user  system elapsed 
##  31.324   0.593  32.095</code></pre>
<div class="sourceCode" id="cb23"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb23-1" title="1">summarised_cv_n20 &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit_n20, mc_cv)</a>
<a class="sourceLine" id="cb23-2" title="2">summarised_cv_n40 &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit_n40, mc_cv)</a>
<a class="sourceLine" id="cb23-3" title="3">summarised_cv_n60 &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit_n60, mc_cv)</a>
<a class="sourceLine" id="cb23-4" title="4">summarised_cv_n80 &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit_n80, mc_cv)</a>
<a class="sourceLine" id="cb23-5" title="5">summarised_cv_n100 &lt;-<span class="st"> </span><span class="kw"><a href="../reference/summary_mc_cv.html">summary_mc_cv</a></span>(mc_cv_fit_n100, mc_cv)</a>
<a class="sourceLine" id="cb23-6" title="6"></a>
<a class="sourceLine" id="cb23-7" title="7">bound_testing_summaries &lt;-<span class="st"> </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/bind.html">bind_rows</a></span>(summarised_cv_n20,</a>
<a class="sourceLine" id="cb23-8" title="8">                                     summarised_cv_n40,</a>
<a class="sourceLine" id="cb23-9" title="9">                                     summarised_cv_n60,</a>
<a class="sourceLine" id="cb23-10" title="10">                                     summarised_cv_n80,</a>
<a class="sourceLine" id="cb23-11" title="11">                                     summarised_cv_n100) <span class="op">%&gt;%</span></a>
<a class="sourceLine" id="cb23-12" title="12"><span class="st">    </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate</a></span>(<span class="dt">type =</span> <span class="st">"test"</span>)</a></code></pre></div>
<p>It looks like the more facilities we add, the better the coverage…mostly.</p>
<div class="sourceCode" id="cb24"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb24-1" title="1">bound_training_summaries &lt;-<span class="st"> </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/bind.html">bind_rows</a></span>(mc_cv_fit_n20<span class="op">$</span>model_coverage,</a>
<a class="sourceLine" id="cb24-2" title="2">                                      mc_cv_fit_n40<span class="op">$</span>model_coverage,</a>
<a class="sourceLine" id="cb24-3" title="3">                                      mc_cv_fit_n60<span class="op">$</span>model_coverage,</a>
<a class="sourceLine" id="cb24-4" title="4">                                      mc_cv_fit_n80<span class="op">$</span>model_coverage,</a>
<a class="sourceLine" id="cb24-5" title="5">                                      mc_cv_fit_n100<span class="op">$</span>model_coverage) <span class="op">%&gt;%</span></a>
<a class="sourceLine" id="cb24-6" title="6"><span class="st">    </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate</a></span>(<span class="dt">type =</span> <span class="st">"training"</span>)</a>
<a class="sourceLine" id="cb24-7" title="7"></a>
<a class="sourceLine" id="cb24-8" title="8">bound_all_summaries &lt;-<span class="st"> </span><span class="kw"><a href="https://dplyr.tidyverse.org/reference/bind.html">bind_rows</a></span>(bound_testing_summaries,</a>
<a class="sourceLine" id="cb24-9" title="9">                                 bound_training_summaries)</a></code></pre></div>
<div class="sourceCode" id="cb25"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb25-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggplot">ggplot</a></span>(bound_testing_summaries,</a>
<a class="sourceLine" id="cb25-2" title="2">       <span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/aes">aes</a></span>(<span class="dt">x =</span> n_fold,</a>
<a class="sourceLine" id="cb25-3" title="3">               <span class="dt">y =</span> pct_cov,</a>
<a class="sourceLine" id="cb25-4" title="4">               <span class="dt">colour =</span> <span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/factor">factor</a></span>(n_added),</a>
<a class="sourceLine" id="cb25-5" title="5">               <span class="dt">group =</span> <span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/factor">factor</a></span>(n_added))) <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb25-6" title="6"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_point">geom_point</a></span>() <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb25-7" title="7"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_path">geom_line</a></span>() <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb25-8" title="8"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggtheme">theme_minimal</a></span>()</a></code></pre></div>
<p><img src="cross_validation_with_maxcovr_files/figure-html/plot-testing-summaries-1.png" width="700"></p>
<p>Let’s look at this another way, with boxplots for the number of facilities added.</p>
<div class="sourceCode" id="cb26"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb26-1" title="1"><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggplot">ggplot</a></span>(bound_testing_summaries,</a>
<a class="sourceLine" id="cb26-2" title="2">       <span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/aes">aes</a></span>(<span class="dt">x =</span> <span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/factor">factor</a></span>(n_added),</a>
<a class="sourceLine" id="cb26-3" title="3">           <span class="dt">y =</span> n_cov)) <span class="op">+</span></a>
<a class="sourceLine" id="cb26-4" title="4"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_boxplot">geom_boxplot</a></span>() <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb26-5" title="5"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggtheme">theme_minimal</a></span>()</a></code></pre></div>
<p><img src="cross_validation_with_maxcovr_files/figure-html/plot-ncov-1.png" width="700"></p>
<p>We can also compare the % coverage for the test and training datasets</p>
<div class="sourceCode" id="cb27"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb27-1" title="1">bound_all_summaries <span class="op">%&gt;%</span></a>
<a class="sourceLine" id="cb27-2" title="2"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggplot">ggplot</a></span>(<span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/aes">aes</a></span>(<span class="dt">x =</span> <span class="kw"><a href="https://www.rdocumentation.org/packages/base/topics/factor">factor</a></span>(n_added),</a>
<a class="sourceLine" id="cb27-3" title="3">               <span class="dt">y =</span> pct_cov,</a>
<a class="sourceLine" id="cb27-4" title="4">               <span class="dt">fill =</span> type)) <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb27-5" title="5"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/geom_boxplot">geom_boxplot</a></span>() <span class="op">+</span><span class="st"> </span></a>
<a class="sourceLine" id="cb27-6" title="6"><span class="st">    </span><span class="kw"><a href="https://www.rdocumentation.org/packages/ggplot2/topics/ggtheme">theme_minimal</a></span>()</a></code></pre></div>
<p><img src="cross_validation_with_maxcovr_files/figure-html/plot-all-summaries-1.png" width="700"></p>
</div>
  </div>

  <div class="col-md-3 hidden-xs hidden-sm" id="sidebar">
        <div id="tocnav">
      <h2 class="hasAnchor">
<a href="#tocnav" class="anchor"></a>Contents</h2>
      <ul class="nav nav-pills nav-stacked">
<li><a href="#performing-cross-validation-on-max_coverage">Performing cross validation on max_coverage</a></li>
      </ul>
</div>
      </div>

</div>


      <footer><div class="copyright">
  <p>Developed by Nicholas Tierney.</p>
</div>

<div class="pkgdown">
  <p>Site built with <a href="https://pkgdown.r-lib.org/">pkgdown</a> 1.3.0.</p>
</div>
      </footer>
</div>

  

  </body>
</html>
