<!DOCTYPE html>
<!-- Generated by pkgdown: do not edit by hand --><html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Identifying variance components â€¢ specr</title>
<!-- jquery --><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script><!-- Bootstrap --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/css/bootstrap.min.css" integrity="sha256-bZLfwXAP04zRMK2BjiO8iu9pf4FbLqX6zitd+tIvLhE=" crossorigin="anonymous">
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js" integrity="sha256-nuL8/2cJ5NDSSwnKD8VqreErSWHtnEP9E7AySL+1ev4=" crossorigin="anonymous"></script><!-- bootstrap-toc --><link rel="stylesheet" href="../bootstrap-toc.css">
<script src="../bootstrap-toc.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous">
<!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- pkgdown --><link href="../pkgdown.css" rel="stylesheet">
<script src="../pkgdown.js"></script><meta property="og:title" content="Identifying variance components">
<meta property="og:description" content="specr">
<!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body data-spy="scroll" data-target="#toc">
    <div class="container template-article">
      <header><div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <span class="navbar-brand">
        <a class="navbar-link" href="../index.html">specr</a>
        <span class="version label label-default" data-toggle="tooltip" data-placement="bottom" title="Released version">0.2.2</span>
      </span>
    </div>

    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
<li>
  <a href="../index.html">
    <span class="fas fa fas fa-home fa-lg"></span>
     
  </a>
</li>
<li>
  <a href="../articles/specr.html">Get started</a>
</li>
<li>
  <a href="../reference/index.html">Reference</a>
</li>
<li class="dropdown">
  <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
    Articles
     
    <span class="caret"></span>
  </a>
  <ul class="dropdown-menu" role="menu">
<li>
      <a href="../articles/custom-plot.html">Customizing plots</a>
    </li>
    <li>
      <a href="../articles/decompose_var.html">Identifying variance components</a>
    </li>
    <li>
      <a href="../articles/invest-spec.html">Investigating specific specifications</a>
    </li>
    <li>
      <a href="../articles/measurement_models.html">Including latent measurement models</a>
    </li>
    <li>
      <a href="../articles/progress.html">Visualizing progress</a>
    </li>
    <li>
      <a href="../articles/random_effects.html">Including random effects</a>
    </li>
  </ul>
</li>
      </ul>
<ul class="nav navbar-nav navbar-right">
<li>
  <a href="https://github.com/masurp/specr/">
    <span class="fab fa fab fa-github fa-lg"></span>
     
  </a>
</li>
      </ul>
</div>
<!--/.nav-collapse -->
  </div>
<!--/.container -->
</div>
<!--/.navbar -->

      

      </header><script src="decompose_var_files/accessible-code-block-0.0.1/empty-anchor.js"></script><div class="row">
  <div class="col-md-9 contents">
    <div class="page-header toc-ignore">
      <h1 data-toc-skip>Identifying variance components</h1>
            
      
      <small class="dont-index">Source: <a href="https://github.com/masurp/specr/blob/master/vignettes/decompose_var.rmd"><code>vignettes/decompose_var.rmd</code></a></small>
      <div class="hidden name"><code>decompose_var.rmd</code></div>

    </div>

    
    
<p>This vignette proposes a novel approach to decompose and identify variance components in any statistical parameter obtained from a specification curve analysis. It can help to understand which analytical choices explain most of the variance in the specification curve (e.g., the variance in the effect sizes).</p>
<div id="run-the-specification-curve-analysis" class="section level2">
<h2 class="hasAnchor">
<a href="#run-the-specification-curve-analysis" class="anchor"></a>1. Run the specification curve analysis</h2>
<p>In order to have some data to work with, we again run the minimal example included in the package.</p>
<div class="sourceCode" id="cb1"><pre class="downlit">
<span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://masurp.github.io/specr/">specr</a></span><span class="op">)</span>
<span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="http://ggplot2.tidyverse.org">ggplot2</a></span><span class="op">)</span></pre></div>
<pre><code>## Warning: package 'ggplot2' was built under R version 3.6.2</code></pre>
<div class="sourceCode" id="cb3"><pre class="downlit">
<span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://dplyr.tidyverse.org">dplyr</a></span><span class="op">)</span></pre></div>
<pre><code>## Warning: package 'dplyr' was built under R version 3.6.2</code></pre>
<div class="sourceCode" id="cb5"><pre class="downlit">
<span class="co"># run spec analysis</span>
<span class="va">results</span> <span class="op">&lt;-</span> <span class="fu"><a href="../reference/run_specs.html">run_specs</a></span><span class="op">(</span><span class="va">example_data</span>,
                     y <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op">(</span><span class="st">"y1"</span>, <span class="st">"y2"</span><span class="op">)</span>,
                     x <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op">(</span><span class="st">"x1"</span>, <span class="st">"x2"</span><span class="op">)</span>,
                     model <span class="op">=</span> <span class="st">"lm"</span>,
                     controls <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op">(</span><span class="st">"c1"</span>, <span class="st">"c2"</span><span class="op">)</span>,
                     subset <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/list.html">list</a></span><span class="op">(</span>group1 <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/unique.html">unique</a></span><span class="op">(</span><span class="va">example_data</span><span class="op">$</span><span class="va">group1</span><span class="op">)</span>,
                                   group2 <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/unique.html">unique</a></span><span class="op">(</span><span class="va">example_data</span><span class="op">$</span><span class="va">group2</span><span class="op">)</span><span class="op">)</span><span class="op">)</span></pre></div>
</div>
<div id="estimate-a-simple-multilevel-model" class="section level2">
<h2 class="hasAnchor">
<a href="#estimate-a-simple-multilevel-model" class="anchor"></a>2. Estimate a simple multilevel model</h2>
<p>We can think of the specification curve analysis as a factorial design in which we investigate the influence of different types of analytical choices on a resulting coefficient or test statistic. One way to investigate these effects is to ask how much variance in the specification curve is explained by which analytical choice (or combination of analytical choices).</p>
<p>In a first step, we estimate a simple multilevel model without predictors (a so-called null model) with specific random effects that represent the analytical choices. We can used the function <code><a href="https://rdrr.io/pkg/lme4/man/lmer.html">lmer()</a></code> from the package <a href="https://cran.r-project.org/package=lme4">lme4</a> for the estimation.</p>
<div class="sourceCode" id="cb6"><pre class="downlit">
<span class="co"># Package to estimate multilevel models</span>
<span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://github.com/lme4/lme4/">lme4</a></span><span class="op">)</span></pre></div>
<pre><code>## Warning: package 'lme4' was built under R version 3.6.2</code></pre>
<div class="sourceCode" id="cb8"><pre class="downlit">
<span class="co"># Estimate model</span>
<span class="va">m1</span> <span class="op">&lt;-</span> <span class="fu"><a href="https://rdrr.io/pkg/lme4/man/lmer.html">lmer</a></span><span class="op">(</span><span class="va">estimate</span> <span class="op">~</span> <span class="fl">1</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">x</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">y</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">controls</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">subsets</span><span class="op">)</span>, data <span class="op">=</span> <span class="va">results</span><span class="op">)</span>

<span class="co"># Check model summary</span>
<span class="fu"><a href="https://rdrr.io/r/base/summary.html">summary</a></span><span class="op">(</span><span class="va">m1</span><span class="op">)</span></pre></div>
<pre><code>## Linear mixed model fit by REML ['lmerMod']
## Formula: estimate ~ 1 + (1 | x) + (1 | y) + (1 | controls) + (1 | subsets)
##    Data: results
## 
## REML criterion at convergence: 591.7
## 
## Scaled residuals: 
##      Min       1Q   Median       3Q      Max 
## -2.85511 -0.60699 -0.03595  0.55042  2.83210 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  subsets  (Intercept)  0.76166 0.8727  
##  controls (Intercept)  0.03862 0.1965  
##  y        (Intercept) 19.71524 4.4402  
##  x        (Intercept)  0.42317 0.6505  
##  Residual              1.00928 1.0046  
## Number of obs: 192, groups:  subsets, 12; controls, 4; y, 2; x, 2
## 
## Fixed effects:
##             Estimate Std. Error t value
## (Intercept)    4.079      3.186   1.281</code></pre>
<p>We have stored the random effects in the object <code>m1</code>. We already see that most variance is related to the random effect <code>y|intercept</code>. Yet, how much exactly?</p>
</div>
<div id="estimate-intraclass-correlation-coefficients" class="section level2">
<h2 class="hasAnchor">
<a href="#estimate-intraclass-correlation-coefficients" class="anchor"></a>3. Estimate intraclass correlation coefficients</h2>
<p>Specr offers two ways to decompose the variance of the specification curve. First, we can simply compute the intraclass correlation coefficients using the function <code><a href="../reference/icc_specs.html">icc_specs()</a></code>.</p>
<div class="sourceCode" id="cb10"><pre class="downlit">
<span class="fu"><a href="../reference/icc_specs.html">icc_specs</a></span><span class="op">(</span><span class="va">m1</span><span class="op">)</span> <span class="op">%&gt;%</span>
  <span class="fu"><a href="https://dplyr.tidyverse.org/reference/mutate_all.html">mutate_if</a></span><span class="op">(</span><span class="va">is.numeric</span>, <span class="va">round</span>, <span class="fl">2</span><span class="op">)</span></pre></div>
<pre><code>##        grp  vcov  icc percent
## 1  subsets  0.76 0.03    3.47
## 2 controls  0.04 0.00    0.18
## 3        y 19.72 0.90   89.83
## 4        x  0.42 0.02    1.93
## 5 Residual  1.01 0.05    4.60</code></pre>
<p>We can see that 89.8% of the variance in the obtained results is related to choosing different dependent variables. Only small parts of the variance are explained by other analytical choices.</p>
</div>
<div id="plot-variance-components" class="section level2">
<h2 class="hasAnchor">
<a href="#plot-variance-components" class="anchor"></a>4. Plot variance components</h2>
<p>Second, we can alternatively use the function <code><a href="../reference/plot_variance.html">plot_variance()</a></code> to obtain a visualization. The function calls <code><a href="../reference/icc_specs.html">icc_specs()</a></code> automatically. We can hence pass the multilevel results object directly. Further customization via ggplot2 is possible.</p>
<div class="sourceCode" id="cb12"><pre class="downlit">
<span class="fu"><a href="../reference/plot_variance.html">plot_variance</a></span><span class="op">(</span><span class="va">m1</span><span class="op">)</span> <span class="op">+</span>
  <span class="fu"><a href="https://ggplot2.tidyverse.org/reference/lims.html">ylim</a></span><span class="op">(</span><span class="fl">0</span>, <span class="fl">100</span><span class="op">)</span></pre></div>
<p><img src="decompose_var_files/figure-html/unnamed-chunk-4-1.png" width="700"></p>
</div>
<div id="further-investigating-variance-components" class="section level2">
<h2 class="hasAnchor">
<a href="#further-investigating-variance-components" class="anchor"></a>5. Further investigating variance components</h2>
<p>You might ask yourself why we did not include the multilevel model estimation in the <code><a href="../reference/icc_specs.html">icc_specs()</a></code> function. The reason is straight-forward: In many cases, it might be of interest to decompose the variance in different ways (e.g., including interactions between the analytical choices). For example, we could specify the a new multilevel model that includes alternative random effects (e.g., interactions between the analytical choices) and we can again obtain a result table or a plot.</p>
<div class="sourceCode" id="cb13"><pre class="downlit">
<span class="va">m2</span> <span class="op">&lt;-</span> <span class="fu"><a href="https://rdrr.io/pkg/lme4/man/lmer.html">lmer</a></span><span class="op">(</span><span class="va">estimate</span> <span class="op">~</span> <span class="fl">1</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">x</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">y</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">controls</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">subsets</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">x</span><span class="op">:</span><span class="va">y</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">y</span><span class="op">:</span><span class="va">controls</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">y</span><span class="op">:</span><span class="va">subsets</span><span class="op">)</span>, data <span class="op">=</span> <span class="va">results</span><span class="op">)</span>

<span class="co"># Get table</span>
<span class="fu"><a href="../reference/icc_specs.html">icc_specs</a></span><span class="op">(</span><span class="va">m2</span><span class="op">)</span> <span class="op">%&gt;%</span>
  <span class="fu"><a href="https://dplyr.tidyverse.org/reference/mutate_all.html">mutate_if</a></span><span class="op">(</span><span class="va">is.numeric</span>, <span class="va">round</span>, <span class="fl">2</span><span class="op">)</span></pre></div>
<pre><code>##          grp  vcov  icc percent
## 1  y:subsets  0.76 0.03    3.46
## 2    subsets  0.41 0.02    1.86
## 3 y:controls  0.00 0.00    0.00
## 4        x:y  0.02 0.00    0.09
## 5   controls  0.05 0.00    0.21
## 6          y 19.65 0.90   89.64
## 7          x  0.42 0.02    1.91
## 8   Residual  0.62 0.03    2.84</code></pre>
<div class="sourceCode" id="cb15"><pre class="downlit">
<span class="co"># Plot results</span>
<span class="fu"><a href="../reference/plot_variance.html">plot_variance</a></span><span class="op">(</span><span class="va">m2</span><span class="op">)</span></pre></div>
<p><img src="decompose_var_files/figure-html/unnamed-chunk-5-1.png" width="700"></p>
<p>We can see that the interaction between analytical choices relating to subsets and dependent variables explains 3.5%.</p>
</div>
<div id="investigating-other-statistics" class="section level2">
<h2 class="hasAnchor">
<a href="#investigating-other-statistics" class="anchor"></a>6. Investigating other statistics</h2>
<p>Finally it should be noted that any other parameter in the data frame can be investigated as well (e.g., not only the estimate such as a regression coefficient, but also the p.value).</p>
<div class="sourceCode" id="cb16"><pre class="downlit">
<span class="va">m3</span> <span class="op">&lt;-</span> <span class="fu"><a href="https://rdrr.io/pkg/lme4/man/lmer.html">lmer</a></span><span class="op">(</span><span class="va">p.value</span> <span class="op">~</span> <span class="fl">1</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">x</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">y</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">controls</span><span class="op">)</span> <span class="op">+</span> <span class="op">(</span><span class="fl">1</span><span class="op">|</span><span class="va">subsets</span><span class="op">)</span>, data <span class="op">=</span> <span class="va">results</span><span class="op">)</span></pre></div>
<pre><code>## boundary (singular) fit: see ?isSingular</code></pre>
<div class="sourceCode" id="cb18"><pre class="downlit">
<span class="fu"><a href="../reference/icc_specs.html">icc_specs</a></span><span class="op">(</span><span class="va">m3</span><span class="op">)</span> <span class="op">%&gt;%</span>
  <span class="fu"><a href="https://dplyr.tidyverse.org/reference/mutate_all.html">mutate_if</a></span><span class="op">(</span><span class="va">is.numeric</span>, <span class="va">round</span>, <span class="fl">2</span><span class="op">)</span></pre></div>
<pre><code>##        grp vcov  icc percent
## 1  subsets 0.00 0.04    4.07
## 2 controls 0.00 0.00    0.00
## 3        y 0.02 0.30   29.95
## 4        x 0.00 0.01    0.74
## 5 Residual 0.03 0.65   65.25</code></pre>
<div class="sourceCode" id="cb20"><pre class="downlit">
<span class="fu"><a href="../reference/plot_variance.html">plot_variance</a></span><span class="op">(</span><span class="va">m3</span><span class="op">)</span></pre></div>
<p><img src="decompose_var_files/figure-html/unnamed-chunk-6-1.png" width="700"></p>
</div>
  </div>

  <div class="col-md-3 hidden-xs hidden-sm" id="pkgdown-sidebar">

        <nav id="toc" data-toggle="toc"><h2 data-toc-skip>Contents</h2>
    </nav>
</div>

</div>



      <footer><div class="copyright">
  <p>Developed by Philipp K. Masur, Michael Scharkow.</p>
</div>

<div class="pkgdown">
  <p>Site built with <a href="https://pkgdown.r-lib.org/">pkgdown</a> 1.6.1.</p>
</div>

      </footer>
</div>

  


  </body>
</html>
