<!-- Generated by pkgdown: do not edit by hand -->
<!DOCTYPE html>
<html>
  <head>
  <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">

<title>Fit linear regression with mixture-of-normals priors using
  variational approximation methods. — varbvsmix • varbvs</title>

<!-- jquery -->
<script src="https://code.jquery.com/jquery-3.1.0.min.js" integrity="sha384-nrOSfDHtoPMzJHjVTdCopGqIqeYETSXhZDFyniQ8ZHcVy08QesyHcnOUpMpqnmWq" crossorigin="anonymous"></script>
<!-- Bootstrap -->

<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>

<!-- Font Awesome icons -->
<link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-T8Gy5hrqNKT+hzMclPo118YTQO6cYprQmhrYwIiQ/3axmI1hQomh7Ud2hPOy8SP1" crossorigin="anonymous">


<!-- pkgdown -->
<link href="../pkgdown.css" rel="stylesheet">
<script src="../jquery.sticky-kit.min.js"></script>
<script src="../pkgdown.js"></script>

<!-- mathjax -->
<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>

<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->


  </head>

  <body>
    <div class="container template-reference-topic">
      <header>
      <div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar">
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <a class="navbar-brand" href="../index.html">varbvs</a>
    </div>
    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
        <li>
  <a href="../index.html">Home</a>
</li>
<li>
  <a href="../articles/index.html">Vignettes</a>
</li>
<li>
  <a href="../reference/index.html">Reference</a>
</li>
      </ul>
      
      <ul class="nav navbar-nav navbar-right">
        <li>
  <a href="http://github.com/pcarbo/varbvs">
    <span class="fa fa-github fa-lg"></span>
     
  </a>
</li>
      </ul>
    </div><!--/.nav-collapse -->
  </div><!--/.container -->
</div><!--/.navbar -->

      
      </header>

      <div class="row">
  <div class="col-md-9 contents">
    <div class="page-header">
    <h1>Fit linear regression with mixture-of-normals priors using
  variational approximation methods.</h1>
    </div>

    
    <p>Find the "best" fully-factorized approximation to the
  posterior distribution of the coefficients, with linear regression
  likelihood and mixture-of-normals priors on the coefficients. By
  "best", we mean the approximating distribution that locally minimizes
  the Kullback-Leibler divergence between the approximating distribution
  and the exact posterior. In the original formulation (see
  <code><a href='varbvs.html'>varbvs</a></code>), each regression coefficient was drawn
  identically from a spike-and-slab prior. Here, we instead formulate
  the ``slab'' as a mixture of normals.</p>
    

    <pre><span class='fu'>varbvsmix</span>(<span class='no'>X</span>, <span class='no'>Z</span>, <span class='no'>y</span>, <span class='no'>sa</span>, <span class='no'>sigma</span>, <span class='no'>w</span>, <span class='no'>alpha</span>, <span class='no'>mu</span>, <span class='no'>update.sigma</span>, <span class='no'>update.sa</span>,
            <span class='no'>update.w</span>, <span class='no'>w.penalty</span>, <span class='kw'>tol</span> <span class='kw'>=</span> <span class='fl'>1e-4</span>, <span class='kw'>maxiter</span> <span class='kw'>=</span> <span class='fl'>1e4</span>, <span class='kw'>verbose</span> <span class='kw'>=</span> <span class='fl'>TRUE</span>)</pre>
    
    <h2 class="hasAnchor" id="arguments"><a class="anchor" href="#arguments"></a> Arguments</h2>
    <table class="ref-arguments">
    <colgroup><col class="name" /><col class="desc" /></colgroup>
    <tr>
      <th>X</th>
      <td><p>n x p input matrix, where n is the number of samples, and p
    is the number of variables. X cannot be sparse, and cannot have any
    missing values (NA).</p></td>
    </tr>
    <tr>
      <th>Z</th>
      <td><p>n x m covariate data matrix, where m is the number of
    covariates. Do not supply an intercept as a covariate (i.e., a
    column of ones), because an intercept is automatically included in
    the regression model. For no covariates, set <code>Z = NULL</code>.</p></td>
    </tr>
    <tr>
      <th>y</th>
      <td><p>Vector of length n containing values of the continuous
    outcome.</p></td>
    </tr>
    <tr>
      <th>sa</th>
      <td><p>Vector specifying the prior variance of the regression
    coefficients (scaled by <code>sigma</code>) for each mixture
    component. The variance of the first mixture component is the
    "spike", and therefore should be exactly zero.</p></td>
    </tr>
    <tr>
      <th>sigma</th>
      <td><p>Residual variance parameter. If missing, it is
    automatically fitted to the data by computing an approximate
    maximum-likelihood estimate.</p></td>
    </tr>
    <tr>
      <th>w</th>
      <td><p>If missing, it is automatically fitted to the data by
    computing an approximate maximum-likelihood estimate.</p></td>
    </tr>
    <tr>
      <th>alpha</th>
      <td><p>Initial estimates of the approximate posterior mixture
    assignment probabilities. These should be specified as a p x K
    matrix, where K is the number of mixture components. Each row must
    add up to 1.</p></td>
    </tr>
    <tr>
      <th>mu</th>
      <td><p>Initial estimates of the approximate regression coefficients
    conditioned on being drawn from each of the K mixture
    components. These estimates should be provided as a p x K matrix,
    where K is the number of mixture components.</p></td>
    </tr>
    <tr>
      <th>update.sigma</th>
      <td><p>If <code>TRUE</code>, sigma is fitted to data using an
    approximate EM algorithm, in which case argument <code>sigma</code>, if
    provided, is the initial estimate.</p></td>
    </tr>
    <tr>
      <th>update.sa</th>
      <td><p>Currently, estimate of mixture component variances is
    not implemented, so this must be set to <code>TRUE</code>, otherwise an
    error will be generated.</p></td>
    </tr>
    <tr>
      <th>update.w</th>
      <td><p>If <code>TRUE</code>, mixture weights are fitted using an
    approximate EM algorithm, in which case argument <code>w</code>, if
    provided, is the initial estimate.</p></td>
    </tr>
    <tr>
      <th>w.penalty</th>
      <td><p>Penalty term for the mixture weights. It is useful
    for "regularizing" the estimate of <code>w</code> when we do not have a
    lot of information. It should be a vector with one positive entry
    for each mixture component. Larger values place more weight on the
    corresponding mixture components. It is based on the Dirichlet
    distribution with parameters <code>w.penalty</code>. The default is a
    vector of ones, which reduces to a uniform prior on <code>w</code>.</p></td>
    </tr>
    <tr>
      <th>tol</th>
      <td><p>Convergence tolerance for co-ordinate ascent updates.</p></td>
    </tr>
    <tr>
      <th>maxiter</th>
      <td><p>Maximum number of co-ordinate ascent iterations.</p></td>
    </tr>
    <tr>
      <th>verbose</th>
      <td><p>If <code>verbose = TRUE</code>, print progress of algorithm
    to console.</p></td>
    </tr>
    </table>
    
    <h2 class="hasAnchor" id="details"><a class="anchor" href="#details"></a>Details</h2>

    <p>See <a href = 'https://www.overleaf.com/8954189vvpqnwpxhvhq'>https://www.overleaf.com/8954189vvpqnwpxhvhq</a>.</p>
    
    <h2 class="hasAnchor" id="value"><a class="anchor" href="#value"></a>Value</h2>

    <p>An object with S3 class <code>c("varbvsmix","list")</code>.</p>
<p></p>
<dt>n</dt><dd><p>Number of data samples used to fit model.</p></dd>

  <dt>mu.cov</dt><dd><p>Posterior mean regression coefficients for covariates,
    including intercept.</p></dd>

  <dt>update.sigma</dt><dd><p>If <code>TRUE</code>, residual variance parameter
    <code>sigma</code> was fit to data.</p></dd>

  <dt>update.sa</dt><dd><p>If <code>TRUE</code>, mixture variances were fit to data.</p></dd>

  <dt>update.w</dt><dd><p>If <code>TRUE</code>, mixture weights were fit to data.</p></dd>

  <dt>w.penalty</dt><dd><p>Penalty used for updating mixture weights.</p></dd>

  <dt>sigma</dt><dd><p>Fitted or user-specified residual variance parameter.</p></dd>
  
  <dt>sa</dt><dd><p>User-specified mixture variances.</p></dd>

  <dt>w</dt><dd><p>Fitted or user-specified mixture weights.</p></dd>

  <dt>alpha</dt><dd><p>Variational estimates of posterior mixture assignent
    probabilities.</p></dd>

  <dt>mu</dt><dd><p>Variational estimates of posterior mean coefficients.</p></dd>

  <dt>s</dt><dd><p>Variational estimates of posterior variances.</p></dd>

  <dt>logZ</dt><dd><p>Variational lower bound to marginal log-likelihood at each
    iteration of the co-ordinate ascent algorithm.</p></dd>

  <dt>err</dt><dd><p>Maximum difference in the variational posterior
    probabilities at each iteration of the co-ordinate ascent
    algorithm.</p></dd>

    
    <h2 class="hasAnchor" id="see-also"><a class="anchor" href="#see-also"></a>See also</h2>

    <p><code><a href='varbvs.html'>varbvs</a></code></p>
    

    <h2 class="hasAnchor" id="examples"><a class="anchor" href="#examples"></a>Examples</h2>
    <pre class="examples">
<span class='co'># Generate the data set.</span>
<span class='fu'>set.seed</span>(<span class='fl'>1</span>)
<span class='no'>n</span>    <span class='kw'>&lt;-</span> <span class='fl'>200</span>
<span class='no'>p</span>    <span class='kw'>&lt;-</span> <span class='fl'>500</span>
<span class='no'>X</span>    <span class='kw'>&lt;-</span> <span class='fu'><a href='rand.html'>randn</a></span>(<span class='no'>n</span>,<span class='no'>p</span>)
<span class='no'>sd</span>   <span class='kw'>&lt;-</span> <span class='fu'>c</span>(<span class='fl'>0</span>,<span class='fl'>0.2</span>,<span class='fl'>0.5</span>)
<span class='no'>w</span>    <span class='kw'>&lt;-</span> <span class='fu'>c</span>(<span class='fl'>0.9</span>,<span class='fl'>0.05</span>,<span class='fl'>0.05</span>)
<span class='no'>k</span>    <span class='kw'>&lt;-</span> <span class='fu'>sample</span>(<span class='fu'>length</span>(<span class='no'>w</span>),<span class='no'>p</span>,<span class='kw'>replace</span> <span class='kw'>=</span> <span class='fl'>TRUE</span>,<span class='kw'>prob</span> <span class='kw'>=</span> <span class='no'>w</span>)
<span class='no'>beta</span> <span class='kw'>&lt;-</span> <span class='no'>sd</span>[<span class='no'>k</span>] * <span class='fu'>rnorm</span>(<span class='no'>p</span>)
<span class='no'>y</span>    <span class='kw'>&lt;-</span> <span class='fu'>c</span>(<span class='no'>X</span> <span class='kw'>%*%</span> <span class='no'>beta</span> + <span class='fu'>rnorm</span>(<span class='no'>n</span>))

<span class='co'># Fit the model to the data.</span>
<span class='no'>fit</span> <span class='kw'>&lt;-</span> <span class='fu'>varbvsmix</span>(<span class='no'>X</span>,<span class='kw'>NULL</span>,<span class='no'>y</span>,<span class='no'>sd</span>^<span class='fl'>2</span>)

<span class='co'>## Not run: ------------------------------------</span>
<span class='co'># library(lattice)</span>
<span class='co'># print(xyplot(beta.est ~ beta.true,</span>
<span class='co'>#              data.frame(beta.true = beta,</span>
<span class='co'>#                         beta.fitted = rowSums(fit$alpha * fit$mu)),</span>
<span class='co'>#              pch = 20,col = "royalblue",cex = 1))</span>
<span class='co'>## ---------------------------------------------</span></pre>
  </div>
  <div class="col-md-3 hidden-xs hidden-sm" id="sidebar">
    <h2>Contents</h2>
    <ul class="nav nav-pills nav-stacked">
      <li><a href="#arguments">Arguments</a></li>
      
      <li><a href="#details">Details</a></li>

      <li><a href="#value">Value</a></li>

      <li><a href="#see-also">See also</a></li>
      
      <li><a href="#examples">Examples</a></li>
    </ul>

    <h2>Author</h2>
    Peter Carbonetto <a href='mailto:peter.carbonetto@gmail.com'>peter.carbonetto@gmail.com</a>
  </div>
</div>

      <footer>
      <div class="copyright">
  <p>Developed by Peter Carbonetto, Matthew Stephens, David Gerard.</p>
</div>

<div class="pkgdown">
  <p>Site built with <a href="http://hadley.github.io/pkgdown/">pkgdown</a>.</p>
</div>

      </footer>
   </div>

  </body>
</html>
