<!DOCTYPE html>
<!-- Generated by pkgdown: do not edit by hand --><html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Basic autoencoder • ruta</title>
<!-- jquery --><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script><!-- Bootstrap --><link href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.4.0/lumen/bootstrap.min.css" rel="stylesheet" crossorigin="anonymous">
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js" integrity="sha256-nuL8/2cJ5NDSSwnKD8VqreErSWHtnEP9E7AySL+1ev4=" crossorigin="anonymous"></script><!-- bootstrap-toc --><link rel="stylesheet" href="../../bootstrap-toc.css">
<script src="../../bootstrap-toc.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous">
<!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- pkgdown --><link href="../../pkgdown.css" rel="stylesheet">
<script src="../../pkgdown.js"></script><meta property="og:title" content="Basic autoencoder">
<meta property="og:description" content="ruta">
<!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body data-spy="scroll" data-target="#toc">
    

    <div class="container template-article">
      <header><div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <span class="navbar-brand">
        <a class="navbar-link" href="../../index.html">ruta</a>
        <span class="version label label-default" data-toggle="tooltip" data-placement="bottom" title="">1.2.0</span>
      </span>
    </div>

    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
<li>
  <a href="../../">
    <span class="fa fa-home"></span>
     
    Home
  </a>
</li>
<li>
  <a href="../../articles/examples/autoencoder_basic.html">
    <span class="fa fa-code"></span>
     
    Get started
  </a>
</li>
<li class="dropdown">
  <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
    Examples
     
    <span class="caret"></span>
  </a>
  <ul class="dropdown-menu" role="menu">
<li>
      <a href="../../articles/architectures.html">Building neural architectures</a>
    </li>
    <li class="divider">
    </li>
<li class="dropdown-header">Autoencoder variants</li>
    <li>
      <a href="../../articles/examples/autoencoder_contractive.html">Contractive autoencoder</a>
    </li>
    <li>
      <a href="../../articles/examples/autoencoder_denoising.html">Denoising autoencoder</a>
    </li>
    <li>
      <a href="../../articles/examples/autoencoder_robust.html">Robust autoencoder</a>
    </li>
    <li>
      <a href="../../articles/examples/autoencoder_sparse.html">Sparse autoencoder</a>
    </li>
    <li>
      <a href="../../articles/examples/autoencoder_variational.html">Variational autoencoder</a>
    </li>
    <li>
      <a href="../../articles/examples/autoencoder_convolutional.html">Convolutional autoencoder</a>
    </li>
  </ul>
</li>
<li>
  <a href="../../articles/troubleshooting.html">
    <span class="fa fa-wrench"></span>
     
    Troubleshooting
  </a>
</li>
      </ul>
<ul class="nav navbar-nav navbar-right">
<li>
  <a href="../../reference/">
    <span class="fa fa-book"></span>
     
    Documentation
  </a>
</li>
<li>
  <a href="https://github.com/fdavidcl/ruta" class="external-link">
    <span class="fa fa-github"></span>
     
  </a>
</li>
      </ul>
</div>
<!--/.nav-collapse -->
  </div>
<!--/.container -->
</div>
<!--/.navbar -->

      

      </header><div class="row">
  <div class="col-md-9 contents">
    <div class="page-header toc-ignore">
      <h1 data-toc-skip>Basic autoencoder</h1>
            
      
      <small class="dont-index">Source: <a href="https://github.com/fdavidcl/ruta/blob/HEAD/vignettes/examples/autoencoder_basic.Rmd" class="external-link"><code>vignettes/examples/autoencoder_basic.Rmd</code></a></small>
      <div class="hidden name"><code>autoencoder_basic.Rmd</code></div>

    </div>

    
    
<p>The main objective of most autoencoders is to learn a new set of
features out of nonlinear transformations which present some useful
properties, such as lower dimension, distance preservation or noise
resilience. In this article we explore the possibilities of using just a
basic autoencoder with Ruta.</p>
<p>Ruta provides to ways of creating and training autoencoders. The
first is the straight-forward <code>autoencode</code> function. It
receives a data matrix as input and returns the learned features by an
autoencoder with a single hidden layer of the specified size:</p>
<div class="sourceCode" id="cb1"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="kw"><a href="https://rdrr.io/r/base/library.html" class="external-link">library</a></span><span class="op">(</span><span class="va"><a href="https://github.com/fdavidcl/ruta" class="external-link">ruta</a></span><span class="op">)</span></span>
<span></span>
<span><span class="va">x</span> <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/matrix.html" class="external-link">as.matrix</a></span><span class="op">(</span><span class="va">iris</span><span class="op">[</span>, <span class="fl">1</span><span class="op">:</span><span class="fl">4</span><span class="op">]</span><span class="op">)</span></span>
<span><span class="va">iris2</span> <span class="op">=</span> <span class="fu"><a href="../../reference/autoencode.html">autoencode</a></span><span class="op">(</span><span class="va">x</span>, <span class="fl">2</span><span class="op">)</span></span></code></pre></div>
<p>Since this function only returns the learned features, one cannot
reuse the trained autoencoder. This is useful, however, for
low-dimension visualizations:</p>
<div class="sourceCode" id="cb2"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="fu"><a href="https://rdrr.io/r/graphics/plot.default.html" class="external-link">plot</a></span><span class="op">(</span><span class="va">iris2</span>, col <span class="op">=</span> <span class="va">iris</span><span class="op">[</span>, <span class="fl">5</span><span class="op">]</span><span class="op">)</span></span></code></pre></div>
<p><img src="/articles/examples/autoencoder_basic_files/figure-html/aepl-1.png" width="700"></p>
<p>For more customization options and reusability, we can build an
autoencoder object (of class <code>"ruta_autoencoder"</code>). This can
be done via the <code>autoencoder</code> family of functions.</p>
<div class="sourceCode" id="cb3"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">network</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/input.html">input</a></span><span class="op">(</span><span class="op">)</span> <span class="op">+</span> <span class="fu"><a href="../../reference/dense.html">dense</a></span><span class="op">(</span><span class="fl">36</span>, <span class="st">"tanh"</span><span class="op">)</span> <span class="op">+</span> <span class="fu"><a href="../../reference/output.html">output</a></span><span class="op">(</span><span class="st">"sigmoid"</span><span class="op">)</span></span>
<span><span class="va">my_ae</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/autoencoder.html">autoencoder</a></span><span class="op">(</span><span class="va">network</span>, loss <span class="op">=</span> <span class="st">"binary_crossentropy"</span><span class="op">)</span></span></code></pre></div>
<p>Notice that this allows to define a deeper architecture: in this
example we have created an autoencoder with one 36-unit hidden layer,
apart from the input and output layers which will have as many units as
features in the training data. We have also defined activation functions
for each layer: <span class="math inline">\(\tanh\)</span> is a common
activation for the encoding layer, and setting <code>sigmoid</code> as
the output will ensure an output in the range <span class="math inline">\([0,1]\)</span>. We can also specify the loss
function we want to optimize. Here, <code>"binary_crossentropy"</code>
is an objective function which works well with binary data and data in
the <span class="math inline">\([0, 1]\)</span> interval, but we can
specify any <a href="https://keras.io/losses/" class="external-link">loss available in
Keras</a>, another common one would be
<code>"mean_squared_error"</code>.</p>
<p>Now we need data to train the autoencoder. The <a href="https://cran.r-project.org/package=keras" class="external-link">keras</a> package
provides several well-known datasets:</p>
<div class="sourceCode" id="cb4"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">mnist</span> <span class="op">=</span> <span class="fu">keras</span><span class="fu">::</span><span class="fu"><a href="https://rdrr.io/pkg/keras/man/dataset_mnist.html" class="external-link">dataset_mnist</a></span><span class="op">(</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># Normalization to the [0, 1] interval</span></span>
<span><span class="va">x_train</span> <span class="op">&lt;-</span> <span class="fu">keras</span><span class="fu">::</span><span class="fu"><a href="https://rstudio.github.io/reticulate/reference/array_reshape.html" class="external-link">array_reshape</a></span><span class="op">(</span></span>
<span>  <span class="va">mnist</span><span class="op">$</span><span class="va">train</span><span class="op">$</span><span class="va">x</span>, <span class="fu"><a href="https://rdrr.io/r/base/c.html" class="external-link">c</a></span><span class="op">(</span><span class="fu"><a href="https://rdrr.io/r/base/dim.html" class="external-link">dim</a></span><span class="op">(</span><span class="va">mnist</span><span class="op">$</span><span class="va">train</span><span class="op">$</span><span class="va">x</span><span class="op">)</span><span class="op">[</span><span class="fl">1</span><span class="op">]</span>, <span class="fl">784</span><span class="op">)</span></span>
<span><span class="op">)</span></span>
<span><span class="va">x_train</span> <span class="op">&lt;-</span> <span class="va">x_train</span> <span class="op">/</span> <span class="fl">255.0</span></span>
<span><span class="va">x_test</span> <span class="op">&lt;-</span> <span class="fu">keras</span><span class="fu">::</span><span class="fu"><a href="https://rstudio.github.io/reticulate/reference/array_reshape.html" class="external-link">array_reshape</a></span><span class="op">(</span></span>
<span>  <span class="va">mnist</span><span class="op">$</span><span class="va">test</span><span class="op">$</span><span class="va">x</span>, <span class="fu"><a href="https://rdrr.io/r/base/c.html" class="external-link">c</a></span><span class="op">(</span><span class="fu"><a href="https://rdrr.io/r/base/dim.html" class="external-link">dim</a></span><span class="op">(</span><span class="va">mnist</span><span class="op">$</span><span class="va">test</span><span class="op">$</span><span class="va">x</span><span class="op">)</span><span class="op">[</span><span class="fl">1</span><span class="op">]</span>, <span class="fl">784</span><span class="op">)</span></span>
<span><span class="op">)</span></span>
<span><span class="va">x_test</span> <span class="op">&lt;-</span> <span class="va">x_test</span> <span class="op">/</span> <span class="fl">255.0</span></span></code></pre></div>
<p>To train this autoencoder we can specify the <a href="https://keras.io/optimizers/" class="external-link">optimizer</a>
(e.g. <code>"sgd"</code>, <code>"rmsprop"</code>…), number of epochs and
other parameters.</p>
<div class="sourceCode" id="cb5"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">model</span> <span class="op">=</span> <span class="fu"><a href="../../reference/train.ruta_autoencoder.html">train</a></span><span class="op">(</span><span class="va">my_ae</span>, <span class="va">x_train</span>, epochs <span class="op">=</span> <span class="fl">10</span><span class="op">)</span></span></code></pre></div>
<p>Once we have a trained model, we can use it for several purposes. We
can encode input data and decode the resulting codifications, for
example:</p>
<div class="sourceCode" id="cb6"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">decoded</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/reconstruct.html">reconstruct</a></span><span class="op">(</span><span class="va">model</span>, <span class="va">x_test</span><span class="op">)</span></span></code></pre></div>
<p>This is equivalent to doing the same task in two steps, as
follows:</p>
<div class="sourceCode" id="cb7"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">encoded</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/encode.html">encode</a></span><span class="op">(</span><span class="va">model</span>, <span class="va">x_test</span><span class="op">)</span></span>
<span><span class="va">decoded</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/decode.html">decode</a></span><span class="op">(</span><span class="va">model</span>, <span class="va">encoded</span><span class="op">)</span></span></code></pre></div>
<p>Lastly, let’s display our reconstructions as images:</p>
<div class="sourceCode" id="cb8"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">plot_digit</span> <span class="op">&lt;-</span> <span class="kw">function</span><span class="op">(</span><span class="va">digit</span>, <span class="va">...</span><span class="op">)</span> <span class="op">{</span></span>
<span>  <span class="fu"><a href="https://rdrr.io/r/graphics/image.html" class="external-link">image</a></span><span class="op">(</span><span class="fu">keras</span><span class="fu">::</span><span class="fu"><a href="https://rstudio.github.io/reticulate/reference/array_reshape.html" class="external-link">array_reshape</a></span><span class="op">(</span><span class="va">digit</span>, <span class="fu"><a href="https://rdrr.io/r/base/c.html" class="external-link">c</a></span><span class="op">(</span><span class="fl">28</span>, <span class="fl">28</span><span class="op">)</span>, <span class="st">"F"</span><span class="op">)</span><span class="op">[</span>, <span class="fl">28</span><span class="op">:</span><span class="fl">1</span><span class="op">]</span>, xaxt <span class="op">=</span> <span class="st">"n"</span>, yaxt <span class="op">=</span> <span class="st">"n"</span>, col<span class="op">=</span><span class="fu"><a href="https://rdrr.io/r/grDevices/gray.html" class="external-link">gray</a></span><span class="op">(</span><span class="op">(</span><span class="fl">255</span><span class="op">:</span><span class="fl">0</span><span class="op">)</span><span class="op">/</span><span class="fl">255</span><span class="op">)</span>, <span class="va">...</span><span class="op">)</span></span>
<span><span class="op">}</span></span>
<span></span>
<span><span class="va">plot_sample</span> <span class="op">&lt;-</span> <span class="kw">function</span><span class="op">(</span><span class="va">digits_test</span>, <span class="va">digits_dec</span>, <span class="va">sample</span><span class="op">)</span> <span class="op">{</span></span>
<span>  <span class="va">sample_size</span> <span class="op">&lt;-</span> <span class="fu"><a href="https://rdrr.io/r/base/length.html" class="external-link">length</a></span><span class="op">(</span><span class="va">sample</span><span class="op">)</span></span>
<span>  <span class="fu"><a href="https://rdrr.io/r/graphics/layout.html" class="external-link">layout</a></span><span class="op">(</span></span>
<span>    <span class="fu"><a href="https://rdrr.io/r/base/matrix.html" class="external-link">matrix</a></span><span class="op">(</span><span class="fu"><a href="https://rdrr.io/r/base/c.html" class="external-link">c</a></span><span class="op">(</span><span class="fl">1</span><span class="op">:</span><span class="va">sample_size</span>, <span class="op">(</span><span class="va">sample_size</span> <span class="op">+</span> <span class="fl">1</span><span class="op">)</span><span class="op">:</span><span class="op">(</span><span class="fl">2</span> <span class="op">*</span> <span class="va">sample_size</span><span class="op">)</span><span class="op">)</span>, byrow <span class="op">=</span> <span class="cn">F</span>, nrow <span class="op">=</span> <span class="fl">2</span><span class="op">)</span></span>
<span>  <span class="op">)</span></span>
<span></span>
<span>  <span class="kw">for</span> <span class="op">(</span><span class="va">i</span> <span class="kw">in</span> <span class="va">sample</span><span class="op">)</span> <span class="op">{</span></span>
<span>    <span class="fu"><a href="https://rdrr.io/r/graphics/par.html" class="external-link">par</a></span><span class="op">(</span>mar <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/c.html" class="external-link">c</a></span><span class="op">(</span><span class="fl">0</span>,<span class="fl">0</span>,<span class="fl">0</span>,<span class="fl">0</span><span class="op">)</span> <span class="op">+</span> <span class="fl">1</span><span class="op">)</span></span>
<span>    <span class="fu">plot_digit</span><span class="op">(</span><span class="va">digits_test</span><span class="op">[</span><span class="va">i</span>, <span class="op">]</span><span class="op">)</span></span>
<span>    <span class="fu">plot_digit</span><span class="op">(</span><span class="va">digits_dec</span><span class="op">[</span><span class="va">i</span>, <span class="op">]</span><span class="op">)</span></span>
<span>  <span class="op">}</span></span>
<span><span class="op">}</span></span>
<span></span>
<span><span class="fu">plot_sample</span><span class="op">(</span><span class="va">x_test</span>, <span class="va">decoded</span>, <span class="fl">1</span><span class="op">:</span><span class="fl">10</span><span class="op">)</span></span></code></pre></div>
<p><img src="/articles/examples/autoencoder_basic_files/figure-html/plot-1.png" width="1920"></p>
<p>Basic autoencoders provide some more options, such as weight decay of
the encoding layer and training with validation data as well:</p>
<div class="sourceCode" id="cb9"><pre class="downlit sourceCode r">
<code class="sourceCode R"><span><span class="va">my_ae_wd</span> <span class="op">&lt;-</span> <span class="fu"><a href="../../reference/add_weight_decay.html">add_weight_decay</a></span><span class="op">(</span><span class="va">my_ae</span>, decay <span class="op">=</span> <span class="fl">0.01</span><span class="op">)</span></span>
<span></span>
<span><span class="fu"><a href="../../reference/train.ruta_autoencoder.html">train</a></span><span class="op">(</span></span>
<span>  <span class="va">my_ae_wd</span>,</span>
<span>  <span class="va">x_train</span>,</span>
<span>  epochs <span class="op">=</span> <span class="fl">20</span>,</span>
<span>  optimizer <span class="op">=</span> <span class="st">"adam"</span>,</span>
<span>  batch_size <span class="op">=</span> <span class="fl">64</span>,</span>
<span>  validation_data <span class="op">=</span> <span class="va">x_test</span>,</span>
<span>  metrics <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/list.html" class="external-link">list</a></span><span class="op">(</span><span class="st">"binary_accuracy"</span><span class="op">)</span></span>
<span><span class="op">)</span></span></code></pre></div>
<p>For more details, read <a href="https://ruta.software/reference/">the
documentation</a>. For the rest of autoencoder variants, see <a href="https://ruta.software/articles/examples/">other examples</a>.</p>
  </div>

  <div class="col-md-3 hidden-xs hidden-sm" id="pkgdown-sidebar">

      </div>

</div>



      <footer><div class="copyright">
  <p></p>
<p>Developed by <a href="http://deivi.ch" class="external-link">David Charte</a>, Francisco Charte, Francisco Herrera.</p>
</div>

<div class="pkgdown">
  <p></p>
<p>Site built with <a href="https://pkgdown.r-lib.org/" class="external-link">pkgdown</a> 2.0.7.</p>
</div>

      </footer>
</div>

  


  

  </body>
</html>
