<!-- Generated by pkgdown: do not edit by hand -->
<!DOCTYPE html>
<html lang="en">
  <head>
  <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">

<title>ClusterFunctions for Docker — makeClusterFunctionsDocker • batchtools</title>


<!-- jquery -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<!-- Bootstrap -->

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/css/bootstrap.min.css" integrity="sha256-bZLfwXAP04zRMK2BjiO8iu9pf4FbLqX6zitd+tIvLhE=" crossorigin="anonymous" />

<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js" integrity="sha256-nuL8/2cJ5NDSSwnKD8VqreErSWHtnEP9E7AySL+1ev4=" crossorigin="anonymous"></script>

<!-- bootstrap-toc -->
<link rel="stylesheet" href="../bootstrap-toc.css">
<script src="../bootstrap-toc.js"></script>

<!-- Font Awesome icons -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous" />
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous" />

<!-- clipboard.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script>

<!-- headroom.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script>

<!-- pkgdown -->
<link href="../pkgdown.css" rel="stylesheet">
<script src="../pkgdown.js"></script>




<meta property="og:title" content="ClusterFunctions for Docker — makeClusterFunctionsDocker" />
<meta property="og:description" content="Cluster functions for Docker/Docker Swarm (https://docs.docker.com/engine/swarm/).
The submitJob function executes
docker [docker.args] run --detach=true [image.args] [resources] [image] [cmd].
Arguments docker.args, image.args and image can be set on construction.
The resources part takes the named resources ncpus and memory
from submitJobs and maps them to the arguments --cpu-shares and --memory
(in Megabytes). The resource threads is mapped to the environment variables &amp;#8220;OMP_NUM_THREADS&amp;#8221;
and &amp;#8220;OPENBLAS_NUM_THREADS&amp;#8221;.
To reliably identify jobs in the swarm, jobs are labeled with &amp;#8220;batchtools=[job.hash]&amp;#8221; and named
using the current login name (label &amp;#8220;user&amp;#8221;) and the job hash (label &amp;#8220;batchtools&amp;#8221;).
listJobsRunning uses docker [docker.args] ps --format={{.ID}} to filter for running jobs.
killJobs uses docker [docker.args] kill [batch.id] to filter for running jobs.
These cluster functions use a Hook to remove finished jobs before a new submit and every time the Registry
is synchronized (using syncRegistry).
This is currently required because docker does not remove terminated containers automatically.
Use docker ps -a --filter 'label=batchtools' --filter 'status=exited' to identify and remove terminated
containers manually (or usa a cron job)." />




<!-- mathjax -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script>

<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->



  </head>

  <body data-spy="scroll" data-target="#toc">
    <div class="container template-reference-topic">
      <header>
      <div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <span class="navbar-brand">
        <a class="navbar-link" href="../index.html">batchtools</a>
        <span class="version label label-default" data-toggle="tooltip" data-placement="bottom" title="Released version">0.9.14</span>
      </span>
    </div>

    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
        <li>
  <a href="../index.html">
    <span class="fas fa fas fa-home fa-lg"></span>
     
  </a>
</li>
<li>
  <a href="../articles/batchtools.html">Get started</a>
</li>
<li>
  <a href="../reference/index.html">Reference</a>
</li>
<li>
  <a href="../news/index.html">Changelog</a>
</li>
      </ul>
      <ul class="nav navbar-nav navbar-right">
        <li>
  <a href="https://github.com/mllg/batchtools/">
    <span class="fab fa fab fa-github fa-lg"></span>
     
  </a>
</li>
      </ul>
      
    </div><!--/.nav-collapse -->
  </div><!--/.container -->
</div><!--/.navbar -->

      

      </header>

<div class="row">
  <div class="col-md-9 contents">
    <div class="page-header">
    <h1>ClusterFunctions for Docker</h1>
    <small class="dont-index">Source: <a href='https://github.com/mllg/batchtools/blob/master/R/clusterFunctionsDocker.R'><code>R/clusterFunctionsDocker.R</code></a></small>
    <div class="hidden name"><code>makeClusterFunctionsDocker.Rd</code></div>
    </div>

    <div class="ref-description">
    <p>Cluster functions for Docker/Docker Swarm (<a href='https://docs.docker.com/engine/swarm/'>https://docs.docker.com/engine/swarm/</a>).</p>
<p>The <code>submitJob</code> function executes
<code>docker [docker.args] run --detach=true [image.args] [resources] [image] [cmd]</code>.
Arguments <code>docker.args</code>, <code>image.args</code> and <code>image</code> can be set on construction.
The <code>resources</code> part takes the named resources <code>ncpus</code> and <code>memory</code>
from <code><a href='submitJobs.html'>submitJobs</a></code> and maps them to the arguments <code>--cpu-shares</code> and <code>--memory</code>
(in Megabytes). The resource <code>threads</code> is mapped to the environment variables &#8220;OMP_NUM_THREADS&#8221;
and &#8220;OPENBLAS_NUM_THREADS&#8221;.
To reliably identify jobs in the swarm, jobs are labeled with &#8220;batchtools=[job.hash]&#8221; and named
using the current login name (label &#8220;user&#8221;) and the job hash (label &#8220;batchtools&#8221;).</p>
<p><code>listJobsRunning</code> uses <code>docker [docker.args] ps --format={{.ID}}</code> to filter for running jobs.</p>
<p><code>killJobs</code> uses <code>docker [docker.args] kill [batch.id]</code> to filter for running jobs.</p>
<p>These cluster functions use a <a href='runHook.html'>Hook</a> to remove finished jobs before a new submit and every time the <a href='makeRegistry.html'>Registry</a>
is synchronized (using <code><a href='syncRegistry.html'>syncRegistry</a></code>).
This is currently required because docker does not remove terminated containers automatically.
Use <code>docker ps -a --filter 'label=batchtools' --filter 'status=exited'</code> to identify and remove terminated
containers manually (or usa a cron job).</p>
    </div>

    <pre class="usage"><span class='fu'>makeClusterFunctionsDocker</span><span class='op'>(</span>
  <span class='va'>image</span>,
  docker.args <span class='op'>=</span> <span class='fu'><a href='https://rdrr.io/r/base/character.html'>character</a></span><span class='op'>(</span><span class='fl'>0L</span><span class='op'>)</span>,
  image.args <span class='op'>=</span> <span class='fu'><a href='https://rdrr.io/r/base/character.html'>character</a></span><span class='op'>(</span><span class='fl'>0L</span><span class='op'>)</span>,
  scheduler.latency <span class='op'>=</span> <span class='fl'>1</span>,
  fs.latency <span class='op'>=</span> <span class='fl'>65</span>
<span class='op'>)</span></pre>

    <h2 class="hasAnchor" id="arguments"><a class="anchor" href="#arguments"></a>Arguments</h2>
    <table class="ref-arguments">
    <colgroup><col class="name" /><col class="desc" /></colgroup>
    <tr>
      <th>image</th>
      <td><p>[<code><a href='https://rdrr.io/r/base/character.html'>character(1)</a></code>]<br />
Name of the docker image to run.</p></td>
    </tr>
    <tr>
      <th>docker.args</th>
      <td><p>[<code>character</code>]<br />
Additional arguments passed to &#8220;docker&#8221; *before* the command (&#8220;run&#8221;, &#8220;ps&#8221; or &#8220;kill&#8221;) to execute (e.g., the docker host).</p></td>
    </tr>
    <tr>
      <th>image.args</th>
      <td><p>[<code>character</code>]<br />
Additional arguments passed to &#8220;docker run&#8221; (e.g., to define mounts or environment variables).</p></td>
    </tr>
    <tr>
      <th>scheduler.latency</th>
      <td><p>[<code><a href='https://rdrr.io/r/base/numeric.html'>numeric(1)</a></code>]<br />
Time to sleep after important interactions with the scheduler to ensure a sane state.
Currently only triggered after calling <code><a href='submitJobs.html'>submitJobs</a></code>.</p></td>
    </tr>
    <tr>
      <th>fs.latency</th>
      <td><p>[<code><a href='https://rdrr.io/r/base/numeric.html'>numeric(1)</a></code>]<br />
Expected maximum latency of the file system, in seconds.
Set to a positive number for network file systems like NFS which enables more robust (but also more expensive) mechanisms to
access files and directories.
Usually safe to set to <code>0</code> to disable the heuristic, e.g. if you are working on a local file system.</p></td>
    </tr>
    </table>

    <h2 class="hasAnchor" id="value"><a class="anchor" href="#value"></a>Value</h2>

    <p>[<code><a href='makeClusterFunctions.html'>ClusterFunctions</a></code>].</p>
    <h2 class="hasAnchor" id="see-also"><a class="anchor" href="#see-also"></a>See also</h2>

    <div class='dont-index'><p>Other ClusterFunctions: 
<code><a href='makeClusterFunctionsInteractive.html'>makeClusterFunctionsInteractive</a>()</code>,
<code><a href='makeClusterFunctionsLSF.html'>makeClusterFunctionsLSF</a>()</code>,
<code><a href='makeClusterFunctionsMulticore.html'>makeClusterFunctionsMulticore</a>()</code>,
<code><a href='makeClusterFunctionsOpenLava.html'>makeClusterFunctionsOpenLava</a>()</code>,
<code><a href='makeClusterFunctionsSGE.html'>makeClusterFunctionsSGE</a>()</code>,
<code><a href='makeClusterFunctionsSSH.html'>makeClusterFunctionsSSH</a>()</code>,
<code><a href='makeClusterFunctionsSlurm.html'>makeClusterFunctionsSlurm</a>()</code>,
<code><a href='makeClusterFunctionsSocket.html'>makeClusterFunctionsSocket</a>()</code>,
<code><a href='makeClusterFunctionsTORQUE.html'>makeClusterFunctionsTORQUE</a>()</code>,
<code><a href='makeClusterFunctions.html'>makeClusterFunctions</a>()</code></p></div>

  </div>
  <div class="col-md-3 hidden-xs hidden-sm" id="pkgdown-sidebar">
    <nav id="toc" data-toggle="toc" class="sticky-top">
      <h2 data-toc-skip>Contents</h2>
    </nav>
  </div>
</div>


      <footer>
      <div class="copyright">
  <p>Developed by Michel Lang, Bernd Bischl.</p>
</div>

<div class="pkgdown">
  <p>Site built with <a href="https://pkgdown.r-lib.org/">pkgdown</a> 1.6.1.</p>
</div>

      </footer>
   </div>

  


  </body>
</html>


