<!DOCTYPE html>
<!-- Generated by pkgdown: do not edit by hand --><html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>h5Seurat File Format Specification • SeuratDisk</title>
<!-- jquery --><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script><!-- Bootstrap --><link href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.4.0/flatly/bootstrap.min.css" rel="stylesheet" crossorigin="anonymous">
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js" integrity="sha256-nuL8/2cJ5NDSSwnKD8VqreErSWHtnEP9E7AySL+1ev4=" crossorigin="anonymous"></script><!-- bootstrap-toc --><link rel="stylesheet" href="../bootstrap-toc.css">
<script src="../bootstrap-toc.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous">
<!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- pkgdown --><link href="../pkgdown.css" rel="stylesheet">
<script src="../pkgdown.js"></script><meta property="og:title" content="h5Seurat File Format Specification">
<meta property="og:description" content="SeuratDisk">
<!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body data-spy="scroll" data-target="#toc">
    <div class="container template-article">
      <header><div class="navbar navbar-default navbar-fixed-top" role="navigation">
  <div class="container">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <span class="navbar-brand">
        <a class="navbar-link" href="../index.html">SeuratDisk</a>
        <span class="version label label-danger" data-toggle="tooltip" data-placement="bottom" title="Unreleased version">0.0.0.9011</span>
      </span>
    </div>

    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav">
<li>
  <a href="../index.html">
    <span class="fas fa fas fa-home fa-lg"></span>
     
  </a>
</li>
<li>
  <a href="../reference/index.html">Reference</a>
</li>
<li class="dropdown">
  <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
    Articles
     
    <span class="caret"></span>
  </a>
  <ul class="dropdown-menu" role="menu">
<li>
      <a href="../articles/convert-anndata.html">Conversions: h5Seurat and AnnData</a>
    </li>
    <li>
      <a href="../articles/h5Seurat-load.html">Saving and Loading Data from an h5Seurat File</a>
    </li>
    <li>
      <a href="../articles/h5Seurat-spec.html">h5Seurat File Format Specification</a>
    </li>
  </ul>
</li>
      </ul>
<ul class="nav navbar-nav navbar-right">
<li>
  <a href="https://github.com/mojaveazure/seurat-disk/">
    <span class="fab fa fab fa-github fa-lg"></span>
     
  </a>
</li>
      </ul>
</div>
<!--/.nav-collapse -->
  </div>
<!--/.container -->
</div>
<!--/.navbar -->

      

      </header><div class="row">
  <div class="col-md-9 contents">
    <div class="page-header toc-ignore">
      <h1 data-toc-skip>h5Seurat File Format Specification</h1>
            
      
      <small class="dont-index">Source: <a href="https://github.com/mojaveazure/seurat-disk/blob/master/vignettes/h5Seurat-spec.Rmd"><code>vignettes/h5Seurat-spec.Rmd</code></a></small>
      <div class="hidden name"><code>h5Seurat-spec.Rmd</code></div>

    </div>

    
    
<div id="overall-file-structure" class="section level2">
<h2 class="hasAnchor">
<a href="#overall-file-structure" class="anchor"></a>Overall File Structure</h2>
<div id="required-attributes" class="section level3">
<h3 class="hasAnchor">
<a href="#required-attributes" class="anchor"></a>Required Attributes</h3>
<p>There are three required top-level HDF5 attributes: "project", "active.assay", and "version". Each of these must be a single <a href="#character-representation">character</a>. The "project" attribute corresponds to the project value of a <code>Seurat</code> object; the "active.assay" attribute is the name of the default assay and must be present in the <a href="#assay-expression-data">"assays" group</a>. The "version" corresponds to the version of Seurat that the h5Seurat file is based on.</p>
</div>
<div id="top-level-datasets-and-groups" class="section level3">
<h3 class="hasAnchor">
<a href="#top-level-datasets-and-groups" class="anchor"></a>Top-Level Datasets and Groups</h3>
<p>There are two required top-level HDF5 datasets: "cell.names" and "meta.data". The "cell.names" dataset should be a one-dimensional <a href="#character-representation">character</a> dataset, with a length equal to the number of cells present in the data. Cell names are not stored anywhere else in the h5Seurat file.</p>
<p>The "meta.data" dataset contains cell-level metadata. It should be stored as either an <a href="#data-frame-representation">HDF5 dataset or group</a>, depending on the contents of the meta data. See the <a href="#data-frame-representation">data frame representation</a> for more details.</p>
</div>
</div>
<div id="assay-expression-data" class="section level2">
<h2 class="hasAnchor">
<a href="#assay-expression-data" class="anchor"></a>Assay Expression Data</h2>
<p><a href="https://rdrr.io/cran/Seurat/man/Assay-class.html"><code>Assay</code></a> objects are stored in the top-level group "assays"; each assay is stored as its own group within the "assays" group. Within each assay group, there must be a dataset named "features" and either a dataset or group named "data"; the "features" dataset must be a one-dimensional <a href="#character-representation">character</a> dataset with a length equal to the number of total features within the assay. The "data" entry is a matrix, with dimensions of <span class="math inline">\(m_{features} x n_{cells}\)</span>; this entry may be either a dataset, if "data" is a <a href="#dense-matrix-representation">dense matrix</a>, or a group, if "data" is a <a href="#sparse-matrix-representation">sparse matrix</a>. Assay groups must also have an attribute named "key"; this is a single <a href="#character-representation">character</a> value.</p>
<p>Assay groups may also have the following optional groups and datasets:</p>
<ul>
<li>"counts": either a <a href="#dense-matrix-representation">dense</a> or <a href="#sparse-matrix-representation">sparse</a> matrix; must have the same dimesions as "data"</li>
<li>"scale.data": a <a href="#dense-matrix-representation">dense</a> matrix; if "scale.data" is present, a one-dimensional <a href="#character-representation">character</a> dataset must also be present. The "scale.data" matrix must be of dimensions <span class="math inline">\(m_{scaled features} x n_{cells}\)</span>
</li>
<li>"meta.features": a <a href="#data-frame-representation">data frame</a> with the same number of rows as values present in "features"</li>
<li>"variable.features": a one-dimensional <a href="#character-representation">character</a> dataset</li>
<li>"misc": a <a href="#list-and-custom-class-representation">list</a>
</li>
</ul>
<p>Subclasses of <a href="https://rdrr.io/cran/Seurat/man/Assay-class.html"><code>Assay</code></a> objects must also follow the same rules as <a href="#list-and-custom-class-representation">custom S4 classes</a>.</p>
</div>
<div id="dimensional-reductions" class="section level2">
<h2 class="hasAnchor">
<a href="#dimensional-reductions" class="anchor"></a>Dimensional Reductions</h2>
<p><a href="https://rdrr.io/cran/Seurat/man/DimReduc-class.html">Dimensional reduction information</a> is stored in the top-level group "reductions"; each dimensional reduction is stored as its own group within the "reductions" group. Within each dimensional reduction group, there are three required attributes: "active.assay", "key", and "global"; "active.assay" must be one or more <a href="#character-representation">character</a> values where each value is a name of an <a href="#assay-expression-data">assay</a>, "key" must be a single a <a href="#character-representation">character</a> value, and "global" must be a single <a href="#logical-representation">logical</a> value. In addition, there must also be a dataset named "cell.embeddings" representing a <a href="#dense-matrix-representation">dense matrix</a>. This matrix must have the same number of rows as cells present in the h5Seurat file.</p>
<p>Dimensional reduction groups may also have the following optional groups and datasets:</p>
<ul>
<li>"feature.loadings": ...</li>
<li>"feature.loadings.projected": ...</li>
<li>"misc": a <a href="#list-and-custom-class-representation">list</a>
</li>
<li>"jackstraw": a <a href="#list-and-custom-class-representation">custom S4 group</a>
</li>
</ul>
</div>
<div id="nearest-neighbor-graphs" class="section level2">
<h2 class="hasAnchor">
<a href="#nearest-neighbor-graphs" class="anchor"></a>Nearest-Neighbor Graphs</h2>
<p><a href="https://rdrr.io/cran/Seurat/man/Graph-class.html">Nearest-neighbor graphs</a> are stored in the top-level group "graphs"; each graph is stored as its own group within the "graphs" group. Graph names become graph group names. Graphs are stored as <a href="#sparse-matrix-representation">sparse matrices</a> with an additional HDF5 attribute: "assay.used". This HDF5 attribute should be a single <a href="#character-representation">character</a> value.</p>
</div>
<div id="spatial-image-data" class="section level2">
<h2 class="hasAnchor">
<a href="#spatial-image-data" class="anchor"></a>Spatial Image Data</h2>
<p>Spatial image data is stored in the top-level group "images"; each image is stored as its own group within the "images" group. Actual structure of the image group is dependent on the structure of the spatial image data. However, it follows the same rules as <a href="#list-and-custom-class-representation">custom S4 classes</a>.</p>
<p><strong>Note</strong>: spatial images are only supported in objects that were generated by a version of Seurat that has spatial support. Currently, this is restricted to version 3.1.5.9900 or higher.</p>
<!-- ## Cell-Level Metadata -->
</div>
<div id="command-logs" class="section level2">
<h2 class="hasAnchor">
<a href="#command-logs" class="anchor"></a>Command Logs</h2>
</div>
<div id="miscellaneous-information-and-tool-specific-results" class="section level2">
<h2 class="hasAnchor">
<a href="#miscellaneous-information-and-tool-specific-results" class="anchor"></a>Miscellaneous Information and Tool-Specific Results</h2>
<p>Miscellaneous information is stored in the top-level group "misc"; this group follows the same runs as <a href="#list-and-custom-class-representation">lists</a>. The "misc" group is required to be present, but not required to be filled.</p>
<p>Tool-specific results are stored in the top-level group "tools"; this group follows the same runs as <a href="#list-and-custom-class-representation">lists</a>. The "tools" group is required to be present, but not required to be filled.</p>
</div>
<div id="common-data-structures" class="section level2">
<h2 class="hasAnchor">
<a href="#common-data-structures" class="anchor"></a>Common Data Structures</h2>
<p>Some data types are found commonly throughout <code>Seurat</code> objects</p>
<div id="character-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#character-representation" class="anchor"></a>Character Representation</h3>
<p>All <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/character.html">character</a> values (strings in other languages) should be encoding as variable-length UTF-8 strings; this applies to HDF5 datasets (including standalone <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_STRING-class.html">string</a> datasets as well as parts of <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_COMPOUND-class.html">compound</a> datasets) and HDF5 attributes.</p>
<pre><code>#&gt; Class: H5T_STRING
#&gt; Datatype: H5T_STRING {
#&gt;       STRSIZE H5T_VARIABLE;
#&gt;       STRPAD H5T_STR_NULLTERM;
#&gt;       CSET H5T_CSET_UTF8;
#&gt;       CTYPE H5T_C_S1;
#&gt;    }</code></pre>
</div>
<div id="dense-matrix-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#dense-matrix-representation" class="anchor"></a>Dense Matrix Representation</h3>
<p>Dense matrices should be stored as a two-dimensional dataset of any type. Datasets should be written in a <a href="https://en.wikipedia.org/wiki/Row-_and_column-major_order">column-major</a> order. For column-major implementations (eg. R, Fortran), dataset dimensions on-disk should be the same as dimensions in-memory (eg. <span class="math inline">\(m_{diskrow} x n_{diskcol} \sim m_{memrow} x n_{memcol}\)</span>). For row-major implmentations (eg. C/C++, Python), dataset dimensions on-disk should appear <em>transposed</em> to dimensions in-memory (eg. <span class="math inline">\(m_{diskrow} x n_{diskcol} \sim n_{memrow} x m_{memcol}\)</span>); row-major implmemetnations transpose datasets prior to reading and writing data.</p>
<pre><code>#&gt;      [,1] [,2] [,3] [,4]
#&gt; [1,]    0    0    0    1
#&gt; [2,]    0    1    1    1
#&gt; [3,]    1    1    0    0
#&gt; Class: H5D
#&gt; Dataset: /densemat
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_IEEE_F64LE
#&gt; Space: Type=Simple     Dims=3 x 4     Maxdims=Inf x Inf
#&gt; Chunk: 32 x 32
#&gt;      [,1] [,2] [,3] [,4]
#&gt; [1,]    0    0    0    1
#&gt; [2,]    0    1    1    1
#&gt; [3,]    1    1    0    0</code></pre>
</div>
<div id="sparse-matrix-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#sparse-matrix-representation" class="anchor"></a>Sparse Matrix Representation</h3>
<p>Sparse matrices are stored as an HDF5 group with three datasets: "indices", "indptr", and "data"; the "indices" and "data" datasets must be the same length. "data" represents each non-zero element of the matrix. "indices" represents the <span class="math inline">\(0\)</span>-based row numbers for each value in "data"</p>
<pre><code>#&gt; 3 x 4 sparse Matrix of class "dgCMatrix"
#&gt;             
#&gt; [1,] . . . 1
#&gt; [2,] . 1 1 1
#&gt; [3,] 1 1 . .
#&gt; Class: H5Group
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Group: /sparsemat
#&gt; Attributes: dims
#&gt; Listing:
#&gt;     name    obj_type dataset.dims dataset.type_class
#&gt;     data H5I_DATASET            6          H5T_FLOAT
#&gt;  indices H5I_DATASET            6        H5T_INTEGER
#&gt;   indptr H5I_DATASET            5        H5T_INTEGER
#&gt; Class: H5D
#&gt; Dataset: /sparsemat/indices
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_STD_I32LE
#&gt; Space: Type=Simple     Dims=6     Maxdims=Inf
#&gt; Chunk: 2048
#&gt; [1] 2 1 2 1 0 1
#&gt; Class: H5D
#&gt; Dataset: /sparsemat/data
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_IEEE_F64LE
#&gt; Space: Type=Simple     Dims=6     Maxdims=Inf
#&gt; Chunk: 1024
#&gt; [1] 1 1 1 1 1 1</code></pre>
<p>"indptr" represents the points in "data" at which a new column is started. This dataset is <span class="math inline">\(0\)</span>-based and should be <span class="math inline">\(n_{columns} + 1\)</span> in length.</p>
<pre><code>#&gt; Class: H5D
#&gt; Dataset: /sparsemat/indptr
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_STD_I32LE
#&gt; Space: Type=Simple     Dims=5     Maxdims=Inf
#&gt; Chunk: 2048
#&gt; [1] 0 1 3 4 6</code></pre>
<p>The "indices", "indptr", and "data" datasets correspond to the "i", "p", and "x" slots in a <a href="https://rdrr.io/cran/Matrix/man/dgCMatrix-class.html"><code>dgCMatrix</code></a>, respectively.</p>
<p>There may optionally be an HDF5 attribute called "dims"; this attribute should be a two <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_INTEGER-class.html">integer</a> values corresponding to the number of rows and number of columns, in that order, in the sparse matrix.</p>
<pre><code>#&gt; [1] 3 4</code></pre>
</div>
<div id="factor-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#factor-representation" class="anchor"></a>Factor Representation</h3>
<p><a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/factor.html">Factors</a> should be stored as an HDF5 group with two datasets: "levels" and "values"</p>
<pre><code>#&gt; [1] g1 g2 g1 g1 g2
#&gt; Levels: g1 g2
#&gt; Class: H5Group
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Group: /factor
#&gt; Listing:
#&gt;    name    obj_type dataset.dims dataset.type_class
#&gt;  levels H5I_DATASET            2         H5T_STRING
#&gt;  values H5I_DATASET            5        H5T_INTEGER</code></pre>
<p>The "levels" dataset should be a <a href="#character-representation">character</a> dataset with one entry per <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/levels.html">level</a></p>
<pre><code>#&gt; Class: H5D
#&gt; Dataset: /factor/levels
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_STRING {
#&gt;       STRSIZE H5T_VARIABLE;
#&gt;       STRPAD H5T_STR_NULLTERM;
#&gt;       CSET H5T_CSET_UTF8;
#&gt;       CTYPE H5T_C_S1;
#&gt;    }
#&gt; Space: Type=Simple     Dims=2     Maxdims=Inf
#&gt; Chunk: 1024
#&gt; [1] "g1" "g2"</code></pre>
<p>The "values" dataset should be an integer dataset with one entry per value in the original factor. These integers should correspond to the factor level they had in R</p>
<pre><code>#&gt; Class: H5D
#&gt; Dataset: /factor/values
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_STD_I32LE
#&gt; Space: Type=Simple     Dims=5     Maxdims=Inf
#&gt; Chunk: 2048
#&gt; [1] 1 2 1 1 2</code></pre>
<p>The number of unique entries in "values" should not exceed the number of unique entries in "levels"</p>
<p><details><summary>Rationale</summary> Storing factors in this manner seems excessive from an R perspective. HDF5 has the concept of <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_ENUM-class.html">enumerated datasets (enums)</a> which are an efficient way to store R factors on-disk. However, <a href="http://docs.h5py.org/en/stable/special.html#enumerated-types">some implementations of HDF5 do not support enums</a> and thus lose factor level information. In order to make h5Seurat as cross-language as possible, we’ve opted to store factors as HDF5 groups instead of HDF5 enums. </details></p>
</div>
<div id="logical-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#logical-representation" class="anchor"></a>Logical Representation</h3>
<p><a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/logical.html">Logical</a> values (booleans in other languages) are encoded as integers in the following manner: <code>FALSE</code> is encoded as <code>0</code>, <code>TRUE</code> is encoded as <code>1</code>, and <code>NA</code> is encoded as <code>2</code></p>
<pre><code>#&gt; [1]  TRUE FALSE    NA
#&gt; Class: H5D
#&gt; Dataset: /logicals
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Attributes: s3class
#&gt; Datatype: H5T_STD_I32LE
#&gt; Space: Type=Simple     Dims=3     Maxdims=Inf
#&gt; Chunk: 2048
#&gt; [1] 1 0 2</code></pre>
<p>An optional HDF5 attribute named "s3class" with the value "logical" is allowed to enforce reading in the dataset as logical values. This HDF5 attribute is a single <a href="#character-representation">character</a> value.</p>
<p><details><summary>Rationale</summary></details></p>
<p>Unlike most languages, <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/logical.html">logicals</a> (or booleans) can take one of <em>three</em> values: <code>TRUE</code>, <code>FALSE</code>, or <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/NA.html"><code>NA</code></a>; as such, an extra integer value is needed to handle the additional logical value. Typically, these values are stored as <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_ENUM-class.html">enums</a> with mappings between the logical representation and integer value. However, <a href="http://docs.h5py.org/en/stable/special.html#enumerated-types">some implementations of HDF5 do not support enums</a> and thus lose the logical representation. Since the mappings are lost, all logicals are stored as integers instead.</p>
<p></p>
</div>
<div id="data-frame-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#data-frame-representation" class="anchor"></a>Data Frame Representation</h3>
<p>There are two ways of storing <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/data.frame.html">data frames</a> in h5Seurat files: as <a href="#data-frame-datasets">datasets</a> or as <a href="#data-frame-groups">groups</a>. Data frame groups are required when data frames contain <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/factor.html">factors</a>; if no factors are present, data frames can be stored in either type.</p>
<p><details><summary>Rationale</summary><a href="#factor-representation">Storing factors</a> on-disk in an HDF5 file poses a unique set of challenges. Namely, <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_ENUM-class.html">enumerated datasets (enums)</a>, which are an ideal method of storing mapping values, <a href="http://docs.h5py.org/en/stable/special.html#enumerated-types">are not supported in some implementations of HDF5</a>. As factor level information is lost under implementations that do not support HDF5 enums, we need a method of storing factors in a cross-language manner. Two options were presented: groups or <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_COMPOUND-class.html">compound datasets</a>. While the former seems excessive, the latter presents issues with unequal dataset length. Therefore, to accomodate factor level information in data frames, we utilize HDF5 groups to store data frames when one or more columns are <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/factor.html">factors</a>. </details></p>
<div id="data-frame-datasets" class="section level4">
<h4 class="hasAnchor">
<a href="#data-frame-datasets" class="anchor"></a>Data Frame Datasets</h4>
<p>Data frames stored as datasets should be a one-dimensional <a href="https://hhoeflin.github.io/hdf5r/reference/H5T_COMPOUND-class.html">compound dataset</a>. The single dimension should be equal to the number of observations (number of rows) in the data frame. Each data type in the compound dataset must adhere to the same requirements as standard datasets (eg. <a href="#character-representation">character encodings</a>, <a href="#logical-representation">logical mapping</a>, etc). The compound labels should correspond to the data frame column names.</p>
<pre><code>#&gt;    x y
#&gt; 1 g1 1
#&gt; 2 g1 2
#&gt; 3 g2 3
#&gt; 4 g1 4
#&gt; 5 g2 5
#&gt; Class: H5D
#&gt; Dataset: /dfdataset
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Access type: H5F_ACC_RDWR
#&gt; Datatype: H5T_COMPOUND {
#&gt;       H5T_STRING {
#&gt;          STRSIZE H5T_VARIABLE;
#&gt;          STRPAD H5T_STR_NULLTERM;
#&gt;          CSET H5T_CSET_UTF8;
#&gt;          CTYPE H5T_C_S1;
#&gt;       } "x" : 0;
#&gt;       H5T_STD_I32LE "y" : 8;
#&gt;    }
#&gt; Space: Type=Simple     Dims=5     Maxdims=Inf
#&gt; Chunk: 682
#&gt;    x y
#&gt; 1 g1 1
#&gt; 2 g1 2
#&gt; 3 g2 3
#&gt; 4 g1 4
#&gt; 5 g2 5</code></pre>
<p>Row names are not stored with the dataset itself, but may be stored elsewhere in the h5Seurat file, typically named <code>dataset_name.row.names</code>; an optional HDF5 attribute called "logicals" containing the names of <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/logical.html">logical</a> columns is allowed. This attribute consists of <a href="#character-representation">character values</a>.</p>
</div>
<div id="data-frame-groups" class="section level4">
<h4 class="hasAnchor">
<a href="#data-frame-groups" class="anchor"></a>Data Frame Groups</h4>
<p>Data frames stored as groups are used when <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/factor.html">factors</a> are present in the data frame. Within the data frame group, there should be one dataset or group per column. Columns that are factors are <a href="#factor-representation">stored as groups</a> while all other columns are stored as one-dimensional datasets. Each dataset within the group must adhere to the same requirements as standard datasets (eg. <a href="#character-representation">character encodings</a>, <a href="#logical-representation">logical mapping</a>, etc). The names of the datasets within the group correspond to the data frame column names</p>
<pre><code>#&gt; Class: H5Group
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Group: /dfgroup
#&gt; Attributes: colnames
#&gt; Listing:
#&gt;  name    obj_type dataset.dims dataset.type_class
#&gt;     x   H5I_GROUP         &lt;NA&gt;               &lt;NA&gt;
#&gt;     y H5I_DATASET            5        H5T_INTEGER</code></pre>
<p>Data frame row names may be stored in a dataset called "row.names" within the group; this dataset should be a one-dimensional <a href="#character-representation">character</a> dataset. There are two optional attributes allowed: "colnames" and "logicals"; the "colnames" attribute contains the column names in the same order as was present in the in-memory <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/data.frame.html">data frame</a> as <a href="#character-representation">character values</a>. This is used to control column order when reading the data frame back into memory. Note, the "colnames" attribute does not need to contain the name of every dataset.</p>
<p>The "logicals" attribute contains the names of <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/logical.html">logical</a> columns; this attribute should consist of <a href="#character-representation">character values</a>.</p>
</div>
</div>
<div id="list-and-custom-class-representation" class="section level3">
<h3 class="hasAnchor">
<a href="#list-and-custom-class-representation" class="anchor"></a>List and Custom Class Representation</h3>
<p><a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/list.html">Lists</a> are stored as HDF5 groups. Each entry in a list must be named; the names serve as the names of datasets and groups within the list group. List values are stored as HDF5 datasets or groups, depending on their R object type. For example, a list within a list would be stored as a group within the first group.</p>
<pre><code>#&gt; $a
#&gt; [1] 1 2 3
#&gt; 
#&gt; $b
#&gt; $b$b1
#&gt; [1] "hello"
#&gt; 
#&gt; $b$b2
#&gt; [1] "tomato" "potato"
#&gt; Class: H5Group
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Group: /list
#&gt; Attributes: names
#&gt; Listing:
#&gt;  name    obj_type dataset.dims dataset.type_class
#&gt;     a H5I_DATASET            3        H5T_INTEGER
#&gt;     b   H5I_GROUP         &lt;NA&gt;               &lt;NA&gt;
#&gt; Class: H5Group
#&gt; Filename: /tmp/RtmpGhTXje/file2fc262fdf9e.h5
#&gt; Group: /list/b
#&gt; Attributes: names
#&gt; Listing:
#&gt;  name    obj_type dataset.dims dataset.type_class
#&gt;    b1 H5I_DATASET            1         H5T_STRING
#&gt;    b2 H5I_DATASET            2         H5T_STRING</code></pre>
<p>Custom classes are stored as <a href="#list-and-custom-class-representation">lists</a> with a special HDF5 attribute. S3 classes are stored with the attribute "s3class", which has a value equal to the class of the object. This attribute is a variable-length <a href="#character-representation">character</a> value.</p>
<p>Custom S4 classes are also stored as <a href="#list-and-custom-class-representation">lists</a> where each entry is a slot in the S4 class. S4 class groups have an attribute named "s4class"; this attribute should be a single-length <a href="#character-representation">character</a> storing the name of the class and the package that defines the class in the form of <code>package:class</code> (eg. <code>Signac:ChromatinAssay</code>); custom S4 classes defined in the <a href="https://satijalab.org/seurat">Seurat</a> package can be named with just the class of the object.</p>
</div>
</div>
  </div>

  <div class="col-md-3 hidden-xs hidden-sm" id="pkgdown-sidebar">

        <nav id="toc" data-toggle="toc"><h2 data-toc-skip>Contents</h2>
    </nav>
</div>

</div>



      <footer><div class="copyright">
  <p>Developed by Paul Hoffman.</p>
</div>

<div class="pkgdown">
  <p>Site built with <a href="https://pkgdown.r-lib.org/">pkgdown</a> 1.5.1.</p>
</div>

      </footer>
</div>

  


  </body>
</html>
