---
title: "mirai - Torch Integration"
vignette: >
  %\VignetteIndexEntry{mirai - Torch Integration}
  %\VignetteEngine{litedown::vignette}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  out.width = "100%"
)
```
### Torch Integration

Custom serialization functions may be registered to handle external pointer type reference objects.

This allows tensors from the [`torch`](https://torch.mlverse.org/) package to be used seamlessly in 'mirai' computations.

#### Setup Steps

1. Set up dameons.

2. Create the serialization configuration, specifying 'class' as 'torch_tensor' and 'vec' as TRUE.

3. Use `everywhere()`, supplying the configuration to the '.serial' argument, and (optionally) making the `torch` package available on all daemons for convenience.

```{r tensors}
library(mirai)
library(torch)

daemons(1)

cfg <- serial_config(
  class = "torch_tensor",
  sfunc = torch:::torch_serialize,
  ufunc = torch::torch_load,
  vec = TRUE
)

everywhere(library(torch), .serial = cfg)
```
#### Example Usage

The below example creates a convolutional neural network using `torch::nn_module()`.

A set of model parameters is also specified.

The model specification and parameters are then passed to and initialized within a 'mirai'.

```{r tensors2}
model <- nn_module(
  initialize = function(in_size, out_size) {
    self$conv1 <- nn_conv2d(in_size, out_size, 5)
    self$conv2 <- nn_conv2d(in_size, out_size, 5)
  },
  forward = function(x) {
    x <- self$conv1(x)
    x <- nnf_relu(x)
    x <- self$conv2(x)
    x <- nnf_relu(x)
    x
  }
)

params <- list(in_size = 1, out_size = 20)

m <- mirai(do.call(model, params), model = model, params = params)

m[]
```
The returned model is an object containing many tensor elements.
```{r tensors3}
m$data$parameters$conv1.weight
```
It is usual for model parameters to then be passed to an optimiser.

This can also be initialized within a 'mirai' process.
```{r tensors4}
optim <- mirai(optim_rmsprop(params = params), params = m$data$parameters)

optim[]

daemons(0)
```
Above, tensors and complex objects containing tensors were passed seamlessly between host and daemon processes, in the same way as any other R object.

The custom serialization in `mirai` leverages R's own native 'refhook' mechanism to allow such completely transparent usage. Designed to be fast and efficient, data copies are minimised and the 'official' serialization methods from the `torch` package are used directly.
