Cannot launch multi-gpu training?
I am trying to run training on multiple GPUs following this script here:github.comdatabricks/databricks-ml-examples/blob/master/llm-models/llamav2/llamav2-7b/06_fine_tune_qlora.py# Databricks notebook source
# MAGIC %md
# MAGIC # Fine tune llama-2-7b-hf with QLORA
# MAGIC
# MAGIC [Llama 2](https://huggingface.co/meta-llama) is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. It is trained with 2T tokens and supports context length window upto 4K tokens. [Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) is the 7B pretrained model, converted for the Hugging Face Transformers format.
# MAGIC
# MAGIC This is to fine-tune [llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) models on the [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) dataset.
# MAGIC
# MAGIC Environment for this notebook:
# MAGIC - Runtime: 13.2 GPU ML Runtime
# MAGIC - Instance: `g5.8xlarge` on AWS, `Standard_NV36ads_A10_v5` on Azure
# MAGIC
# MAGIC We leverage the PEFT library from Hugging Face, as well as QLoRA for more memory efficient finetuning.

# COMMAND ----------

from huggingface_hub import notebook_login

# Login to Huggingface to get access to the model
notebook_login()This file has been truncated.show originalsingle gpu is fine. When I switch to multi-gpu I got:RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!So I tried wrapping the dataloader in accelerate.prepare.then I get:AttributeError: 'DataLoaderShard' object has no attribute 'map'What is the correct path forward?