

# long-t5-tglobal-xl-16384-book-summary: 8-bit quantized version

<a href="https://colab.research.google.com/gist/pszemraj/c19e32baf876deb866c31cd46c86e893/long-t5-xl-accelerate-test.ipynb">
  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>

This is an 8-bit quantized version of the `pszemraj/long-t5-tglobal-xl-16384-book-summary` model, The model has been compressed using `bitsandbytes` and can be loaded with low memory usage.

Refer to the [original model](https://huggingface.co/pszemraj/long-t5-tglobal-xl-16384-book-summary) for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the `4.28.0` [release information](https://github.com/huggingface/transformers/releases/tag/v4.28.0) and the [example repository](https://huggingface.co/ybelkada/bloom-1b7-8bit).

- The total size of the model is only ~3.5 GB (vs original 12 GB)
- Enables low-RAM loading, making it easier to use in memory-limited environments like Colab
- Requires `bitsandbytes` - AFAIK at time of writing, only works on GPU


## Basic Usage

To use the model, install or upgrade `transformers`, `accelerate`, and `bitsandbytes`. Make sure to have `transformers>=4.28.0` and `bitsandbytes>0.37.2`.

```bash
pip install -U -q transformers bitsandbytes accelerate
```

Load the model with `AutoTokenizer` and `AutoModelForSeq2SeqLM`:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "pszemraj/long-t5-tglobal-xl-16384-book-summary-8bit"
tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
```

## More information about long-t5-tglobal-xl-16384-book-summary

- This is an 8-bit quantized version of `pszemraj/long-t5-tglobal-xl-16384-book-summary`.
  - It generalizes reasonably well to academic and narrative text.
  - The XL checkpoint typically generates summaries that are considerably better from a human evaluation perspective.
{"license": ["apache-2.0", "bsd-3-clause"], "tags": ["summarization", "summary", "booksum", "long-document", "long-form", "tglobal-xl", "XL", "8bit", "quantized"], "datasets": ["kmfoda/booksum"], "metrics": ["rouge"], "inference": false, "pipeline_tag": "summarization"}
transformers pytorch longt5 text2text-generation summarization summary booksum long-document long-form tglobal-xl XL 8bit quantized dataset:kmfoda/booksum license:apache-2.0 license:bsd-3-clause autotrain_compatible 8-bit region:us
TAGS
#transformers #pytorch #longt5 #text2text-generation #summarization #summary #booksum #long-document #long-form #tglobal-xl #XL #8bit #quantized #dataset-kmfoda/booksum #license-apache-2.0 #license-bsd-3-clause #autotrain_compatible #8-bit #region-us 



# long-t5-tglobal-xl-16384-book-summary: 8-bit quantized version

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This is an 8-bit quantized version of the 'pszemraj/long-t5-tglobal-xl-16384-book-summary' model, The model has been compressed using 'bitsandbytes' and can be loaded with low memory usage.

Refer to the original model for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the '4.28.0' release information and the example repository.

- The total size of the model is only ~3.5 GB (vs original 12 GB)
- Enables low-RAM loading, making it easier to use in memory-limited environments like Colab
- Requires 'bitsandbytes' - AFAIK at time of writing, only works on GPU


## Basic Usage

To use the model, install or upgrade 'transformers', 'accelerate', and 'bitsandbytes'. Make sure to have 'transformers>=4.28.0' and 'bitsandbytes>0.37.2'.



Load the model with 'AutoTokenizer' and 'AutoModelForSeq2SeqLM':



## More information about long-t5-tglobal-xl-16384-book-summary

- This is an 8-bit quantized version of 'pszemraj/long-t5-tglobal-xl-16384-book-summary'.
  - It generalizes reasonably well to academic and narrative text.
  - The XL checkpoint typically generates summaries that are considerably better from a human evaluation perspective.
# long-t5-tglobal-xl-16384-book-summary: 8-bit quantized version

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This is an 8-bit quantized version of the 'pszemraj/long-t5-tglobal-xl-16384-book-summary' model, The model has been compressed using 'bitsandbytes' and can be loaded with low memory usage.

Refer to the original model for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the '4.28.0' release information and the example repository.

- The total size of the model is only ~3.5 GB (vs original 12 GB)
- Enables low-RAM loading, making it easier to use in memory-limited environments like Colab
- Requires 'bitsandbytes' - AFAIK at time of writing, only works on GPU ## Basic Usage

To use the model, install or upgrade 'transformers', 'accelerate', and 'bitsandbytes'. Make sure to have 'transformers>=4.28.0' and 'bitsandbytes>0.37.2'.



Load the model with 'AutoTokenizer' and 'AutoModelForSeq2SeqLM': ## More information about long-t5-tglobal-xl-16384-book-summary

- This is an 8-bit quantized version of 'pszemraj/long-t5-tglobal-xl-16384-book-summary'.
  - It generalizes reasonably well to academic and narrative text.
  - The XL checkpoint typically generates summaries that are considerably better from a human evaluation perspective.
TAGS
#transformers #pytorch #longt5 #text2text-generation #summarization #summary #booksum #long-document #long-form #tglobal-xl #XL #8bit #quantized #dataset-kmfoda/booksum #license-apache-2.0 #license-bsd-3-clause #autotrain_compatible #8-bit #region-us 
 # long-t5-tglobal-xl-16384-book-summary: 8-bit quantized version

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This is an 8-bit quantized version of the 'pszemraj/long-t5-tglobal-xl-16384-book-summary' model, The model has been compressed using 'bitsandbytes' and can be loaded with low memory usage.

Refer to the original model for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the '4.28.0' release information and the example repository.

- The total size of the model is only ~3.5 GB (vs original 12 GB)
- Enables low-RAM loading, making it easier to use in memory-limited environments like Colab
- Requires 'bitsandbytes' - AFAIK at time of writing, only works on GPU ## Basic Usage

To use the model, install or upgrade 'transformers', 'accelerate', and 'bitsandbytes'. Make sure to have 'transformers>=4.28.0' and 'bitsandbytes>0.37.2'.



Load the model with 'AutoTokenizer' and 'AutoModelForSeq2SeqLM': ## More information about long-t5-tglobal-xl-16384-book-summary

- This is an 8-bit quantized version of 'pszemraj/long-t5-tglobal-xl-16384-book-summary'.
  - It generalizes reasonably well to academic and narrative text.
  - The XL checkpoint typically generates summaries that are considerably better from a human evaluation perspective.
TAGS
#transformers #pytorch #longt5 #text2text-generation #summarization #summary #booksum #long-document #long-form #tglobal-xl #XL #8bit #quantized #dataset-kmfoda/booksum #license-apache-2.0 #license-bsd-3-clause #autotrain_compatible #8-bit #region-us 
# long-t5-tglobal-xl-16384-book-summary: 8-bit quantized version

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This is an 8-bit quantized version of the 'pszemraj/long-t5-tglobal-xl-16384-book-summary' model, The model has been compressed using 'bitsandbytes' and can be loaded with low memory usage.

Refer to the original model for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the '4.28.0' release information and the example repository.

- The total size of the model is only ~3.5 GB (vs original 12 GB)
- Enables low-RAM loading, making it easier to use in memory-limited environments like Colab
- Requires 'bitsandbytes' - AFAIK at time of writing, only works on GPU## Basic Usage

To use the model, install or upgrade 'transformers', 'accelerate', and 'bitsandbytes'. Make sure to have 'transformers>=4.28.0' and 'bitsandbytes>0.37.2'.



Load the model with 'AutoTokenizer' and 'AutoModelForSeq2SeqLM':## More information about long-t5-tglobal-xl-16384-book-summary

- This is an 8-bit quantized version of 'pszemraj/long-t5-tglobal-xl-16384-book-summary'.
  - It generalizes reasonably well to academic and narrative text.
  - The XL checkpoint typically generates summaries that are considerably better from a human evaluation perspective.