

# flan-t5-large-stacked-XSUM-1024

<a href="https://colab.research.google.com/gist/pszemraj/561263b04b33d5aec04a18f572d68011/brief-demo-flan-t5-stacked-xsum.ipynb">
  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>

This model is a fine-tuned version of [google/flan-t5-large](https://huggingface.co/google/flan-t5-large) on the stacked-summaries/stacked-xsum-1024 dataset.

It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119

> Note that the evaluation set is `stacked-summaries/stacked-xsum-1024` and not `xsum` itself
## Model description

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is `[NEXT_CONCEPT]`. You can split an output summary based on this token to see how it split the input text information: `summary_text.split("[NEXT_CONCEPT]")` etc.


## Intended uses & limitations

- max input length (in tokens): 1024

## Training and evaluation data

Refer to `stacked-summaries/stacked-xsum-1024`

Trained for approx 3 epochs before ROUGE scores stabilized on most recent run:


### scores

![stable-scores](https://i.imgur.com/4tvhHVy.png)


### gradients

![gradients](https://i.imgur.com/V6zcmAb.png)
{"language": ["en"], "license": "apache-2.0", "library_name": "transformers", "tags": ["generated_from_trainer", "stacked summaries", "xsum"], "datasets": ["stacked-summaries/stacked-xsum-1024"], "pipeline_tag": "summarization", "model-index": [{"name": "flan-t5-large-stacked-XSUM-1024-WIP-2p8-850-stacked-xsum-1024-evaluated", "results": [{"task": {"type": "summarization", "name": "Summarization"}, "dataset": {"name": "xsum", "type": "xsum", "config": "default", "split": "test"}, "metrics": [{"type": "rouge", "value": 39.3614, "name": "ROUGE-1", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZmZDNhNWM5YjcyMzVjNjUwMWE1NDg4YmRiNGMwY2EyZDYzMGZkY2NlNWE0MzQwNDYzN2JkNzYyOGUxNmI3ZiIsInZlcnNpb24iOjF9.1ucBm8VOqZgLXmUyDkPisiFfHJ8VYvOdvUsk6R_F0QGLIBXOCf2s_pbqHauTyEQM2mAn762DpR5L4AZg7hF_BA"}, {"type": "rouge", "value": 17.5887, "name": "ROUGE-2", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDU3MDQwNjYzMTE2MjU5NTE0ODU1ZmI2ZjhlY2QxODA3YTYyOWExZDdiM2Y4YzZhMTU3N2IwMGQ4M2MxMTNmZiIsInZlcnNpb24iOjF9.lb6R_xg5R1TABUCSRgvEGmdkxhSRavrfllxhsk_NxKA53EC4MXeE6o7nRWPoo2nrBOb5Lcajy_5y4oPOkv84Ag"}, {"type": "rouge", "value": 32.6489, "name": "ROUGE-L", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmFkOTc2MTIxMmYyNTY2MWE3Y2E4ZWYwODQ5MmU3NTIxZWM2Yzg2ZDNkYjE3NDgzM2VjYTMwOTkxNjQ1YmIyYiIsInZlcnNpb24iOjF9.AAAh5SnRDnTMCEXMfEp9N7pwHITv-crNloZTnbW7TMPXtMUe7vzATOxGVMZpMe-Nsf3Wkc3JbUdaZZ9bOb17Ag"}, {"type": "rouge", "value": 32.6435, "name": "ROUGE-LSUM", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjg1ZmNkODZlMzdkODA4MDUxMGQyNjFiMTkyYjIzMTE2NGMyOWQ1NmQ2YjY0OTRmZjVjZWNhODBiOWI1YzVlOCIsInZlcnNpb24iOjF9.GUVl2J3DCRQUqueSuCsFM8v7IDXH7EATFlQbFl730Bo8Y2aolA-V9uN7pkaU9IM1wWBz7hvILElBCE0sln6SAQ"}, {"type": "loss", "value": 1.4964560270309448, "name": "loss", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTViZTkwMzQ3MGNlZDJhNTk3NDE5NzBkMDZjMGEyNzNkZTI4ZmJhMWRlYTMwNmRmN2JhNzdkNTQ3N2FlODBmNyIsInZlcnNpb24iOjF9.lNWUw12R20SwZMZEuUnxYsWrkFBNoU9_5ZOiuFF5aT9QsHJC-FSmZ8DXTdVudv6J-BoeA-l5KYowr7GJfbzlDQ"}, {"type": "gen_len", "value": 18.7302, "name": "gen_len", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWM2MWQzN2YyY2U0NWZhZGFkMjk0MzFlMTA1YTgxYzAzYjhhZmVmZDI5Mzk4ODgzOGU1NjVhNTk3NmYyNzhkMSIsInZlcnNpb24iOjF9.bL56u1G46OIwdIqZJ-6og_T2yCKFTXrlPQeguZps3ixXokfKqlfCDxz3641yKA3AdMlLe5lDcN3UQReHtiWwBg"}]}]}]}
transformers pytorch safetensors t5 text2text-generation generated_from_trainer stacked summaries xsum summarization en dataset:stacked-summaries/stacked-xsum-1024 license:apache-2.0 model-index autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #generated_from_trainer #stacked summaries #xsum #summarization #en #dataset-stacked-summaries/stacked-xsum-1024 #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 



# flan-t5-large-stacked-XSUM-1024

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This model is a fine-tuned version of google/flan-t5-large on the stacked-summaries/stacked-xsum-1024 dataset.

It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119

> Note that the evaluation set is 'stacked-summaries/stacked-xsum-1024' and not 'xsum' itself
## Model description

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is '[NEXT_CONCEPT]'. You can split an output summary based on this token to see how it split the input text information: 'summary_text.split("[NEXT_CONCEPT]")' etc.


## Intended uses & limitations

- max input length (in tokens): 1024

## Training and evaluation data

Refer to 'stacked-summaries/stacked-xsum-1024'

Trained for approx 3 epochs before ROUGE scores stabilized on most recent run:


### scores

!stable-scores


### gradients

!gradients
# flan-t5-large-stacked-XSUM-1024

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This model is a fine-tuned version of google/flan-t5-large on the stacked-summaries/stacked-xsum-1024 dataset.

It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119

> Note that the evaluation set is 'stacked-summaries/stacked-xsum-1024' and not 'xsum' itself ## Model description

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is '[NEXT_CONCEPT]'. You can split an output summary based on this token to see how it split the input text information: 'summary_text.split("[NEXT_CONCEPT]")' etc. ## Intended uses & limitations

- max input length (in tokens): 1024 ## Training and evaluation data

Refer to 'stacked-summaries/stacked-xsum-1024'

Trained for approx 3 epochs before ROUGE scores stabilized on most recent run: ### scores

!stable-scores ### gradients

!gradients
TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #generated_from_trainer #stacked summaries #xsum #summarization #en #dataset-stacked-summaries/stacked-xsum-1024 #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # flan-t5-large-stacked-XSUM-1024

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This model is a fine-tuned version of google/flan-t5-large on the stacked-summaries/stacked-xsum-1024 dataset.

It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119

> Note that the evaluation set is 'stacked-summaries/stacked-xsum-1024' and not 'xsum' itself ## Model description

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is '[NEXT_CONCEPT]'. You can split an output summary based on this token to see how it split the input text information: 'summary_text.split("[NEXT_CONCEPT]")' etc. ## Intended uses & limitations

- max input length (in tokens): 1024 ## Training and evaluation data

Refer to 'stacked-summaries/stacked-xsum-1024'

Trained for approx 3 epochs before ROUGE scores stabilized on most recent run: ### scores

!stable-scores ### gradients

!gradients
TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #generated_from_trainer #stacked summaries #xsum #summarization #en #dataset-stacked-summaries/stacked-xsum-1024 #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# flan-t5-large-stacked-XSUM-1024

<a href="URL
  <img src="URL alt="Open In Colab"/>
</a>

This model is a fine-tuned version of google/flan-t5-large on the stacked-summaries/stacked-xsum-1024 dataset.

It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119

> Note that the evaluation set is 'stacked-summaries/stacked-xsum-1024' and not 'xsum' itself## Model description

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is '[NEXT_CONCEPT]'. You can split an output summary based on this token to see how it split the input text information: 'summary_text.split("[NEXT_CONCEPT]")' etc.## Intended uses & limitations

- max input length (in tokens): 1024## Training and evaluation data

Refer to 'stacked-summaries/stacked-xsum-1024'

Trained for approx 3 epochs before ROUGE scores stabilized on most recent run:### scores

!stable-scores### gradients

!gradients