
# GovReport Summarization - 8192 tokens

- `ccdv/govreport-summarization` with the changes of:
  - data cleaned with the [clean-text python package](https://pypi.org/project/clean-text/)
  - total tokens for each column computed and added in new columns according to the `long-t5` tokenizer (_done **after** cleaning_)
---

## train info

```python
RangeIndex: 8200 entries, 0 to 8199
Data columns (total 4 columns):
 #   Column             Non-Null Count  Dtype 
---  ------             --------------  ----- 
 0   report             8200 non-null   string
 1   summary            8200 non-null   string
 2   input_token_len    8200 non-null   Int64 
 3   summary_token_len  8200 non-null   Int64 
dtypes: Int64(2), string(2)
memory usage: 272.4 KB
```

## token length distribution (long-t5)

![tokens](https://i.imgur.com/RS4fQLw.png)

---
task_categories:summarization size_categories:1K<n<10K source_datasets:ccdv/govreport-summarization language:en license:apache-2.0 govreport long document region:us
{"language": ["en"], "license": "apache-2.0", "size_categories": ["1K<n<10K"], "source_datasets": "ccdv/govreport-summarization", "task_categories": ["summarization"], "pretty_name": "GovReport Summarization - 8192 tokens", "tags": ["govreport", "long document"]}
TAGS
#task_categories-summarization #size_categories-1K<n<10K #source_datasets-ccdv/govreport-summarization #language-English #license-apache-2.0 #govreport #long document #region-us 


# GovReport Summarization - 8192 tokens

- 'ccdv/govreport-summarization' with the changes of:
  - data cleaned with the clean-text python package
  - total tokens for each column computed and added in new columns according to the 'long-t5' tokenizer (_done after cleaning_)
---

## train info



## token length distribution (long-t5)

!tokens

---
# GovReport Summarization - 8192 tokens

- 'ccdv/govreport-summarization' with the changes of:
  - data cleaned with the clean-text python package
  - total tokens for each column computed and added in new columns according to the 'long-t5' tokenizer (_done after cleaning_)
--- ## train info ## token length distribution (long-t5)

!tokens

---
TAGS
#task_categories-summarization #size_categories-1K<n<10K #source_datasets-ccdv/govreport-summarization #language-English #license-apache-2.0 #govreport #long document #region-us 
 # GovReport Summarization - 8192 tokens

- 'ccdv/govreport-summarization' with the changes of:
  - data cleaned with the clean-text python package
  - total tokens for each column computed and added in new columns according to the 'long-t5' tokenizer (_done after cleaning_)
--- ## train info ## token length distribution (long-t5)

!tokens

---
passage: TAGS
#task_categories-summarization #size_categories-1K<n<10K #source_datasets-ccdv/govreport-summarization #language-English #license-apache-2.0 #govreport #long document #region-us 
# GovReport Summarization - 8192 tokens

- 'ccdv/govreport-summarization' with the changes of:
  - data cleaned with the clean-text python package
  - total tokens for each column computed and added in new columns according to the 'long-t5' tokenizer (_done after cleaning_)
---## train info## token length distribution (long-t5)

!tokens

---