# zh-tw-pythia-ta8000-v1-it1-sg-001

This dataset is a part of the `zh-tw-llm` project.

* Tokenizer: `zh-tw-pythia-tokenizer-a8000-v1`
* Built with: `sharegpt`
* Rows: `train` `8150`, `test` `84`
* Max length: `2048`
* Full config:
  ```json
  {"build_with": ["sharegpt"], "preview_length": 512, "sharegpt_settings": {"source_dataset": "zetavg/ShareGPT-Processed", "train_on_inputs": false, "languages": [{"en": 0.3}, {"zh": 0.2}, "zh_Hant"], "rows_limit": 10000, "test_size": 0.01, "test_split_seed": 42, "test_rows_limit": 100}}
  ```
region:us
{"dataset_info": {"dataset_size": 125318970.0, "download_size": 36093839, "features": [{"name": "input_ids", "sequence": "int32"}, {"name": "attention_mask", "sequence": "int8"}, {"name": "labels", "sequence": "int64"}, {"dtype": "string", "name": "preview"}], "splits": [{"name": "train", "num_bytes": 123951700.0, "num_examples": 8150}, {"name": "test", "num_bytes": 1367270.0, "num_examples": 84}]}}
TAGS
#region-us 

# zh-tw-pythia-ta8000-v1-it1-sg-001

This dataset is a part of the 'zh-tw-llm' project.

* Tokenizer: 'zh-tw-pythia-tokenizer-a8000-v1'
* Built with: 'sharegpt'
* Rows: 'train' '8150', 'test' '84'
* Max length: '2048'
* Full config:
  
# zh-tw-pythia-ta8000-v1-it1-sg-001

This dataset is a part of the 'zh-tw-llm' project.

* Tokenizer: 'zh-tw-pythia-tokenizer-a8000-v1'
* Built with: 'sharegpt'
* Rows: 'train' '8150', 'test' '84'
* Max length: '2048'
* Full config:
TAGS
#region-us 
 # zh-tw-pythia-ta8000-v1-it1-sg-001

This dataset is a part of the 'zh-tw-llm' project.

* Tokenizer: 'zh-tw-pythia-tokenizer-a8000-v1'
* Built with: 'sharegpt'
* Rows: 'train' '8150', 'test' '84'
* Max length: '2048'
* Full config:
passage: TAGS
#region-us 
# zh-tw-pythia-ta8000-v1-it1-sg-001

This dataset is a part of the 'zh-tw-llm' project.

* Tokenizer: 'zh-tw-pythia-tokenizer-a8000-v1'
* Built with: 'sharegpt'
* Rows: 'train' '8150', 'test' '84'
* Max length: '2048'
* Full config: