region:us
---
dataset_info:
  features:
  - name: dataType
    dtype: string
  - name: title
    dtype: string
  - name: content
    dtype: string
  - name: uniqueKey
    dtype: string
  - name: titleUkey
    dtype: string
  - name: id
    dtype: int64
  splits:
  - name: train
    num_bytes: 58043923125
    num_examples: 16905023
  download_size: 25662051889
  dataset_size: 58043923125
duplicated_from: TigerResearch/pretrain_zh
---
# Dataset Card for "pretrain_zh"

[Tigerbot](https://github.com/TigerResearch/TigerBot) pretrain数据的中文部分。

包含(未压缩前) 中文书籍zh-books 12G, 中文互联网zh-webtext 25G, 中文百科zh-wiki 19G

更多语料请关注开源模型及持续更新 [https://github.com/TigerResearch/TigerBot](https://github.com/TigerResearch/TigerBot) 

<p align="center" width="40%">
</p>

## Usage
```python
import datasets

ds_sft = datasets.load_dataset('TigerResearch/pretrain_zh')
```