ROOTS Subset: roots_indic-ml_pib

# pib

- Dataset uid: `pib`

### Description

Sentence aligned parallel corpus between 11 Indian Languages, crawled and extracted from the press information bureau
website.


### Homepage

- https://huggingface.co/datasets/pib
- http://preon.iiit.ac.in/~jerin/bhasha/

### Licensing

Creative Commons Attribution-ShareAlike 4.0 International

### Speaker Locations



### Sizes

- 0.0609 % of total
- 0.6301 % of indic-hi
- 3.2610 % of indic-ur
- 0.6029 % of indic-ta
- 3.0834 % of indic-or
- 1.9757 % of indic-mr
- 0.2181 % of indic-bn
- 1.8901 % of indic-pa
- 1.5457 % of indic-gu
- 0.4695 % of indic-ml
- 0.5767 % of indic-te

### BigScience processing steps

#### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ur

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ta

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs

#### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300


language:ml license:cc-by-sa-4.0 region:us
{"language": "ml", "license": "cc-by-sa-4.0", "extra_gated_prompt": "By accessing this dataset, you agree to abide by the BigScience Ethical Charter. The charter can be found at:\nhttps://hf.co/spaces/bigscience/ethical-charter", "extra_gated_fields": {"I have read and agree to abide by the BigScience Ethical Charter": "checkbox"}}
TAGS
#language-Malayalam #license-cc-by-sa-4.0 #region-us 

ROOTS Subset: roots_indic-ml_pib

# pib

- Dataset uid: 'pib'

### Description

Sentence aligned parallel corpus between 11 Indian Languages, crawled and extracted from the press information bureau
website.


### Homepage

- URL
- URL

### Licensing

Creative Commons Attribution-ShareAlike 4.0 International

### Speaker Locations



### Sizes

- 0.0609 % of total
- 0.6301 % of indic-hi
- 3.2610 % of indic-ur
- 0.6029 % of indic-ta
- 3.0834 % of indic-or
- 1.9757 % of indic-mr
- 0.2181 % of indic-bn
- 1.8901 % of indic-pa
- 1.5457 % of indic-gu
- 0.4695 % of indic-ml
- 0.5767 % of indic-te

### BigScience processing steps

#### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ur

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ta

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs

#### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300


# pib

- Dataset uid: 'pib' ### Description

Sentence aligned parallel corpus between 11 Indian Languages, crawled and extracted from the press information bureau
website. ### Homepage

- URL
- URL ### Licensing

Creative Commons Attribution-ShareAlike 4.0 International ### Speaker Locations ### Sizes

- 0.0609 % of total
- 0.6301 % of indic-hi
- 3.2610 % of indic-ur
- 0.6029 % of indic-ta
- 3.0834 % of indic-or
- 1.9757 % of indic-mr
- 0.2181 % of indic-bn
- 1.8901 % of indic-pa
- 1.5457 % of indic-gu
- 0.4695 % of indic-ml
- 0.5767 % of indic-te ### BigScience processing steps #### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ur

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ta

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs #### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300
TAGS
#language-Malayalam #license-cc-by-sa-4.0 #region-us 
 # pib

- Dataset uid: 'pib' ### Description

Sentence aligned parallel corpus between 11 Indian Languages, crawled and extracted from the press information bureau
website. ### Homepage

- URL
- URL ### Licensing

Creative Commons Attribution-ShareAlike 4.0 International ### Speaker Locations ### Sizes

- 0.0609 % of total
- 0.6301 % of indic-hi
- 3.2610 % of indic-ur
- 0.6029 % of indic-ta
- 3.0834 % of indic-or
- 1.9757 % of indic-mr
- 0.2181 % of indic-bn
- 1.8901 % of indic-pa
- 1.5457 % of indic-gu
- 0.4695 % of indic-ml
- 0.5767 % of indic-te ### BigScience processing steps #### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ur

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ta

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs #### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300 #### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300