# Dataset Card for "marc-multilingual-encodings-v4"
# marc-code-mixed-small

These encodings are based on the review_tokens of [msislam/marc-code-mixed-small](https://huggingface.co/datasets/msislam/marc-code-mixed-small).

It contains German (DE), English (EN), Spanish (ES), and French (FR) languages. 

The labels are 0 (DE), 1 (EN), 2 (ES), and 3 (FR). 

Each review contains all four languages.

Total number of tokens:
* In training set: 10195342
* In test set: 842760
* In validation set: 842760

The encodings are created using the tokenizer [xlm-roberta-base](https://huggingface.co/xlm-roberta-base).
region:us
{"dataset_info": {"features": [{"name": "input_ids", "sequence": "int64"}, {"name": "attention_mask", "sequence": "int64"}, {"name": "labels", "sequence": "int64"}], "splits": [{"name": "train", "num_bytes": 245408208, "num_examples": 60000}, {"name": "test", "num_bytes": 20286240, "num_examples": 5000}, {"name": "validation", "num_bytes": 20286240, "num_examples": 5000}], "download_size": 27474371, "dataset_size": 285980688}}
TAGS
#region-us 

# Dataset Card for "marc-multilingual-encodings-v4"
# marc-code-mixed-small

These encodings are based on the review_tokens of msislam/marc-code-mixed-small.

It contains German (DE), English (EN), Spanish (ES), and French (FR) languages. 

The labels are 0 (DE), 1 (EN), 2 (ES), and 3 (FR). 

Each review contains all four languages.

Total number of tokens:
* In training set: 10195342
* In test set: 842760
* In validation set: 842760

The encodings are created using the tokenizer xlm-roberta-base.
# Dataset Card for "marc-multilingual-encodings-v4" # marc-code-mixed-small

These encodings are based on the review_tokens of msislam/marc-code-mixed-small.

It contains German (DE), English (EN), Spanish (ES), and French (FR) languages. 

The labels are 0 (DE), 1 (EN), 2 (ES), and 3 (FR). 

Each review contains all four languages.

Total number of tokens:
* In training set: 10195342
* In test set: 842760
* In validation set: 842760

The encodings are created using the tokenizer xlm-roberta-base.
TAGS
#region-us 
 # Dataset Card for "marc-multilingual-encodings-v4" # marc-code-mixed-small

These encodings are based on the review_tokens of msislam/marc-code-mixed-small.

It contains German (DE), English (EN), Spanish (ES), and French (FR) languages. 

The labels are 0 (DE), 1 (EN), 2 (ES), and 3 (FR). 

Each review contains all four languages.

Total number of tokens:
* In training set: 10195342
* In test set: 842760
* In validation set: 842760

The encodings are created using the tokenizer xlm-roberta-base.
passage: TAGS
#region-us 
# Dataset Card for "marc-multilingual-encodings-v4"# marc-code-mixed-small

These encodings are based on the review_tokens of msislam/marc-code-mixed-small.

It contains German (DE), English (EN), Spanish (ES), and French (FR) languages. 

The labels are 0 (DE), 1 (EN), 2 (ES), and 3 (FR). 

Each review contains all four languages.

Total number of tokens:
* In training set: 10195342
* In test set: 842760
* In validation set: 842760

The encodings are created using the tokenizer xlm-roberta-base.