# OpenSubtitles

- Source: https://huggingface.co/datasets/open_subtitles
- Num examples: 3,505,276
- Language: English

```python
from datasets import load_dataset

load_dataset("tdtunlp/open_subtitles_envi")
```

- Format for Translation task

```python
def preprocess(
    sample,
    instruction_key="### Instruction:",
    input_key="Input:",
    response_key="<|endofprompt|>",
    end_key="<|endoftext|>",
    en2vi=True,
):
    if en2vi:
        if random.random() < 0.5:
            instruction = "Translate the following sentences from English into Vietnamese."
        else:
            instruction = "Dịch các câu sau từ tiếng Anh sang tiếng Việt."
        
        input = sample['en'].strip()
        response = sample['vi'].strip()
    else:
        if random.random() < 0.5:
            instruction = "Translate the following sentences from Vietnamese into English."
        else:
            instruction = "Dịch các câu sau từ tiếng Việt sang tiếng Anh."
        
        input = sample['vi'].strip()
        response = sample['en'].strip()

    return {'text': """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
{instruction_key}
{instruction}
{input_key} 
{input}
{response_key}
{response}
{end_key}""".format(
    instruction_key=instruction_key,
    instruction=instruction,
    input_key=input_key,
    input=input,
    response_key=response_key,
    response=response,
    end_key=end_key,
)}

"""
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
Dịch các câu sau từ tiếng Anh sang tiếng Việt.
Input: 
Line up, I say!
<|endofprompt|>
Sắp hàng, nghe chưa!
<|endoftext|>
"""
```
task_categories:translation size_categories:1M<n<10M language:en language:vi LM region:us
{"language": ["en", "vi"], "size_categories": ["1M<n<10M"], "task_categories": ["translation"], "dataset_info": {"features": [{"name": "en", "dtype": "string"}, {"name": "vi", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 280063489, "num_examples": 3505276}], "download_size": 176803145, "dataset_size": 280063489}, "tags": ["LM"]}
TAGS
#task_categories-translation #size_categories-1M<n<10M #language-English #language-Vietnamese #LM #region-us 

# OpenSubtitles

- Source: URL
- Num examples: 3,505,276
- Language: English



- Format for Translation task


# OpenSubtitles

- Source: URL
- Num examples: 3,505,276
- Language: English



- Format for Translation task
TAGS
#task_categories-translation #size_categories-1M<n<10M #language-English #language-Vietnamese #LM #region-us 
 # OpenSubtitles

- Source: URL
- Num examples: 3,505,276
- Language: English



- Format for Translation task
passage: TAGS
#task_categories-translation #size_categories-1M<n<10M #language-English #language-Vietnamese #LM #region-us 
# OpenSubtitles

- Source: URL
- Num examples: 3,505,276
- Language: English



- Format for Translation task