
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# iva_mt_wslot-m2m100_418M-en-sv

This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the iva_mt_wslot dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0107
- Bleu: 71.0808
- Gen Len: 19.7647

## Model description

More information needed

## How to use

First please make sure to install `pip install transformers`. First download model: 

```python
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
import torch

def translate(input_text, lang):
    input_ids = tokenizer(input_text, return_tensors="pt")
    generated_tokens = model.generate(**input_ids, forced_bos_token_id=tokenizer.get_lang_id(lang))
    return tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

model_name = "cartesinus/iva_mt_wslot-m2m100_418M-0.1.0-en-sv"
tokenizer = M2M100Tokenizer.from_pretrained(model_name, src_lang="en", tgt_lang="sv")
model = M2M100ForConditionalGeneration.from_pretrained(model_name)
```

Then you can translate either plain text like this:
```python
print(translate("set the temperature on my thermostat", "sv"))
```
or you can translate with slot annotations that will be restored in tgt language:
```python
print(translate("wake me up at <a>nine am<a> on <b>friday<b>", "sv"))
```
Limitations of translation with slot transfer:
1) Annotated words must be placed between semi-xml tags like this "this is \<a\>example\<a\>"
2) There is no closing tag for example "\<\a\>" in above example - this is done on purpose to omit problems with backslash escape
3) If the sentence consists of more than one slot then simply use the next alphabet letter. For example "this is \<a\>example\<a\> with more than \<b\>one\<b\> slot"
4) Please do not add space before the first or last annotated word because this particular model was trained this way and it most probably will lower its results

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 7
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| 0.0151        | 1.0   | 1885  | 0.0120          | 67.2332 | 19.3956 |
| 0.0095        | 2.0   | 3770  | 0.0105          | 69.8147 | 19.675  |
| 0.0065        | 3.0   | 5655  | 0.0104          | 70.239  | 19.8404 |
| 0.0049        | 4.0   | 7540  | 0.0104          | 70.3673 | 19.7154 |
| 0.0038        | 5.0   | 9425  | 0.0105          | 70.1632 | 19.7743 |
| 0.0026        | 6.0   | 11310 | 0.0105          | 70.7959 | 19.7809 |
| 0.0021        | 7.0   | 13195 | 0.0107          | 71.0808 | 19.7647 |


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3

## Citation

If you use this model, please cite the following:
```
@article{Sowanski2023SlotLI,
  title={Slot Lost in Translation? Not Anymore: A Machine Translation Model for Virtual Assistants with Type-Independent Slot Transfer},
  author={Marcin Sowanski and Artur Janicki},
  journal={2023 30th International Conference on Systems, Signals and Image Processing (IWSSIP)},
  year={2023},
  pages={1-5}
}
```
{"language": ["en", "sv"], "license": "mit", "tags": ["generated_from_trainer"], "datasets": ["cartesinus/iva_mt_wslot"], "metrics": ["bleu"], "pipeline_tag": "translation", "model-index": [{"name": "iva_mt_wslot-m2m100_418M-en-sv", "results": [{"task": {"type": "text2text-generation", "name": "Sequence-to-sequence Language Modeling"}, "dataset": {"name": "iva_mt_wslot", "type": "iva_mt_wslot", "config": "en-sv", "split": "validation", "args": "en-sv"}, "metrics": [{"type": "bleu", "value": 71.0808, "name": "Bleu"}]}]}]}
transformers pytorch tensorboard safetensors m2m_100 text2text-generation generated_from_trainer translation en sv dataset:cartesinus/iva_mt_wslot doi:10.57967/hf/1049 license:mit model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #safetensors #m2m_100 #text2text-generation #generated_from_trainer #translation #en #sv #dataset-cartesinus/iva_mt_wslot #doi-10.57967/hf/1049 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 

iva\_mt\_wslot-m2m100\_418M-en-sv
=================================


This model is a fine-tuned version of facebook/m2m100\_418M on the iva\_mt\_wslot dataset.
It achieves the following results on the evaluation set:


* Loss: 0.0107
* Bleu: 71.0808
* Gen Len: 19.7647


Model description
-----------------


More information needed


How to use
----------


First please make sure to install 'pip install transformers'. First download model:


Then you can translate either plain text like this:


or you can translate with slot annotations that will be restored in tgt language:


Limitations of translation with slot transfer:


1. Annotated words must be placed between semi-xml tags like this "this is <a>example<a>"
2. There is no closing tag for example "<\a>" in above example - this is done on purpose to omit problems with backslash escape
3. If the sentence consists of more than one slot then simply use the next alphabet letter. For example "this is <a>example<a> with more than <b>one<b> slot"
4. Please do not add space before the first or last annotated word because this particular model was trained this way and it most probably will lower its results


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7
* mixed\_precision\_training: Native AMP


### Training results



### Framework versions


* Transformers 4.28.1
* Pytorch 2.0.0+cu118
* Datasets 2.11.0
* Tokenizers 0.13.3


If you use this model, please cite the following:



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.1
* Pytorch 2.0.0+cu118
* Datasets 2.11.0
* Tokenizers 0.13.3


If you use this model, please cite the following:
TAGS
#transformers #pytorch #tensorboard #safetensors #m2m_100 #text2text-generation #generated_from_trainer #translation #en #sv #dataset-cartesinus/iva_mt_wslot #doi-10.57967/hf/1049 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.1
* Pytorch 2.0.0+cu118
* Datasets 2.11.0
* Tokenizers 0.13.3


If you use this model, please cite the following:
TAGS
#transformers #pytorch #tensorboard #safetensors #m2m_100 #text2text-generation #generated_from_trainer #translation #en #sv #dataset-cartesinus/iva_mt_wslot #doi-10.57967/hf/1049 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7
* mixed\_precision\_training: Native AMP### Training results### Framework versions


* Transformers 4.28.1
* Pytorch 2.0.0+cu118
* Datasets 2.11.0
* Tokenizers 0.13.3


If you use this model, please cite the following: