
# Model Card of `lmqg/mbart-large-cc25-dequad-qg-ae`
This model is fine-tuned version of [facebook/mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25) for question generation and answer extraction jointly on the [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).


### Overview
- **Language model:** [facebook/mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25)   
- **Language:** de  
- **Training data:** [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) (default)
- **Online Demo:** [https://autoqg.net/](https://autoqg.net/)
- **Repository:** [https://github.com/asahi417/lm-question-generation](https://github.com/asahi417/lm-question-generation)
- **Paper:** [https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)

### Usage
- With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
```python
from lmqg import TransformersQG

# initialize model
model = TransformersQG(language="de", model="lmqg/mbart-large-cc25-dequad-qg-ae")

# model prediction
question_answer_pairs = model.generate_qa("das erste weltweit errichtete Hermann Brehmer 1855 im niederschlesischen ''Görbersdorf'' (heute Sokołowsko, Polen).")

```

- With `transformers`
```python
from transformers import pipeline

pipe = pipeline("text2text-generation", "lmqg/mbart-large-cc25-dequad-qg-ae")

# answer extraction
answer = pipe("generate question: Empfangs- und Sendeantenne sollen in ihrer Polarisation übereinstimmen, andernfalls <hl> wird die Signalübertragung stark gedämpft. <hl>")

# question generation
question = pipe("extract answers: Sommerzeit <hl> Frühling <hl>: Umstellung von Normalzeit auf Sommerzeit – die Uhr wird um eine Stunde ''vor''gestellt. Herbst: Umstellung von Sommerzeit auf Normalzeit – die Uhr wird um eine Stunde ''zurück''gestellt. Als Sommerzeit wird die gegenüber der Zonenzeit meist um eine Stunde vorgestellte Uhrzeit bezeichnet, die während eines bestimmten Zeitraums im Sommerhalbjahr (und oft auch etwas darüber hinaus) als gesetzliche Zeit dient. Eine solche Regelung wird fast nur in Ländern der gemäßigten Zonen angewandt. Die mitteleuropäische Sommerzeit beginnt am letzten Sonntag im März um 2:00 Uhr MEZ, indem die Stundenzählung um eine Stunde von 2:00 Uhr auf 3:00 Uhr vorgestellt wird. Sie endet jeweils am letzten Sonntag im Oktober um 3:00 Uhr MESZ, indem die Stundenzählung um eine Stunde von 3:00 Uhr auf 2:00 Uhr zurückgestellt wird.")

```

## Evaluation


- ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/mbart-large-cc25-dequad-qg-ae/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_dequad.default.json) 

|            |   Score | Type    | Dataset                                                          |
|:-----------|--------:|:--------|:-----------------------------------------------------------------|
| BERTScore  |   80.57 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_1     |   11.17 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_2     |    4.71 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_3     |    1.96 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_4     |    0.78 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| METEOR     |   15.43 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| MoverScore |   56.4  | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| ROUGE_L    |   12.36 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |


- ***Metric (Question & Answer Generation)***:  [raw metric file](https://huggingface.co/lmqg/mbart-large-cc25-dequad-qg-ae/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_dequad.default.json)

|                                 |   Score | Type    | Dataset                                                          |
|:--------------------------------|--------:|:--------|:-----------------------------------------------------------------|
| QAAlignedF1Score (BERTScore)    |   82.49 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| QAAlignedF1Score (MoverScore)   |   54.84 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| QAAlignedPrecision (BERTScore)  |   81.39 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| QAAlignedPrecision (MoverScore) |   54.58 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| QAAlignedRecall (BERTScore)     |   83.67 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| QAAlignedRecall (MoverScore)    |   55.13 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |


- ***Metric (Answer Extraction)***: [raw metric file](https://huggingface.co/lmqg/mbart-large-cc25-dequad-qg-ae/raw/main/eval/metric.first.answer.paragraph_sentence.answer.lmqg_qg_dequad.default.json)

|                  |   Score | Type    | Dataset                                                          |
|:-----------------|--------:|:--------|:-----------------------------------------------------------------|
| AnswerExactMatch |   22.69 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| AnswerF1Score    |   48.09 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| BERTScore        |   78.8  | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_1           |   21.99 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_2           |   14.92 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_3           |   10.06 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| Bleu_4           |    6.86 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| METEOR           |   25.56 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| MoverScore       |   63.5  | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |
| ROUGE_L          |   20.84 | default | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) |



## Training hyperparameters

The following hyperparameters were used during fine-tuning:
 - dataset_path: lmqg/qg_dequad
 - dataset_name: default
 - input_types: ['paragraph_answer', 'paragraph_sentence']
 - output_types: ['question', 'answer']
 - prefix_types: ['qg', 'ae']
 - model: facebook/mbart-large-cc25
 - max_length: 512
 - max_length_output: 32
 - epoch: 11
 - batch: 2
 - lr: 0.0001
 - fp16: False
 - random_seed: 1
 - gradient_accumulation_steps: 32
 - label_smoothing: 0.15

The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/mbart-large-cc25-dequad-qg-ae/raw/main/trainer_config.json).

## Citation
```
@inproceedings{ushio-etal-2022-generative,
    title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
    author = "Ushio, Asahi  and
        Alva-Manchego, Fernando  and
        Camacho-Collados, Jose",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, U.A.E.",
    publisher = "Association for Computational Linguistics",
}

```

{"language": "de", "license": "cc-by-4.0", "tags": ["question generation", "answer extraction"], "datasets": ["lmqg/qg_dequad"], "metrics": ["bleu4", "meteor", "rouge-l", "bertscore", "moverscore"], "pipeline_tag": "text2text-generation", "widget": [{"text": "generate question: Empfangs- und Sendeantenne sollen in ihrer Polarisation \u00fcbereinstimmen, andernfalls <hl> wird die Signal\u00fcbertragung stark ged\u00e4mpft. <hl>", "example_title": "Question Generation Example 1"}, {"text": "generate question: das erste weltweit errichtete Hermann Brehmer <hl> 1855 <hl> im niederschlesischen ''G\u00f6rbersdorf'' (heute Soko\u0142owsko, Polen).", "example_title": "Question Generation Example 2"}, {"text": "generate question: Er muss Zyperngrieche sein und wird direkt f\u00fcr <hl> f\u00fcnf Jahre <hl> gew\u00e4hlt (Art. 43 Abs. 1 der Verfassung) und verf\u00fcgt \u00fcber weitreichende Exekutivkompetenzen.", "example_title": "Question Generation Example 3"}, {"text": "extract answers: Sommerzeit <hl> Fr\u00fchling <hl>: Umstellung von Normalzeit auf Sommerzeit \u2013 die Uhr wird um eine Stunde ''vor''gestellt. Herbst: Umstellung von Sommerzeit auf Normalzeit \u2013 die Uhr wird um eine Stunde ''zur\u00fcck''gestellt. Als Sommerzeit wird die gegen\u00fcber der Zonenzeit meist um eine Stunde vorgestellte Uhrzeit bezeichnet, die w\u00e4hrend eines bestimmten Zeitraums im Sommerhalbjahr (und oft auch etwas dar\u00fcber hinaus) als gesetzliche Zeit dient. Eine solche Regelung wird fast nur in L\u00e4ndern der gem\u00e4\u00dfigten Zonen angewandt. Die mitteleurop\u00e4ische Sommerzeit beginnt am letzten Sonntag im M\u00e4rz um 2:00 Uhr MEZ, indem die Stundenz\u00e4hlung um eine Stunde von 2:00 Uhr auf 3:00 Uhr vorgestellt wird. Sie endet jeweils am letzten Sonntag im Oktober um 3:00 Uhr MESZ, indem die Stundenz\u00e4hlung um eine Stunde von 3:00 Uhr auf 2:00 Uhr zur\u00fcckgestellt wird.", "example_title": "Answer Extraction Example 1"}, {"text": "extract answers: Iran === Landwirtschaft === Die landwirtschaftliche Nutzfl\u00e4che betr\u00e4gt trotz zahlreicher Gebirge und W\u00fcsten 10 % der Landesfl\u00e4che, wobei ein Drittel k\u00fcnstlich bew\u00e4ssert wird. Die Landwirtschaft ist einer der gr\u00f6\u00dften Arbeitgeber des Landes. Wichtige Produkte sind Pistazien, Weizen, Reis, Zucker, Baumwolle, Fr\u00fcchte, N\u00fcsse, Datteln, Wolle und Kaviar. Seit der Revolution von 1979 wurde der Anbau von Weintrauben wegen des islamischen Alkoholverbots auf den 200.000 Hektar Rebfl\u00e4che fast vollst\u00e4ndig auf Tafeltrauben und Rosinen umgestellt. Bei Rosinen ist <hl> der Iran <hl> inzwischen nach der T\u00fcrkei der zweitgr\u00f6\u00dfte Exporteur der Welt, bei Safran mit ungef\u00e4hr 90 % Marktanteil des globalen Bedarfs mit Abstand der gr\u00f6\u00dfte.", "example_title": "Answer Extraction Example 2"}], "model-index": [{"name": "lmqg/mbart-large-cc25-dequad-qg-ae", "results": [{"task": {"type": "text2text-generation", "name": "Text2text Generation"}, "dataset": {"name": "lmqg/qg_dequad", "type": "default", "args": "default"}, "metrics": [{"type": "bleu4_question_generation", "value": 0.78, "name": "BLEU4 (Question Generation)"}, {"type": "rouge_l_question_generation", "value": 12.36, "name": "ROUGE-L (Question Generation)"}, {"type": "meteor_question_generation", "value": 15.43, "name": "METEOR (Question Generation)"}, {"type": "bertscore_question_generation", "value": 80.57, "name": "BERTScore (Question Generation)"}, {"type": "moverscore_question_generation", "value": 56.4, "name": "MoverScore (Question Generation)"}, {"type": "qa_aligned_f1_score_bertscore_question_answer_generation_with_gold_answer", "value": 82.49, "name": "QAAlignedF1Score-BERTScore (Question & Answer Generation (with Gold Answer))"}, {"type": "qa_aligned_recall_bertscore_question_answer_generation_with_gold_answer", "value": 83.67, "name": "QAAlignedRecall-BERTScore (Question & Answer Generation (with Gold Answer))"}, {"type": "qa_aligned_precision_bertscore_question_answer_generation_with_gold_answer", "value": 81.39, "name": "QAAlignedPrecision-BERTScore (Question & Answer Generation (with Gold Answer))"}, {"type": "qa_aligned_f1_score_moverscore_question_answer_generation_with_gold_answer", "value": 54.84, "name": "QAAlignedF1Score-MoverScore (Question & Answer Generation (with Gold Answer))"}, {"type": "qa_aligned_recall_moverscore_question_answer_generation_with_gold_answer", "value": 55.13, "name": "QAAlignedRecall-MoverScore (Question & Answer Generation (with Gold Answer))"}, {"type": "qa_aligned_precision_moverscore_question_answer_generation_with_gold_answer", "value": 54.58, "name": "QAAlignedPrecision-MoverScore (Question & Answer Generation (with Gold Answer))"}, {"type": "bleu4_answer_extraction", "value": 6.86, "name": "BLEU4 (Answer Extraction)"}, {"type": "rouge_l_answer_extraction", "value": 20.84, "name": "ROUGE-L (Answer Extraction)"}, {"type": "meteor_answer_extraction", "value": 25.56, "name": "METEOR (Answer Extraction)"}, {"type": "bertscore_answer_extraction", "value": 78.8, "name": "BERTScore (Answer Extraction)"}, {"type": "moverscore_answer_extraction", "value": 63.5, "name": "MoverScore (Answer Extraction)"}, {"type": "answer_f1_score__answer_extraction", "value": 48.09, "name": "AnswerF1Score (Answer Extraction)"}, {"type": "answer_exact_match_answer_extraction", "value": 22.69, "name": "AnswerExactMatch (Answer Extraction)"}]}]}]}
transformers pytorch mbart text2text-generation question generation answer extraction de dataset:lmqg/qg_dequad arxiv:2210.03992 license:cc-by-4.0 model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #mbart #text2text-generation #question generation #answer extraction #de #dataset-lmqg/qg_dequad #arxiv-2210.03992 #license-cc-by-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 

Model Card of 'lmqg/mbart-large-cc25-dequad-qg-ae'
==================================================


This model is fine-tuned version of facebook/mbart-large-cc25 for question generation and answer extraction jointly on the lmqg/qg\_dequad (dataset\_name: default) via 'lmqg'.


### Overview


* Language model: facebook/mbart-large-cc25
* Language: de
* Training data: lmqg/qg\_dequad (default)
* Online Demo: URL
* Repository: URL
* Paper: URL


### Usage


* With 'lmqg'
* With 'transformers'


Evaluation
----------


* *Metric (Question Generation)*: raw metric file



* *Metric (Question & Answer Generation)*: raw metric file



* *Metric (Answer Extraction)*: raw metric file



Training hyperparameters
------------------------


The following hyperparameters were used during fine-tuning:


* dataset\_path: lmqg/qg\_dequad
* dataset\_name: default
* input\_types: ['paragraph\_answer', 'paragraph\_sentence']
* output\_types: ['question', 'answer']
* prefix\_types: ['qg', 'ae']
* model: facebook/mbart-large-cc25
* max\_length: 512
* max\_length\_output: 32
* epoch: 11
* batch: 2
* lr: 0.0001
* fp16: False
* random\_seed: 1
* gradient\_accumulation\_steps: 32
* label\_smoothing: 0.15


The full configuration can be found at fine-tuning config file.



### Overview


* Language model: facebook/mbart-large-cc25
* Language: de
* Training data: lmqg/qg\_dequad (default)
* Online Demo: URL
* Repository: URL
* Paper: URL ### Usage


* With 'lmqg'
* With 'transformers'


Evaluation
----------


* *Metric (Question Generation)*: raw metric file



* *Metric (Question & Answer Generation)*: raw metric file



* *Metric (Answer Extraction)*: raw metric file



Training hyperparameters
------------------------


The following hyperparameters were used during fine-tuning:


* dataset\_path: lmqg/qg\_dequad
* dataset\_name: default
* input\_types: ['paragraph\_answer', 'paragraph\_sentence']
* output\_types: ['question', 'answer']
* prefix\_types: ['qg', 'ae']
* model: facebook/mbart-large-cc25
* max\_length: 512
* max\_length\_output: 32
* epoch: 11
* batch: 2
* lr: 0.0001
* fp16: False
* random\_seed: 1
* gradient\_accumulation\_steps: 32
* label\_smoothing: 0.15


The full configuration can be found at fine-tuning config file.
TAGS
#transformers #pytorch #mbart #text2text-generation #question generation #answer extraction #de #dataset-lmqg/qg_dequad #arxiv-2210.03992 #license-cc-by-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 
 ### Overview


* Language model: facebook/mbart-large-cc25
* Language: de
* Training data: lmqg/qg\_dequad (default)
* Online Demo: URL
* Repository: URL
* Paper: URL ### Usage


* With 'lmqg'
* With 'transformers'


Evaluation
----------


* *Metric (Question Generation)*: raw metric file



* *Metric (Question & Answer Generation)*: raw metric file



* *Metric (Answer Extraction)*: raw metric file



Training hyperparameters
------------------------


The following hyperparameters were used during fine-tuning:


* dataset\_path: lmqg/qg\_dequad
* dataset\_name: default
* input\_types: ['paragraph\_answer', 'paragraph\_sentence']
* output\_types: ['question', 'answer']
* prefix\_types: ['qg', 'ae']
* model: facebook/mbart-large-cc25
* max\_length: 512
* max\_length\_output: 32
* epoch: 11
* batch: 2
* lr: 0.0001
* fp16: False
* random\_seed: 1
* gradient\_accumulation\_steps: 32
* label\_smoothing: 0.15


The full configuration can be found at fine-tuning config file.
TAGS
#transformers #pytorch #mbart #text2text-generation #question generation #answer extraction #de #dataset-lmqg/qg_dequad #arxiv-2210.03992 #license-cc-by-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 
### Overview


* Language model: facebook/mbart-large-cc25
* Language: de
* Training data: lmqg/qg\_dequad (default)
* Online Demo: URL
* Repository: URL
* Paper: URL### Usage


* With 'lmqg'
* With 'transformers'


Evaluation
----------


* *Metric (Question Generation)*: raw metric file



* *Metric (Question & Answer Generation)*: raw metric file



* *Metric (Answer Extraction)*: raw metric file



Training hyperparameters
------------------------


The following hyperparameters were used during fine-tuning:


* dataset\_path: lmqg/qg\_dequad
* dataset\_name: default
* input\_types: ['paragraph\_answer', 'paragraph\_sentence']
* output\_types: ['question', 'answer']
* prefix\_types: ['qg', 'ae']
* model: facebook/mbart-large-cc25
* max\_length: 512
* max\_length\_output: 32
* epoch: 11
* batch: 2
* lr: 0.0001
* fp16: False
* random\_seed: 1
* gradient\_accumulation\_steps: 32
* label\_smoothing: 0.15


The full configuration can be found at fine-tuning config file.