
# Model Card for Model hybrinfox/ukraine-invasion_propaganda-detection-FR

This model aims at identifying propaganda on the topic of the Ukrainian invasion in  press articles.

## Model Details

### Model Description

The model is a fine-tuned version of camembert-base (https://huggingface.co/camembert-base) on the Propagandist Pseudo-News dataset (https://github.com/hybrinfox/ppn) 

- **Owned by:** Airbus Defence and Space
- **Developed for:** HYBRINFOX consortium (Airbus Defence and Space - Paris Sciences et Lettres, Ecole Normale Supérieure Ulm, Institut Jean-Nicod  - Université de Rennes, Inria, IRISA, Mondeca)
- **Funded by :** French National Research Agency (ANR-21-ASIA-0003)
- **Model type:** Text classification
- **Language(s) (NLP):** fr
- **License:** CC BY-NB 4.0
- **Finetuned from model :** camembert-base


## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The model can be used directly to classify press articles written in French about the Ukraine invasion or related topic. The output corresponds to the probability of belonging to each class, 0 for regular press articles and 1 for propagandist article.

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

This model should not be used to categorize news sources as propagandist or not, but can help identify pro-Russian narratives and Russian values. This model is not trained to identify the auhors' intentions and should not be used to make such conclusions.

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

This model has been trained with articles from different sources, but all articles from the propaganda class share the same narrative. Moreover, all articles shared the same topic of the Russio-Ukrainian conflict.
The model is not infaillible and shouldn't be use to make critical decisions when judging an article, its authors, or the corresponding news outlet.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

We recommend that you use this model for research purposes and to always cross its predictions with the informed opinion of other sources before taking any conclusion.

## How to Get Started with the Model

Use the code below to get started with the model.

```
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-classification", model="hybrinfox/ukraine-invasion_propaganda-detection-FR")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("hybrinfox/ukraine-invasion_propaganda-detection-FR")
model = AutoModelForSequenceClassification.from_pretrained("hybrinfox/ukraine-invasion_propaganda-detection-FR")
```

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The model has been trained using the data from the Propagandist Pseudo-News dataset available at https://github.com/hybrinfox/ppn for the positive class. Additional articles on the same topic, but from mainstream sources has been used for the negative class. Please, read the paper for more details.

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Training Hyperparameters

- **Training regime:** 

Batch size: 8
Learning rate: 5e-5
Number of fine-tuning epochs: 3
Optimizer: Adam with default settings
Loss function: Binary Cross-Entropy

## Evaluation

The model was evaluated during training with the training metrics, as well as with the validation loss

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

The previously described dataset has been split between train/val/test with a 80/10/10 ratio. The reported results are on the test set, after using the training set for training and validation for controling the model learning. 

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

The reported metrics are the F1 scores and losses on the three sets.

### Results


| Split | Loss | F1 score |
|---|---|---|
| Train |0.0012|1.0000|
| Val   |0.0179|0.9973|
| Test  |0.0796|0.9873|


#### Summary



## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** T4
- **Hours used:** 0.3
- **Cloud Provider:** GCP
- **Compute Region:** europe-west1
- **Carbon Emitted:** 0.01 kg.CO2 eq

Thanks to fine-tuning a general foundation model, the environmental impact of training our propaganda detector is negligible, being the equivalent of 40 meters traveled by an internal combustion engine car. The low-carbon energy used in the compute region also helped to reduce the environmental impact of the training.

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

Faye, G., Icard, B., Casanova, M., Chanson, J., Maine, F., Bancilhon, F., Gadek, G., Gravier, G., & Égré, P. (2024). Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification, In EACL Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language (UnImplicit 2024).

**BibTeX:**

The corresponding article is to be published at the UnImplicit 2024 workshop at EACL 2024.



## Model Card Authors 

HYBRINFOX consortium

## Model Card Contact

hybrinfox@gmail.com
{"language": ["fr"], "license": "mit", "pipeline_tag": "text-classification"}
transformers safetensors camembert text-classification fr arxiv:1910.09700 license:mit autotrain_compatible endpoints_compatible region:us