---
license: mit
tags:
  - generated_from_trainer
metrics:
  - f1
  - accuracy
model-index:
  - name: Kemenkeu-Sentiment-Classifier
    results:
      - task:
          name: Text Classification
          type: text-classification
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.66
          - name: F1
            type: f1
            value: 0.6368
language:
  - id
pipeline_tag: text-classification
widget:
  - text: sudah beli makan buat sahur?
    example_title: "contoh tidak relevan"
  - text: Mengawal APBN, Indonesia Maju
    example_title: "contoh kalimat"
---

# Kemenkeu-Sentiment-Classifier

This model is a fine-tuned version of [indobenchmark/indobert-base-p1](https://huggingface.co/indobenchmark/indobert-base-p1) on the MoF-DAC Mini Challenge#1 dataset.
It achieves the following results on the evaluation set:
- Accuracy: 0.66
- F1: 0.6368

Leaderboard score:
- Public score: 0.63733
- Private score: 0.65733

## Model description & limitations

- This model can be used to classify text with four possible outputs [netral, tdk-relevan, negatif, and positif]
- only for specific cases related to the Ministry Of Finance Indonesia

## How to use
You can use this model directly with a pipeline

```python
pretrained_name = "hanifnoerr/Kemenkeu-Sentiment-Classifier"
class_model = pipeline(tokenizer=pretrained_name, model=pretrained_name)

test_data = "Mengawal APBN, Indonesia Maju"
class_model(test_data)
```

## Training and evaluation data

The following hyperparameters were used during training:
- learning_rate: 1e-05
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
| 1.0131        | 1.0   | 500  | 0.8590          | 0.644    | 0.5964 |
| 0.7133        | 2.0   | 1000 | 0.8639          | 0.63     | 0.5924 |
| 0.5261        | 3.0   | 1500 | 0.9002          | 0.66     | 0.6368 |


### Framework versions

- Transformers 4.27.4
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3