transformers tensorboard safetensors t5 text2text-generation generated_from_trainer base_model:google/flan-t5-small license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
model-index:
- name: t5-dialogue-classification
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-dialogue-classification

This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2827
- Classification Report:                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.14      0.09      0.11        11
           CC       0.35      0.55      0.43        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       1.00      1.00      1.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.40      0.40      0.40         5
    FAM/SOCHX       0.88      0.96      0.91        45
        GENHX       0.77      0.83      0.80        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.58      0.79      0.67        14
 PASTSURGICAL       1.00      0.86      0.92         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.85      0.65      0.73        17

     accuracy                           0.73       200
    macro avg       0.39      0.40      0.39       200
 weighted avg       0.69      0.73      0.71       200


## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Classification Report                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 163  | 0.3372          |                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.00      0.00      0.00        11
           CC       0.20      0.09      0.13        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       0.00      0.00      0.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.50      0.20      0.29         5
    FAM/SOCHX       0.90      0.96      0.92        45
        GENHX       0.63      0.77      0.69        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.31      0.64      0.42        14
 PASTSURGICAL       0.78      1.00      0.88         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.75      0.35      0.48        17

     accuracy                           0.65       200
    macro avg       0.29      0.30      0.28       200
 weighted avg       0.61      0.65      0.61       200
 |
| No log        | 2.0   | 326  | 0.2335          |                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.00      0.00      0.00        11
           CC       0.26      0.55      0.35        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       0.00      0.00      0.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.67      0.40      0.50         5
    FAM/SOCHX       0.88      0.96      0.91        45
        GENHX       0.79      0.87      0.83        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.46      0.79      0.58        14
 PASTSURGICAL       1.00      0.86      0.92         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.83      0.59      0.69        17

     accuracy                           0.73       200
    macro avg       0.34      0.35      0.33       200
 weighted avg       0.68      0.72      0.69       200
 |
| No log        | 3.0   | 489  | 0.2770          |                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.00      0.00      0.00        11
           CC       0.26      0.45      0.33        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       0.00      0.00      0.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.50      0.40      0.44         5
    FAM/SOCHX       0.88      0.96      0.91        45
        GENHX       0.75      0.85      0.80        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.43      0.64      0.51        14
 PASTSURGICAL       0.78      1.00      0.88         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.79      0.65      0.71        17

     accuracy                           0.71       200
    macro avg       0.31      0.34      0.32       200
 weighted avg       0.65      0.71      0.68       200
 |
| 0.3713        | 4.0   | 652  | 0.2746          |                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.11      0.09      0.10        11
           CC       0.35      0.55      0.43        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       1.00      1.00      1.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.25      0.20      0.22         5
    FAM/SOCHX       0.88      0.93      0.90        45
        GENHX       0.86      0.83      0.85        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.52      0.79      0.63        14
 PASTSURGICAL       1.00      1.00      1.00         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.86      0.71      0.77        17

     accuracy                           0.73       200
    macro avg       0.38      0.40      0.39       200
 weighted avg       0.71      0.73      0.71       200
 |
| 0.3713        | 5.0   | 815  | 0.2827          |                precision    recall  f1-score   support

      ALLERGY       1.00      0.92      0.96        12
   ASSESSMENT       0.14      0.09      0.11        11
           CC       0.35      0.55      0.43        11
    DIAGNOSIS       0.00      0.00      0.00         1
  DISPOSITION       1.00      1.00      1.00         1
     EDCOURSE       0.00      0.00      0.00         4
         EXAM       0.40      0.40      0.40         5
    FAM/SOCHX       0.88      0.96      0.91        45
        GENHX       0.77      0.83      0.80        53
        GYNHX       0.00      0.00      0.00         1
      IMAGING       0.00      0.00      0.00         1
IMMUNIZATIONS       0.00      0.00      0.00         1
         LABS       0.00      0.00      0.00         1
  MEDICATIONS       0.83      1.00      0.91        10
OTHER_HISTORY       0.00      0.00      0.00         3
PASTMEDICALHX       0.58      0.79      0.67        14
 PASTSURGICAL       1.00      0.86      0.92         7
         PLAN       0.00      0.00      0.00         1
   PROCEDURES       0.00      0.00      0.00         1
          ROS       0.85      0.65      0.73        17

     accuracy                           0.73       200
    macro avg       0.39      0.40      0.39       200
 weighted avg       0.69      0.73      0.71       200
 |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
