---
license: apache-2.0
base_model: distilroberta-base
tags:
  - text-classification
  - equivalence-classification
  - generated_from_trainer
metrics:
  - accuracy
  - f1
model-index:
  - name: Text Equivalence Classification
    results: []
datasets:
  - glue
pipeline_tag: text-classification



# Text Equivalence Classification


This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) specifically for the task of determining whether two sentences are equivalent in meaning. It was trained on a curated dataset combining multiple sources to cover a wide range of topics and styles.

## Model Description

The model aims to capture the nuanced relationships between sentences, leveraging the powerful pre-trained DistilRoBERTa base for understanding contextual meanings. It has been fine-tuned with a specific focus on identifying semantic equivalence, making it suitable for applications such as paraphrase identification, duplicate question detection, and semantic search tasks.

## Intended Uses & Limitations

This model is intended for use in applications requiring the determination of semantic equivalence between pairs of sentences. It can be utilized in creating more efficient search algorithms, enhancing chatbot understandings, and improving content moderation by identifying duplicate or similar content.

However, the model may exhibit limitations when dealing with sentences that contain highly domain-specific jargon, rare languages, or nuanced cultural references. It is recommended to test the model with your specific data and use case to ensure its effectiveness.

## Training and Evaluation Data

The model was trained on a dataset named "DatasetX," which is a composite of several public datasets including portions of the Quora Question Pairs and the Microsoft Research Paraphrase Corpus. This combination was chosen to create a robust model capable of understanding a wide variety of sentence structures and meanings.

## Training Procedure

### Training Hyperparameters

The following hyperparameters were used during training:
- Learning rate: 5e-05
- Train batch size: 8
- Eval batch size: 8
- Seed: 42
- Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- LR scheduler type: linear
- Number of epochs: 3

### Training Results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
| 0.5396        | 1.09  | 500  | 0.5511          | 0.8309   | 0.8848 |
| 0.3722        | 2.18  | 1000 | 0.4973          | 0.8529   | 0.8944 |

### Framework Versions

- Transformers 4.37.2
- Pytorch 2.2.0
- Datasets 2.17.0
- Tokenizers 0.15.2

{}
transformers tensorboard safetensors roberta text-classification generated_from_trainer base_model:distilroberta-base license:apache-2.0 autotrain_compatible endpoints_compatible region:us