---
language: en
tags:
- protein
- protbert
- masked-language-modeling
- bioinformatics
- sequence-prediction
datasets:
- custom
license: mit
library_name: transformers
pipeline_tag: fill-mask
---

# ProtBERT-Unmasking

This model is a fine-tuned version of ProtBERT specifically optimized for unmasking protein sequences. It can predict masked amino acids in protein sequences based on the surrounding context.

## Model Description

- **Base Model**: ProtBERT
- **Task**: Protein Sequence Unmasking
- **Training**: Fine-tuned on masked protein sequences
- **Use Case**: Predicting missing or masked amino acids in protein sequences
- **Optimal Use**: Best performance on E. coli sequences with known amino acids K, C, Y, H, S, M

For detailed information about the training methodology and approach, please refer to our paper:
[https://arxiv.org/abs/2408.00892](https://arxiv.org/abs/2408.00892)

## Usage

```python
from transformers import AutoModelForMaskedLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForMaskedLM.from_pretrained("your-username/protbert-sequence-unmasking")
tokenizer = AutoTokenizer.from_pretrained("your-username/protbert-sequence-unmasking")

# Example usage for E. coli sequence with known amino acids (K,C,Y,H,S,M)
sequence = "MALN[MASK]KFGP[MASK]LVRK"
inputs = tokenizer(sequence, return_tensors="pt")
outputs = model(**inputs)
predictions = outputs.logits
```

## Inference API

The model is optimized for:
- **Organism**: E. coli
- **Known Amino Acids**: K, C, Y, H, S, M
- **Task**: Predicting unknown amino acids in a sequence

Example API usage:
```python
from transformers import pipeline

unmasker = pipeline('fill-mask', model='your-username/protbert-sequence-unmasking')
sequence = "K[MASK]YHS[MASK]"  # Example with known amino acids K,Y,H,S
results = unmasker(sequence)

for result in results:
    print(f"Predicted amino acid: {result['token_str']}, Score: {result['score']:.3f}")
```

## Limitations and Biases

- This model is specifically designed for protein sequence unmasking in E. coli
- Optimal performance is achieved when working with sequences containing known amino acids K, C, Y, H, S, M
- The model may not perform optimally for:
  - Sequences from other organisms
  - Sequences without the specified known amino acids
  - Other protein-related tasks

## Training Details

The complete details of the training methodology, dataset preparation, and model evaluation can be found in our paper:
[https://arxiv.org/abs/2408.00892](https://arxiv.org/abs/2408.00892)