
# Model Description
ClinicalDistilBERT-i2b2-2010 is a lightweight BERT-based model developed by fine-tuning [ClinicalDistilBERT](https://huggingface.co/nlpie/clinical-distilbert) on the i2b2-2010 dataset for clinical Named Entity Recognition (NER). It is specifically designed to recognise entities from three categories: `problem`, `treatment`, and `test`. 

# Architecture
The architecture of this model remains the same as the ClinicalDistilBERT model. The size of the hidden dimension and the embedding layer are both set to 768. The vocabulary size is 28996. The number of transformer layers is 6, and the expansion rate of the feed-forward layer is 4. Overall, this model contains approximately 65 million parameters.

# Use Cases
This model is suited for clinical NER and for medical tasks  that require identification and classification of problems, treatments, and tests. 

# Citation
If you use this model, please consider citing the following paper:

```bibtex
@misc{https://doi.org/10.48550/arxiv.2302.04725,
  doi = {10.48550/ARXIV.2302.04725},
  url = {https://arxiv.org/abs/2302.04725},
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Jauncey, Hannah and Kouchaki, Samaneh and Group, ISARIC Clinical Characterisation and Clifton, Lei and Merson, Laura and Clifton, David A.},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7, 68T50},
  title = {Lightweight Transformers for Clinical Natural Language Processing},
  publisher = {arXiv},
  year = {2023},
  copyright = {arXiv.org perpetual, non-exclusive license}
}


{"license": "mit", "title": "README", "emoji": "\ud83e\uddec", "colorFrom": "gray", "colorTo": "purple", "sdk": "static", "pinned": false}
transformers pytorch distilbert token-classification arxiv:2302.04725 license:mit autotrain_compatible endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #distilbert #token-classification #arxiv-2302.04725 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 


# Model Description
ClinicalDistilBERT-i2b2-2010 is a lightweight BERT-based model developed by fine-tuning ClinicalDistilBERT on the i2b2-2010 dataset for clinical Named Entity Recognition (NER). It is specifically designed to recognise entities from three categories: 'problem', 'treatment', and 'test'. 

# Architecture
The architecture of this model remains the same as the ClinicalDistilBERT model. The size of the hidden dimension and the embedding layer are both set to 768. The vocabulary size is 28996. The number of transformer layers is 6, and the expansion rate of the feed-forward layer is 4. Overall, this model contains approximately 65 million parameters.

# Use Cases
This model is suited for clinical NER and for medical tasks  that require identification and classification of problems, treatments, and tests. 

If you use this model, please consider citing the following paper:

'''bibtex
@misc{URL
  doi = {10.48550/ARXIV.2302.04725},
  url = {URL
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Jauncey, Hannah and Kouchaki, Samaneh and Group, ISARIC Clinical Characterisation and Clifton, Lei and Merson, Laura and Clifton, David A.},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7, 68T50},
  title = {Lightweight Transformers for Clinical Natural Language Processing},
  publisher = {arXiv},
  year = {2023},
  copyright = {URL perpetual, non-exclusive license}
}


# Model Description
ClinicalDistilBERT-i2b2-2010 is a lightweight BERT-based model developed by fine-tuning ClinicalDistilBERT on the i2b2-2010 dataset for clinical Named Entity Recognition (NER). It is specifically designed to recognise entities from three categories: 'problem', 'treatment', and 'test'. # Architecture
The architecture of this model remains the same as the ClinicalDistilBERT model. The size of the hidden dimension and the embedding layer are both set to 768. The vocabulary size is 28996. The number of transformer layers is 6, and the expansion rate of the feed-forward layer is 4. Overall, this model contains approximately 65 million parameters. # Use Cases
This model is suited for clinical NER and for medical tasks  that require identification and classification of problems, treatments, and tests. 

If you use this model, please consider citing the following paper:

'''bibtex
@misc{URL
  doi = {10.48550/ARXIV.2302.04725},
  url = {URL
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Jauncey, Hannah and Kouchaki, Samaneh and Group, ISARIC Clinical Characterisation and Clifton, Lei and Merson, Laura and Clifton, David A.},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7, 68T50},
  title = {Lightweight Transformers for Clinical Natural Language Processing},
  publisher = {arXiv},
  year = {2023},
  copyright = {URL perpetual, non-exclusive license}
}
TAGS
#transformers #pytorch #distilbert #token-classification #arxiv-2302.04725 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 
 # Model Description
ClinicalDistilBERT-i2b2-2010 is a lightweight BERT-based model developed by fine-tuning ClinicalDistilBERT on the i2b2-2010 dataset for clinical Named Entity Recognition (NER). It is specifically designed to recognise entities from three categories: 'problem', 'treatment', and 'test'. # Architecture
The architecture of this model remains the same as the ClinicalDistilBERT model. The size of the hidden dimension and the embedding layer are both set to 768. The vocabulary size is 28996. The number of transformer layers is 6, and the expansion rate of the feed-forward layer is 4. Overall, this model contains approximately 65 million parameters. # Use Cases
This model is suited for clinical NER and for medical tasks  that require identification and classification of problems, treatments, and tests. 

If you use this model, please consider citing the following paper:

'''bibtex
@misc{URL
  doi = {10.48550/ARXIV.2302.04725},
  url = {URL
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Jauncey, Hannah and Kouchaki, Samaneh and Group, ISARIC Clinical Characterisation and Clifton, Lei and Merson, Laura and Clifton, David A.},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7, 68T50},
  title = {Lightweight Transformers for Clinical Natural Language Processing},
  publisher = {arXiv},
  year = {2023},
  copyright = {URL perpetual, non-exclusive license}
}
TAGS
#transformers #pytorch #distilbert #token-classification #arxiv-2302.04725 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 
# Model Description
ClinicalDistilBERT-i2b2-2010 is a lightweight BERT-based model developed by fine-tuning ClinicalDistilBERT on the i2b2-2010 dataset for clinical Named Entity Recognition (NER). It is specifically designed to recognise entities from three categories: 'problem', 'treatment', and 'test'.# Architecture
The architecture of this model remains the same as the ClinicalDistilBERT model. The size of the hidden dimension and the embedding layer are both set to 768. The vocabulary size is 28996. The number of transformer layers is 6, and the expansion rate of the feed-forward layer is 4. Overall, this model contains approximately 65 million parameters.# Use Cases
This model is suited for clinical NER and for medical tasks  that require identification and classification of problems, treatments, and tests. 

If you use this model, please consider citing the following paper:

'''bibtex
@misc{URL
  doi = {10.48550/ARXIV.2302.04725},
  url = {URL
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Jauncey, Hannah and Kouchaki, Samaneh and Group, ISARIC Clinical Characterisation and Clifton, Lei and Merson, Laura and Clifton, David A.},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7, 68T50},
  title = {Lightweight Transformers for Clinical Natural Language Processing},
  publisher = {arXiv},
  year = {2023},
  copyright = {URL perpetual, non-exclusive license}
}