
# Language Detection Model

The model presented in the following repository represents a fine-tuned version of `BertForSequenceClassification` 
pretrained on [multilingual texts](https://huggingface.co/bert-base-multilingual-uncased).

## Training/fine-tuning

The model has been fine-tuned based on [Language Detection](https://www.kaggle.com/datasets/basilb2s/language-detection) 
dataset found on *Kaggle*. The entire process of the dataset analysis as well as a complete description of the training procedure
can be found in [one of my *Kaggle* notebooks](https://www.kaggle.com/code/sergeypolivin/language-recognition-using-bert) 
which has been used for the purpose of a faster model training on *GPU*.

## Supported languages

The model has been fine-tuned to detect one of the following 17 languages:

- Arabic
- Danish
- Dutch
- English
- French
- German
- Greek
- Hindi
- Italian
- Kannada
- Malayalam
- Portugeese
- Russian
- Spanish
- Sweedish
- Tamil
- Turkish

## References

1. [BERT multilingual base model (uncased)](https://huggingface.co/bert-base-multilingual-uncased)
2. [Language Detection Dataset](https://www.kaggle.com/datasets/basilb2s/language-detection)
{"language": ["ar", "da", "nl", "en", "fr", "de", "el", "hi", "it", "kn", "ml", "pt", "ru", "es", "sv", "ta", "tr"], "license": "mit", "base_model": "bert-base-multilingual-uncased", "pipeline_tag": "text-classification", "widget": [{"text": "I have seen it somewhere...", "example_title": "English"}, {"text": "Ik heb het al gezien", "example_title": "Dutch"}, {"text": "\u0418\u043d\u0442\u0435\u0440\u0435\u0441\u043d\u0430\u044f \u0438\u0434\u0435\u044f", "example_title": "Russian"}, {"text": "Que vamos a hacer?", "example_title": "Spanish"}, {"text": "Hvor er der en pengeautomat?", "example_title": "Danish"}, {"text": "\u0625\u0646\u0647 \u0645\u0634\u0648\u0642 \u062c\u062f\u0627", "example_title": "Arabic"}, {"text": "Es ist sehr interessant", "example_title": "German"}, {"text": "c'est tr\u00e8s int\u00e9ressant", "example_title": "French"}, {"text": "Non ho mai visto una tale bellezza", "example_title": "Italian"}, {"text": "Jag har aldrig sett en s\u00e5dan sk\u00f6nhet", "example_title": "Swedish"}, {"text": "B\u00f6yle bir g\u00fczellik g\u00f6rmedim", "example_title": "Turkish"}, {"text": "\u0c85\u0ca6\u0ccd\u0cad\u0cc1\u0ca4 \u0c95\u0cb2\u0ccd\u0caa\u0ca8\u0cc6", "example_title": "Kannada"}, {"text": "\u0b85\u0bb1\u0bcd\u0baa\u0bc1\u0ba4\u0bae\u0bbe\u0ba9 \u0baf\u0bcb\u0b9a\u0ba9\u0bc8", "example_title": "Tamil"}, {"text": "\u03a5\u03c0\u03ad\u03c1\u03bf\u03c7\u03b7 \u03b9\u03b4\u03ad\u03b1", "example_title": "Greek"}, {"text": "Eu nunca estive aqui", "example_title": "Portugeese"}, {"text": "\u092e\u0948\u0902 \u092f\u0939\u093e\u0902 \u0915\u092d\u0940 \u0928\u0939\u0940\u0902 \u0917\u092f\u093e", "example_title": "Hindi"}, {"text": "\u0d1e\u0d3e\u0d7b \u0d07\u0d35\u0d3f\u0d1f\u0d46 \u0d2a\u0d4b\u0d2f\u0d3f\u0d1f\u0d4d\u0d1f\u0d3f\u0d32\u0d4d\u0d32", "example_title": "Malayam"}]}
transformers safetensors bert text-classification ar da nl en fr de el hi it kn ml pt ru es sv ta tr base_model:bert-base-multilingual-uncased license:mit autotrain_compatible endpoints_compatible region:us