# HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) fine-tuned for **Part-of-Speech Tagging (POS) Task**.  


# Documentation


## Prerequisite
Since we use subword-nmt BPE encoding, input needs to be pre-tokenize using [BEST](https://huggingface.co/datasets/best2009) standard before inputting into HoogBERTa 
```
pip install attacut
```

## Getting Start
To initialize the model from hub, use the following commands
```python
from transformers import RobertaTokenizerFast, RobertaForTokenClassification
from attacut import tokenize
import torch

tokenizer = RobertaTokenizerFast.from_pretrained("lst-nectec/HoogBERTa-POS-lst20")
model = RobertaForTokenClassification.from_pretrained("lst-nectec/HoogBERTa-POS-lst20")
```

To do POS Tagging, use the following commands

```python
from transformers import pipeline

nlp = pipeline('token-classification', model=model, tokenizer=tokenizer, aggregation_strategy="none")

sentence = "วันที่ 12 มีนาคมนี้ ฉันจะไปเที่ยววัดพระแก้ว ที่กรุงเทพ"
all_sent = []
sentences = sentence.split(" ")
for sent in sentences:
    all_sent.append(" ".join(tokenize(sent)).replace("_","[!und:]"))

sentence = " _ ".join(all_sent)

print(nlp(sentence))
```

For batch processing,

```python
from transformers import pipeline

nlp = pipeline('token-classification', model=model, tokenizer=tokenizer, aggregation_strategy="none")

sentenceL = ["วันที่ 12 มีนาคมนี้","ฉันจะไปเที่ยววัดพระแก้ว ที่กรุงเทพ"]
inputList = []
for sentX in sentenceL:
  sentences = sentX.split(" ")
  all_sent = []
  for sent in sentences:
      all_sent.append(" ".join(tokenize(sent)).replace("_","[!und:]"))

  sentence = " _ ".join(all_sent)
  inputList.append(sentence)

print(nlp(inputList))
```

# Huggingface Models
1. `HoogBERTaEncoder`
 - [HoogBERTa](https://huggingface.co/lst-nectec/HoogBERTa): `Feature Extraction` and `Mask Language Modeling`
2. `HoogBERTaMuliTaskTagger`:
 - [HoogBERTa-NER-lst20](https://huggingface.co/lst-nectec/HoogBERTa-NER-lst20): `Named-entity recognition (NER)` based on LST20
 - [HoogBERTa-POS-lst20](https://huggingface.co/lst-nectec/HoogBERTa-POS-lst20): `Part-of-speech tagging (POS)` based on LST20
 - [HoogBERTa-SENTENCE-lst20](https://huggingface.co/lst-nectec/HoogBERTa-SENTENCE-lst20): `Clause Boundary Classification` based on LST20


# Citation

Please cite as:

``` bibtex
@inproceedings{porkaew2021hoogberta,
  title = {HoogBERTa: Multi-task Sequence Labeling using Thai Pretrained Language Representation},
  author = {Peerachet Porkaew, Prachya Boonkwan and Thepchai Supnithi},
  booktitle = {The Joint International Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2021)},
  year = {2021},
  address={Online}
}
```

Download full-text [PDF](https://drive.google.com/file/d/1hwdyIssR5U_knhPE2HJigrc0rlkqWeLF/view?usp=sharing)  
Check out the code on [Github](https://github.com/lstnlp/HoogBERTa)
{"language": ["th"], "library_name": "transformers", "datasets": ["lst20"], "widget": [{"text": "\u0e27\u0e31\u0e19 \u0e17\u0e35\u0e48 _ 12 _ \u0e21\u0e35\u0e19\u0e32\u0e04\u0e21 \u0e19\u0e35\u0e49 _ \u0e09\u0e31\u0e19 \u0e08\u0e30 \u0e44\u0e1b \u0e40\u0e17\u0e35\u0e48\u0e22\u0e27 \u0e27\u0e31\u0e14\u0e1e\u0e23\u0e30\u0e41\u0e01\u0e49\u0e27 _ \u0e17\u0e35\u0e48 \u0e01\u0e23\u0e38\u0e07\u0e40\u0e17\u0e1e"}]}
transformers pytorch roberta token-classification th dataset:lst20 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #roberta #token-classification #th #dataset-lst20 #autotrain_compatible #endpoints_compatible #region-us 

# HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) fine-tuned for Part-of-Speech Tagging (POS) Task.  


# Documentation


## Prerequisite
Since we use subword-nmt BPE encoding, input needs to be pre-tokenize using BEST standard before inputting into HoogBERTa 


## Getting Start
To initialize the model from hub, use the following commands


To do POS Tagging, use the following commands



For batch processing,



# Huggingface Models
1. 'HoogBERTaEncoder'
 - HoogBERTa: 'Feature Extraction' and 'Mask Language Modeling'
2. 'HoogBERTaMuliTaskTagger':
 - HoogBERTa-NER-lst20: 'Named-entity recognition (NER)' based on LST20
 - HoogBERTa-POS-lst20: 'Part-of-speech tagging (POS)' based on LST20
 - HoogBERTa-SENTENCE-lst20: 'Clause Boundary Classification' based on LST20


Please cite as:



Download full-text PDF  
Check out the code on Github
# HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) fine-tuned for Part-of-Speech Tagging (POS) Task. # Documentation ## Prerequisite
Since we use subword-nmt BPE encoding, input needs to be pre-tokenize using BEST standard before inputting into HoogBERTa ## Getting Start
To initialize the model from hub, use the following commands


To do POS Tagging, use the following commands



For batch processing, # Huggingface Models
1. 'HoogBERTaEncoder'
 - HoogBERTa: 'Feature Extraction' and 'Mask Language Modeling'
2. 'HoogBERTaMuliTaskTagger':
 - HoogBERTa-NER-lst20: 'Named-entity recognition (NER)' based on LST20
 - HoogBERTa-POS-lst20: 'Part-of-speech tagging (POS)' based on LST20
 - HoogBERTa-SENTENCE-lst20: 'Clause Boundary Classification' based on LST20


Please cite as:



Download full-text PDF  
Check out the code on Github
TAGS
#transformers #pytorch #roberta #token-classification #th #dataset-lst20 #autotrain_compatible #endpoints_compatible #region-us 
 # HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) fine-tuned for Part-of-Speech Tagging (POS) Task. # Documentation ## Prerequisite
Since we use subword-nmt BPE encoding, input needs to be pre-tokenize using BEST standard before inputting into HoogBERTa ## Getting Start
To initialize the model from hub, use the following commands


To do POS Tagging, use the following commands



For batch processing, # Huggingface Models
1. 'HoogBERTaEncoder'
 - HoogBERTa: 'Feature Extraction' and 'Mask Language Modeling'
2. 'HoogBERTaMuliTaskTagger':
 - HoogBERTa-NER-lst20: 'Named-entity recognition (NER)' based on LST20
 - HoogBERTa-POS-lst20: 'Part-of-speech tagging (POS)' based on LST20
 - HoogBERTa-SENTENCE-lst20: 'Clause Boundary Classification' based on LST20


Please cite as:



Download full-text PDF  
Check out the code on Github
TAGS
#transformers #pytorch #roberta #token-classification #th #dataset-lst20 #autotrain_compatible #endpoints_compatible #region-us 
# HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) fine-tuned for Part-of-Speech Tagging (POS) Task.# Documentation## Prerequisite
Since we use subword-nmt BPE encoding, input needs to be pre-tokenize using BEST standard before inputting into HoogBERTa## Getting Start
To initialize the model from hub, use the following commands


To do POS Tagging, use the following commands



For batch processing,# Huggingface Models
1. 'HoogBERTaEncoder'
 - HoogBERTa: 'Feature Extraction' and 'Mask Language Modeling'
2. 'HoogBERTaMuliTaskTagger':
 - HoogBERTa-NER-lst20: 'Named-entity recognition (NER)' based on LST20
 - HoogBERTa-POS-lst20: 'Part-of-speech tagging (POS)' based on LST20
 - HoogBERTa-SENTENCE-lst20: 'Clause Boundary Classification' based on LST20


Please cite as:



Download full-text PDF  
Check out the code on Github