
ERNIE
Overview
ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks,
including ERNIE1.0, ERNIE2.0,
ERNIE3.0, ERNIE-Gram, ERNIE-health, etc.
These models are contributed by nghuyong and the official code can be found in PaddleNLP (in PaddlePaddle).
Usage example
Take ernie-1.0-base-zh as an example:
Python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")
Model checkpoints
|     Model Name      | Language |           Description           |
|:-------------------:|:--------:|:-------------------------------:|
|  ernie-1.0-base-zh  | Chinese  | Layer:12, Heads:12, Hidden:768  |
|  ernie-2.0-base-en  | English  | Layer:12, Heads:12, Hidden:768  |
| ernie-2.0-large-en  | English  | Layer:24, Heads:16, Hidden:1024 |
|  ernie-3.0-base-zh  | Chinese  | Layer:12, Heads:12, Hidden:768  |
| ernie-3.0-medium-zh | Chinese  |  Layer:6, Heads:12, Hidden:768  |
|  ernie-3.0-mini-zh  | Chinese  |  Layer:6, Heads:12, Hidden:384  |
| ernie-3.0-micro-zh  | Chinese  |  Layer:4, Heads:12, Hidden:384  |
|  ernie-3.0-nano-zh  | Chinese  |  Layer:4, Heads:12, Hidden:312  |
|   ernie-health-zh   | Chinese  | Layer:12, Heads:12, Hidden:768  |
|    ernie-gram-zh    | Chinese  | Layer:12, Heads:12, Hidden:768  |
You can find all the supported models from huggingface's model hub: huggingface.co/nghuyong, and model details from paddle's official
repo: PaddleNLP
and ERNIE.
Resources