text
🚀 Llama-3-ELYZA-JP-8B
🚀 Llama-3-ELYZA-JP-8B
new_line
new_line
text
ELYZA, Inc. has developed two large language models (LLMs) for Japanese called "Llama-3-ELYZA-JP-70B" with 70 billion parameters and "Llama-3-ELYZA-JP-8B" with 8 billion parameters, based on Meta's "Llama 3" series. These models have been fine-tuned through additional pre-training and post-training to improve Japanese language capabilities significantly.
ELYZA, Inc. has developed two large language models (LLMs) for Japanese called "Llama-3-ELYZA-JP-70B" with 70 billion parameters and "Llama-3-ELYZA-JP-8B" with 8 billion parameters, based on Meta's "Llama 3" series. These models have been fine-tuned through additional pre-training and post-training to improve Japanese language capabilities significantly.
new_line
new_line
text
Key Points:
Key Points:
new_line
new_line
text
Performance: 
Performance: 
new_line
text
- Llama-3-ELYZA-JP-70B surpasses global models such as GPT-4, Claude 3 Sonnet, and Gemini 1.5 Flash.
- Llama-3-ELYZA-JP-70B surpasses global models such as GPT-4, Claude 3 Sonnet, and Gemini 1.5 Flash.
new_line
text
- Llama-3-ELYZA-JP-8B matches models like GPT-3.5 Turbo and Claude 3 Haiku despite having fewer parameters.
- Llama-3-ELYZA-JP-8B matches models like GPT-3.5 Turbo and Claude 3 Haiku despite having fewer parameters.
new_line
new_line
text
Availability:
Availability:
new_line
text
- The 8B model is available on Hugging Face Hub and can be used for both research and commercial purposes under the Llama 3 Community License.
- The 8B model is available on Hugging Face Hub and can be used for both research and commercial purposes under the Llama 3 Community License.
new_line
new_line
text
Methodology:
Methodology:
new_line
text
- ELYZA enhanced the Japanese performance of the Llama 3 models through additional training with high-quality Japanese corpora and Instruction Tuning with proprietary datasets.
- ELYZA enhanced the Japanese performance of the Llama 3 models through additional training with high-quality Japanese corpora and Instruction Tuning with proprietary datasets.
new_line
new_line
text
Benchmarks:
Benchmarks:
new_line
text
- Evaluations using ELYZA Tasks 100 and Japanese MT-Bench showed significant improvements in Japanese language generation.
- Evaluations using ELYZA Tasks 100 and Japanese MT-Bench showed significant improvements in Japanese language generation.
new_line
new_line
text
Inference Speed:
Inference Speed:
new_line
text
- To address inference speed issues due to model size, ELYZA implemented Speculative Decoding, which achieved up to 1.6 times faster inference for the 70B model.
- To address inference speed issues due to model size, ELYZA implemented Speculative Decoding, which achieved up to 1.6 times faster inference for the 70B model.
new_line
new_line
text
Overall, ELYZA's models demonstrate state-of-the-art performance in Japanese language tasks and are optimized for both efficiency and effectiveness.
Overall, ELYZA's models demonstrate state-of-the-art performance in Japanese language tasks and are optimized for both efficiency and effectiveness.
new_line
new_line
text
Model URL:
Model URL:
new_line
text
- 
- 
resource
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
model
elyza/Llama-3-ELYZA-JP-8B
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
new_line
text
- 
- 
resource
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-AWQ
model
elyza/Llama-3-ELYZA-JP-8B-AWQ
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-AWQ
new_line
text
- 
- 
resource
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-GGUF
model
elyza/Llama-3-ELYZA-JP-8B-GGUF
https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-GGUF
new_line
new_line
text
Blog post (in Japanese):
Blog post (in Japanese):
new_line
link
https://note.com/elyza/n/n360b6084fdbd
https://note.com/elyza/n/n360b6084fdbd
🚀 Llama-3-ELYZA-JP-8B

ELYZA, Inc. has developed two large language models (LLMs) for Japanese called "Llama-3-ELYZA-JP-70B" with 70 billion parameters and "Llama-3-ELYZA-JP-8B" with 8 billion parameters, based on Meta's "Llama 3" series. These models have been fine-tuned through additional pre-training and post-training to improve Japanese language capabilities significantly.

Key Points:

Performance: 
- Llama-3-ELYZA-JP-70B surpasses global models such as GPT-4, Claude 3 Sonnet, and Gemini 1.5 Flash.
- Llama-3-ELYZA-JP-8B matches models like GPT-3.5 Turbo and Claude 3 Haiku despite having fewer parameters.

Availability:
- The 8B model is available on Hugging Face Hub and can be used for both research and commercial purposes under the Llama 3 Community License.

Methodology:
- ELYZA enhanced the Japanese performance of the Llama 3 models through additional training with high-quality Japanese corpora and Instruction Tuning with proprietary datasets.

Benchmarks:
- Evaluations using ELYZA Tasks 100 and Japanese MT-Bench showed significant improvements in Japanese language generation.

Inference Speed:
- To address inference speed issues due to model size, ELYZA implemented Speculative Decoding, which achieved up to 1.6 times faster inference for the 70B model.

Overall, ELYZA's models demonstrate state-of-the-art performance in Japanese language tasks and are optimized for both efficiency and effectiveness.

Model URL:
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-AWQ
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-GGUF

Blog post (in Japanese):
https://note.com/elyza/n/n360b6084fdbd