text
Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥
Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥
new_line
new_line
text
> WhisperSpeech X Llama 3.1 8B
> WhisperSpeech X Llama 3.1 8B
new_line
text
> Trained on 50K hours of speech (7 languages)
> Trained on 50K hours of speech (7 languages)
new_line
text
> Continually trained on 45hrs 10x A1000s
> Continually trained on 45hrs 10x A1000s
new_line
text
> MLS -> WhisperVQ tokens -> Llama 3.1
> MLS -> WhisperVQ tokens -> Llama 3.1
new_line
text
> Instruction tuned on 1.89M samples
> Instruction tuned on 1.89M samples
new_line
text
> 70% speech, 20% transcription, 10% text
> 70% speech, 20% transcription, 10% text
new_line
text
> Apache 2.0 licensed ⚡
> Apache 2.0 licensed ⚡
new_line
new_line
text
Architecture:
Architecture:
new_line
text
> WhisperSpeech/ VQ for Semantic Tokens
> WhisperSpeech/ VQ for Semantic Tokens
new_line
text
> Llama 3.1 8B Instruct for Text backbone
> Llama 3.1 8B Instruct for Text backbone
new_line
text
> Early fusion (Chameleon)
> Early fusion (Chameleon)
new_line
new_line
text
I'm super bullish on HomeBrew/ Jan and early fusion, audio and text, multimodal models!
I'm super bullish on HomeBrew/ Jan and early fusion, audio and text, multimodal models!
new_line
new_line
text
(P.S. Play with the demo on Hugging Face: 
(P.S. Play with the demo on Hugging Face: 
resource
https://huggingface.co/spaces/jan-hq/Ichigo-llama3.1-s-instruct
space
jan-hq/Ichigo-llama3.1-s-instruct
https://huggingface.co/spaces/jan-hq/Ichigo-llama3.1-s-instruct
text
)
)
new_line
Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥

> WhisperSpeech X Llama 3.1 8B
> Trained on 50K hours of speech (7 languages)
> Continually trained on 45hrs 10x A1000s
> MLS -> WhisperVQ tokens -> Llama 3.1
> Instruction tuned on 1.89M samples
> 70% speech, 20% transcription, 10% text
> Apache 2.0 licensed ⚡

Architecture:
> WhisperSpeech/ VQ for Semantic Tokens
> Llama 3.1 8B Instruct for Text backbone
> Early fusion (Chameleon)

I'm super bullish on HomeBrew/ Jan and early fusion, audio and text, multimodal models!

(P.S. Play with the demo on Hugging Face: https://huggingface.co/spaces/jan-hq/Ichigo-llama3.1-s-instruct)

🔥
victor philipp-zettl Norod78 diwank John6666 YaTharThShaRma999 s9133 DRRK0 Timilla Jooinjang blanchon edwixx anuragsingh17ai KingNish ariG23498 WaveCut
👍
vigos YaTharThShaRma999 chenduo Yadukrishnan edwixx
❤️
Sanzid JPG2000
😎
alandao
👀
den0620
🚀
Svngoku