text
Good folks at 
Good folks at 
mention
@nvidia
nvidia
text
  and @Tsinghua_Uni have released LLAMA-MESH - A Revolutionary Approach to 3D Content Generation!
  and @Tsinghua_Uni have released LLAMA-MESH - A Revolutionary Approach to 3D Content Generation!
new_line
new_line
text
This innovative framework enables the direct generation of 3D meshes from natural language prompts while maintaining strong language capabilities.
This innovative framework enables the direct generation of 3D meshes from natural language prompts while maintaining strong language capabilities.
new_line
new_line
text
Here is the Architecture & Implementation!
Here is the Architecture & Implementation!
new_line
new_line
text
>> Core Components
>> Core Components
new_line
new_line
text
Model Foundation  
Model Foundation  
new_line
text
- If you haven't guessed it yet, it's built on the LLaMA-3.1-8B-Instruct base model  
- If you haven't guessed it yet, it's built on the LLaMA-3.1-8B-Instruct base model  
new_line
text
- Maintains original language capabilities while adding 3D generation  
- Maintains original language capabilities while adding 3D generation  
new_line
text
- Context length is set to 8,000 tokens  
- Context length is set to 8,000 tokens  
new_line
new_line
text
3D Representation Strategy  
3D Representation Strategy  
new_line
text
- Uses the OBJ file format for mesh representation  
- Uses the OBJ file format for mesh representation  
new_line
text
- Quantizes vertex coordinates into 64 discrete bins per axis  
- Quantizes vertex coordinates into 64 discrete bins per axis  
new_line
text
- Sorts vertices by z-y-x coordinates, from lowest to highest  
- Sorts vertices by z-y-x coordinates, from lowest to highest  
new_line
text
- Sorts faces by the lowest vertex indices for consistency  
- Sorts faces by the lowest vertex indices for consistency  
new_line
new_line
text
Data Processing Pipeline  
Data Processing Pipeline  
new_line
text
- Filters meshes to a maximum of 500 faces for computational efficiency  
- Filters meshes to a maximum of 500 faces for computational efficiency  
new_line
text
- Applies random rotations (0°, 90°, 180°, 270°) for data augmentation  
- Applies random rotations (0°, 90°, 180°, 270°) for data augmentation  
new_line
text
- Generates ~125k mesh variations from 31k base meshes  
- Generates ~125k mesh variations from 31k base meshes  
new_line
text
- Uses Cap3D-generated captions for text descriptions  
- Uses Cap3D-generated captions for text descriptions  
new_line
new_line
text
>> Training Framework
>> Training Framework
new_line
new_line
text
Dataset Composition  
Dataset Composition  
new_line
text
- 40% Mesh Generation tasks  
- 40% Mesh Generation tasks  
new_line
text
- 20% Mesh Understanding tasks  
- 20% Mesh Understanding tasks  
new_line
text
- 40% General Conversation (UltraChat dataset)  
- 40% General Conversation (UltraChat dataset)  
new_line
text
- 8x training turns for generation, 4x for understanding  
- 8x training turns for generation, 4x for understanding  
new_line
new_line
text
Training Configuration  
Training Configuration  
new_line
text
- Deployed on 32 A100 GPUs (for Nvidia, this is literally in-house)  
- Deployed on 32 A100 GPUs (for Nvidia, this is literally in-house)  
new_line
text
- 21,000 training iterations  
- 21,000 training iterations  
new_line
text
- Global batch size: 128  
- Global batch size: 128  
new_line
text
- AdamW optimizer with a 1e-5 learning rate  
- AdamW optimizer with a 1e-5 learning rate  
new_line
text
- 30-step warmup with cosine scheduling  
- 30-step warmup with cosine scheduling  
new_line
text
- Total training time: approximately 3 days (based on the paper)  
- Total training time: approximately 3 days (based on the paper)  
new_line
new_line
text
This research opens exciting possibilities for intuitive 3D content creation through natural language interaction. The future of digital design is conversational!
This research opens exciting possibilities for intuitive 3D content creation through natural language interaction. The future of digital design is conversational!
Good folks at @nvidia  and @Tsinghua_Uni have released LLAMA-MESH - A Revolutionary Approach to 3D Content Generation!

This innovative framework enables the direct generation of 3D meshes from natural language prompts while maintaining strong language capabilities.

Here is the Architecture & Implementation!

>> Core Components

Model Foundation  
- If you haven't guessed it yet, it's built on the LLaMA-3.1-8B-Instruct base model  
- Maintains original language capabilities while adding 3D generation  
- Context length is set to 8,000 tokens  

3D Representation Strategy  
- Uses the OBJ file format for mesh representation  
- Quantizes vertex coordinates into 64 discrete bins per axis  
- Sorts vertices by z-y-x coordinates, from lowest to highest  
- Sorts faces by the lowest vertex indices for consistency  

Data Processing Pipeline  
- Filters meshes to a maximum of 500 faces for computational efficiency  
- Applies random rotations (0°, 90°, 180°, 270°) for data augmentation  
- Generates ~125k mesh variations from 31k base meshes  
- Uses Cap3D-generated captions for text descriptions  

>> Training Framework

Dataset Composition  
- 40% Mesh Generation tasks  
- 20% Mesh Understanding tasks  
- 40% General Conversation (UltraChat dataset)  
- 8x training turns for generation, 4x for understanding  

Training Configuration  
- Deployed on 32 A100 GPUs (for Nvidia, this is literally in-house)  
- 21,000 training iterations  
- Global batch size: 128  
- AdamW optimizer with a 1e-5 learning rate  
- 30-step warmup with cosine scheduling  
- Total training time: approximately 3 days (based on the paper)  

This research opens exciting possibilities for intuitive 3D content creation through natural language interaction. The future of digital design is conversational!
🔥
Mrdesigner14 John6666 roland0822 DatGG KadirErturk EdilCamil
🚀
John6666 casper911
👍
csabakecskemeti gauravpatil