
# Stambecco 🦌: Italian Instruction-following LLaMA Model

Stambecco is a Italian Instruction-following model based on the [LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/) model.
It comes in two versions: 7b and 13b parameters.

It is trained on an Italian version of the [GPT-4-LLM](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM) dataset, a dataset of `GPT-4` generated instruction-following data.

This repo contains a low-rank adapter for LLaMA-7b.

For more information, please visit [the project's website](https://github.com/mchl-labs/stambecco).

### 💪 Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 8
- gradient_accumulation_steps: 32
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
- mixed_precision_training: Native AMP
- LoRA R: 8
- LoRA target modules: q_proj, v_proj


## Intended uses & limitations

**Usage and License Notices**: Same as [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca), Stambecco is intended and licensed for research use only. The models should not be used outside of research purposes.

Please note that it is highly possible that the model output contains biased, conspiracist, offensive, or otherwise inappropriate and potentially harmful content.
The model is intended for **research purposes only** and should be used with caution at your own risk. **Production usage is not allowed.**
{"language": ["it", "en"], "license": "cc-by-nc-nd-4.0", "library_name": "transformers"}
transformers it en license:cc-by-nc-nd-4.0 endpoints_compatible has_space region:us
TAGS
#transformers #it #en #license-cc-by-nc-nd-4.0 #endpoints_compatible #has_space #region-us 


# Stambecco : Italian Instruction-following LLaMA Model

Stambecco is a Italian Instruction-following model based on the LLaMA model.
It comes in two versions: 7b and 13b parameters.

It is trained on an Italian version of the GPT-4-LLM dataset, a dataset of 'GPT-4' generated instruction-following data.

This repo contains a low-rank adapter for LLaMA-7b.

For more information, please visit the project's website.

###  Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 8
- gradient_accumulation_steps: 32
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
- mixed_precision_training: Native AMP
- LoRA R: 8
- LoRA target modules: q_proj, v_proj


## Intended uses & limitations

Usage and License Notices: Same as Stanford Alpaca, Stambecco is intended and licensed for research use only. The models should not be used outside of research purposes.

Please note that it is highly possible that the model output contains biased, conspiracist, offensive, or otherwise inappropriate and potentially harmful content.
The model is intended for research purposes only and should be used with caution at your own risk. Production usage is not allowed.
# Stambecco : Italian Instruction-following LLaMA Model

Stambecco is a Italian Instruction-following model based on the LLaMA model.
It comes in two versions: 7b and 13b parameters.

It is trained on an Italian version of the GPT-4-LLM dataset, a dataset of 'GPT-4' generated instruction-following data.

This repo contains a low-rank adapter for LLaMA-7b.

For more information, please visit the project's website. ###  Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 8
- gradient_accumulation_steps: 32
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
- mixed_precision_training: Native AMP
- LoRA R: 8
- LoRA target modules: q_proj, v_proj ## Intended uses & limitations

Usage and License Notices: Same as Stanford Alpaca, Stambecco is intended and licensed for research use only. The models should not be used outside of research purposes.

Please note that it is highly possible that the model output contains biased, conspiracist, offensive, or otherwise inappropriate and potentially harmful content.
The model is intended for research purposes only and should be used with caution at your own risk. Production usage is not allowed.
TAGS
#transformers #it #en #license-cc-by-nc-nd-4.0 #endpoints_compatible #has_space #region-us 
 # Stambecco : Italian Instruction-following LLaMA Model

Stambecco is a Italian Instruction-following model based on the LLaMA model.
It comes in two versions: 7b and 13b parameters.

It is trained on an Italian version of the GPT-4-LLM dataset, a dataset of 'GPT-4' generated instruction-following data.

This repo contains a low-rank adapter for LLaMA-7b.

For more information, please visit the project's website. ###  Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 8
- gradient_accumulation_steps: 32
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
- mixed_precision_training: Native AMP
- LoRA R: 8
- LoRA target modules: q_proj, v_proj ## Intended uses & limitations

Usage and License Notices: Same as Stanford Alpaca, Stambecco is intended and licensed for research use only. The models should not be used outside of research purposes.

Please note that it is highly possible that the model output contains biased, conspiracist, offensive, or otherwise inappropriate and potentially harmful content.
The model is intended for research purposes only and should be used with caution at your own risk. Production usage is not allowed.
TAGS
#transformers #it #en #license-cc-by-nc-nd-4.0 #endpoints_compatible #has_space #region-us 
# Stambecco : Italian Instruction-following LLaMA Model

Stambecco is a Italian Instruction-following model based on the LLaMA model.
It comes in two versions: 7b and 13b parameters.

It is trained on an Italian version of the GPT-4-LLM dataset, a dataset of 'GPT-4' generated instruction-following data.

This repo contains a low-rank adapter for LLaMA-7b.

For more information, please visit the project's website.###  Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 8
- gradient_accumulation_steps: 32
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
- mixed_precision_training: Native AMP
- LoRA R: 8
- LoRA target modules: q_proj, v_proj## Intended uses & limitations

Usage and License Notices: Same as Stanford Alpaca, Stambecco is intended and licensed for research use only. The models should not be used outside of research purposes.

Please note that it is highly possible that the model output contains biased, conspiracist, offensive, or otherwise inappropriate and potentially harmful content.
The model is intended for research purposes only and should be used with caution at your own risk. Production usage is not allowed.