
# Phi 2 5B Base

This is a base model based on Phi 2 (2.7B). This model has 5B parameters and is designed for further finetuning to add new knowledge while retaining existing expertise.

## Model Details

* Layers: 60
* Parameters: 4.98B
* Architecture: Phi 2

## Recommended Training Method

The recommended training method (inspired by LLaMA Pro 8B) is to freeze 0-32 layers and train layers 32-60 with new data to allow the model to learn new skills. You can do this by modifying LoRA code.
{"language": ["en"], "tags": ["phi"], "base_model": ["microsoft/phi-2"], "pipeline_tag": "text-generation"}
transformers safetensors phi text-generation custom_code en base_model:microsoft/phi-2 autotrain_compatible endpoints_compatible region:us