Hugging Face's logo

adami1
/
405M_TIES-merge_pile_300B_into_slimp_300B_from_pile_replay5_density-0.5

Text Generation
Transformers
Safetensors
gpt_neox
Merge
mergekit
lazymergekit
btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch
text-generation-inference
Inference Endpoints
Model card Files Community

405M_TIES-merge_pile_300B_into_slimp_300B_from_pile_replay5_density-0.5

405M_TIES-merge_pile_300B_into_slimp_300B_from_pile_replay5_density-0.5 is a merge of the following models using mergekit:

🧩 Configuration

```yamlmodels:

  • model: btherien/Model_-410M_It_-132366_Tr_-slim-pajama-300B-replay5_finetune

    no parameters necessary for base model

  • model: btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch parameters: density: 0.5 weight: 1.0 merge_method: ties base_model: btherien/Model_-410M_It_-132366_Tr_-slim-pajama-300B-replay5_finetune parameters: normalize: true dtype: float16```
Downloads last month
3
Safetensors
Model size
405M params
Tensor type
FP16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.