# GPT-2 (125M) 4k tokens

Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. 
Weights are included and it follows Karpathy's nanoGPT implementation.
The model has been trained for ~1 million iterations with increasing batch size, ending at 32k.
The final loss is 3.9 which is probably due to 768 embedding size.
{}
transformers pytorch safetensors gpt2 text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

# GPT-2 (125M) 4k tokens

Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. 
Weights are included and it follows Karpathy's nanoGPT implementation.
The model has been trained for ~1 million iterations with increasing batch size, ending at 32k.
The final loss is 3.9 which is probably due to 768 embedding size.
# GPT-2 (125M) 4k tokens

Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. 
Weights are included and it follows Karpathy's nanoGPT implementation.
The model has been trained for ~1 million iterations with increasing batch size, ending at 32k.
The final loss is 3.9 which is probably due to 768 embedding size.
TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # GPT-2 (125M) 4k tokens

Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. 
Weights are included and it follows Karpathy's nanoGPT implementation.
The model has been trained for ~1 million iterations with increasing batch size, ending at 32k.
The final loss is 3.9 which is probably due to 768 embedding size.
TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# GPT-2 (125M) 4k tokens

Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. 
Weights are included and it follows Karpathy's nanoGPT implementation.
The model has been trained for ~1 million iterations with increasing batch size, ending at 32k.
The final loss is 3.9 which is probably due to 768 embedding size.