
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# QuantumQuill-GPT2-Medium-Raw-300M

This model is a fine-tuned version gpt2-medium with trained on raw corpus,book,internet website crawl in related field : <br>
science knowledge, scientific fiction stories knowledge, computer knowledge(upcoming), game knowledge(upcoming), virtual reality technology knowledge(upcoming), 
mechanical engineering knowledge(upcoming),robotic knowledge(upcoming)

the main goal is give foundation knowledge on generating science fiction stories

## Model description

Trained not in bulk combined dataset but stage by stage for selecting checkpoint best model using git version controller
1. Stage 1 = 80 step on faizalnf1800/Science-OA-STM-Corpus
2. Stage 2 = 80 step on faizalnf1800/SciFi_Stories_Text_Corpus

Model may forget first trained dataset but i will order it the most foundation knowledge.

## Intended uses & limitations

generating science fiction stories in theme :
1. Science and Technology
2. Computer Science Technology
3. Virtual Reality Game
4. Game Element
5. Robotic
6. Mecha

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.3
- training_steps: 80

### Training results



### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["faizalnf1800/Science-OA-STM-Corpus", "faizalnf1800/SciFi_Stories_Text_Corpus"], "base_model": "gpt2-medium", "model-index": [{"name": "QuantumQuill-GPT2-Medium-Raw-300M", "results": []}]}
transformers tensorboard safetensors gpt2 text-generation generated_from_trainer dataset:faizalnf1800/Science-OA-STM-Corpus dataset:faizalnf1800/SciFi_Stories_Text_Corpus base_model:gpt2-medium license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us