# This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (https://github.com/Emplocity/owca) using LoRa

Training params:<br/>
```
MICRO_BATCH_SIZE = 64
BATCH_SIZE = 128
GRADIENT_ACCUMULATION_STEPS = BATCH_SIZE // MICRO_BATCH_SIZE
EPOCHS = 3
LEARNING_RATE = 2e-5  
CUTOFF_LEN = 256  
LORA_R = 4
LORA_ALPHA = 16
LORA_DROPOUT = 0.05
warmup_steps=100
fp16=True
```

[See inference demo](łowca.ipynb)
{"language": ["pl"], "license": "unlicense", "datasets": ["emplocity/owca"], "pipeline_tag": "text-generation"}
text-generation pl dataset:emplocity/owca license:unlicense region:us
TAGS
#text-generation #pl #dataset-emplocity/owca #license-unlicense #region-us 

# This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (URL using LoRa

Training params:<br/>


See inference demo
# This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (URL using LoRa

Training params:<br/>


See inference demo
TAGS
#text-generation #pl #dataset-emplocity/owca #license-unlicense #region-us 
 # This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (URL using LoRa

Training params:<br/>


See inference demo
TAGS
#text-generation #pl #dataset-emplocity/owca #license-unlicense #region-us 
# This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (URL using LoRa

Training params:<br/>


See inference demo