dataset:bigcode/starcoderdata dataset:nampdn-ai/tiny-codes dataset:garage-bAInd/Open-Platypus dataset:0x70DA/stackoverflow-chat-data dataset:TnT/Multi_CodeNet4Repair license:apache-2.0 region:us
---
license: apache-2.0
datasets:
- bigcode/starcoderdata
- nampdn-ai/tiny-codes
- garage-bAInd/Open-Platypus
- 0x70DA/stackoverflow-chat-data
- TnT/Multi_CodeNet4Repair
---

# RWKV world 32k model fooocus on coding ability.
This model finetuned from RWKV world 7B  with context 32k, focus on multi turn coding.

# Trainning details
4*A800 27hours with 1B tokens
![image.png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/Tot0WkvPvwkyyWy_4-E3-.png)


# datasets
mainly tiny codes and add a lots of long context multi turn datasets. 
only finetuend in User: xxx\n\nAssistant: xxx\n format

# Showcases
![09713ffd8b5c21a525065a50964dd5f.jpg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_pmGc7y1j2a5i_GCWuq7D.jpeg)

# other
if using RWKV runner to run this model, need to wait for updates in chat mode, as default chat using Question: xxx\n\nAnswer: xxx and have a default system prompt so far.

completions mode is more suitable for this model

# How

easiest way to use this model by using RWKV runner  https://github.com/josStorer/RWKV-Runner, now RWKV runner support AMD GPUs in config webGPU

or you can use https://github.com/cgisky1980/ai00_rwkv_server as well

BTW, runner support  GPT-format like API


