
# BEE-spoke-data/beecoder-220M-python




This is `BEE-spoke-data/smol_llama-220M-GQA` fine-tuned for code generation on:

- filtered version of stack-smol-XL
- deduped version of 'algebraic stack' from proof-pile-2
- cleaned and deduped pypi (last dataset)

This model (and the base model) were both trained using ctx length 2048.  

## examples

> Example script for inference testing: [here](https://gist.github.com/pszemraj/c7738f664a64b935a558974d23a7aa8c)

It has its limitations at 220M, but seems decent for single-line or docstring generation, and/or being used for speculative decoding for such purposes.



![image/png](https://cdn-uploads.huggingface.co/production/uploads/60bccec062080d33f875cd0c/bLrtpr7Vi_MPvtF7mozDN.png)

The screenshot is on CPU on a laptop.

---
{"language": ["en"], "license": "apache-2.0", "tags": ["python", "codegen", "markdown", "smol_llama"], "datasets": ["BEE-spoke-data/pypi_clean-deduped", "bigcode/the-stack-smol-xl", "EleutherAI/proof-pile-2"], "metrics": ["accuracy"], "base_model": "BEE-spoke-data/smol_llama-220M-GQA", "inference": {"parameters": {"max_new_tokens": 64, "min_new_tokens": 8, "do_sample": true, "epsilon_cutoff": 0.0008, "temperature": 0.3, "top_p": 0.9, "repetition_penalty": 1.02, "no_repeat_ngram_size": 8, "renormalize_logits": true}}, "widget": [{"text": "def add_numbers(a, b):\n    return\n", "example_title": "Add Numbers Function"}, {"text": "class Car:\n    def __init__(self, make, model):\n        self.make = make\n        self.model = model\n\n    def display_car(self):\n", "example_title": "Car Class"}, {"text": "import pandas as pd\ndata = {'Name': ['Tom', 'Nick', 'John'], 'Age': [20, 21, 19]}\ndf = pd.DataFrame(data).convert_dtypes()\n# eda\n", "example_title": "Pandas DataFrame"}, {"text": "def factorial(n):\n    if n == 0:\n        return 1\n    else:\n", "example_title": "Factorial Function"}, {"text": "def fibonacci(n):\n    if n <= 0:\n        raise ValueError(\"Incorrect input\")\n    elif n == 1:\n        return 0\n    elif n == 2:\n        return 1\n    else:\n", "example_title": "Fibonacci Function"}, {"text": "import matplotlib.pyplot as plt\nimport numpy as np\nx = np.linspace(0, 10, 100)\n# simple plot\n", "example_title": "Matplotlib Plot"}, {"text": "def reverse_string(s:str) -> str:\n    return\n", "example_title": "Reverse String Function"}, {"text": "def is_palindrome(word:str) -> bool:\n    return\n", "example_title": "Palindrome Function"}, {"text": "def bubble_sort(lst: list):\n    n = len(lst)\n    for i in range(n):\n        for j in range(0, n-i-1):\n", "example_title": "Bubble Sort Function"}, {"text": "def binary_search(arr, low, high, x):\n    if high >= low:\n        mid = (high + low) // 2\n        if arr[mid] == x:\n            return mid\n        elif arr[mid] > x:\n", "example_title": "Binary Search Function"}], "pipeline_tag": "text-generation"}
transformers safetensors llama text-generation python codegen markdown smol_llama en dataset:BEE-spoke-data/pypi_clean-deduped dataset:bigcode/the-stack-smol-xl dataset:EleutherAI/proof-pile-2 base_model:BEE-spoke-data/smol_llama-220M-GQA license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us