|
--- |
|
license: mit |
|
datasets: |
|
- karpathy/tiny_shakespeare |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# Bad GPT |
|
|
|
Based on the [Let's build GPT](https://www.youtube.com/watch?v=kCc8FmEb1nY) video from Andrej Karpathy. |
|
|
|
This is just an attempt to recreate the transformer Andrej made in his video with the goal of learning more about torch, transformers, and neural networks in general. |
|
|
|
To run, make sure `python` `3.10` and `poetry` are installed. You can then run `poetry install` to get the dependencies (it's just torch and numpy). |
|
|
|
Finally, you can run the code with `poetry run python ./main.py` |
|
|
|
Note that the first run will train the model and then save the trained weights to `model.pth`. Subsequent runs will load these weights. |
