File size: 740 Bytes
af1a5ca 0e5be5a af1a5ca 3c8279d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
---
license: mit
datasets:
- karpathy/tiny_shakespeare
language:
- en
pipeline_tag: text-generation
---
# Bad GPT
Based on the [Let's build GPT](https://www.youtube.com/watch?v=kCc8FmEb1nY) video from Andrej Karpathy.
This is just an attempt to recreate the transformer Andrej made in his video with the goal of learning more about torch, transformers, and neural networks in general.
To run, make sure `python` `3.10` and `poetry` are installed. You can then run `poetry install` to get the dependencies (it's just torch and numpy).
Finally, you can run the code with `poetry run python ./main.py`
Note that the first run will train the model and then save the trained weights to `model.pth`. Subsequent runs will load these weights. |