
# harry-GPTter

harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of [Google Colaboratory](https://colab.research.google.com/), the model learnt to generate coherent and grammatically correct sentences.

- Code and more information in the [GitHub Repository](https://github.com/ShawnLJW/harry-GPTter) 
- Download the [weights](https://huggingface.co/ShawnLJW/harry-GPTter/resolve/main/checkpoint.pt)

## Text Generation with harry-GPTter
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
>
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
>
> “I’ll see you, Professor Lockhart,” said Hermione, “but so surely now to have solid on it out of her whole bed! You’re thinking —
>
> “Oh hello the unconscious!”
>
> “And now blimey,” said Harry, “it was a very serious for an enormous mother. ...”

## Model Details

harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.

The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.

This model was built for learning purposes. You can probably get better performance by finetuning a pre-trained model.
{"language": ["en"], "pipeline_tag": "text-generation"}
text-generation en region:us
TAGS
#text-generation #en #region-us 


# harry-GPTter

harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of Google Colaboratory, the model learnt to generate coherent and grammatically correct sentences.

- Code and more information in the GitHub Repository 
- Download the weights

## Text Generation with harry-GPTter
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
>
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
>
> “I’ll see you, Professor Lockhart,” said Hermione, “but so surely now to have solid on it out of her whole bed! You’re thinking —
>
> “Oh hello the unconscious!”
>
> “And now blimey,” said Harry, “it was a very serious for an enormous mother. ...”

## Model Details

harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.

The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.

This model was built for learning purposes. You can probably get better performance by finetuning a pre-trained model.
# harry-GPTter

harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of Google Colaboratory, the model learnt to generate coherent and grammatically correct sentences.

- Code and more information in the GitHub Repository 
- Download the weights ## Text Generation with harry-GPTter
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
>
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
>
> “I’ll see you, Professor Lockhart,” said Hermione, “but so surely now to have solid on it out of her whole bed! You’re thinking —
>
> “Oh hello the unconscious!”
>
> “And now blimey,” said Harry, “it was a very serious for an enormous mother. ...” ## Model Details

harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.

The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.

This model was built for learning purposes. You can probably get better performance by finetuning a pre-trained model.
TAGS
#text-generation #en #region-us 
 # harry-GPTter

harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of Google Colaboratory, the model learnt to generate coherent and grammatically correct sentences.

- Code and more information in the GitHub Repository 
- Download the weights ## Text Generation with harry-GPTter
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
>
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
>
> “I’ll see you, Professor Lockhart,” said Hermione, “but so surely now to have solid on it out of her whole bed! You’re thinking —
>
> “Oh hello the unconscious!”
>
> “And now blimey,” said Harry, “it was a very serious for an enormous mother. ...” ## Model Details

harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.

The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.

This model was built for learning purposes. You can probably get better performance by finetuning a pre-trained model.
TAGS
#text-generation #en #region-us 
# harry-GPTter

harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of Google Colaboratory, the model learnt to generate coherent and grammatically correct sentences.

- Code and more information in the GitHub Repository 
- Download the weights## Text Generation with harry-GPTter
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
>
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
>
> “I’ll see you, Professor Lockhart,” said Hermione, “but so surely now to have solid on it out of her whole bed! You’re thinking —
>
> “Oh hello the unconscious!”
>
> “And now blimey,” said Harry, “it was a very serious for an enormous mother. ...”## Model Details

harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.

The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.

This model was built for learning purposes. You can probably get better performance by finetuning a pre-trained model.