GPT2 for German poetry generation
GPT2 for German poetry generationFor this project, I propose to use a pretrained GPT2 model in German and fine-tune it to learn German poetry.ModelA GPT2 model, pretrained in German can be found here:dbmdz/german-gpt2 · Hugging FaceDatasetsThe model can be fine-tuned on the publicly available “Faust” dataset:mobverdb/faust.txt at master · martinth/mobverdb · GitHubAvailable training scriptsA training script to fine-tune a GPT2 model in Flax is availablehere(Optional) Desired project outcomeThe desired project output is a GPT2 model that can generate sensible German poetry. This can be showcased directly on the hub or with a streamlit app.(Optional) ChallengesThe data is written as a dialogue with the speakers names"MEPHISTOPHELES:"before every paragraph. One might need to remove such data for better model quality.(Optional) Links to read uponThere are lots of English GPT2 models for poetry:Generating an Edgar Allan Poe-Styled Poem Using GPT-2 | by Scott Duda | Mediumhttps://www.gwern.net/GPT-2OpenAI’s new language AI is available to try yourself - Vox