Autotrain Zypher Chatbot with multiple ebooks
Subject: Seeking Insights on Zephyr Chatbot Training with Ebooks – Your Thoughts?Hi everyone,I’m currently working on training a Zypher chatbot using multiple ebooks and would love to get your insights or suggestions on my approach. Here’s what I’ve done so far:Base Model Choice: I’m usingHuggingFaceH4/zephyr-7b-betaas my foundation.Training Parameters: I’ve enabledmerge_adapterandauto_find_batch_sizefor efficiency.Training Data Format: My data is in atrain.csvfile, has one column ‘text’ and each row contains up to 2048 tokens from various books - serially from one page then the next.I’m curious about a couple of things:Would incorporating the book title in each row of my training data be beneficial for context, or might it introduce unwanted biases / contaminate?Are there any common pitfalls or challenges I should be aware of in this kind of training setup?Any advice or feedback based on your experiences would be greatly appreciated!Thanks in advance!Ujjwal Singh
2024-03-02T06:01:37Z
Incorporating book titles in your training data could provide valuable context for the chatbot, potentially improving response relevance. However, be cautious of potential biases. Common pitfalls include overfitting to specific book patterns, maintaining consistency across diverse sources, and ensuring data quality. Experimenting with training parameters and continuous evaluation are crucial for optimizing the chatbot’s performance.