Cannot force Llama-2 to produce the desired spelling of a word
I’ve been fine-tuning Llama-2 using LoRA. The task is to classify the title into correct categories/extract relevant named entities.However, when I ran the inference, Llama-2 always produces very unstable spellings although it was trained on another spelling.Example 1:Input: L’Occitane Premium Hand CreamTraining data:Beauty products | HandCreamLlama-2 output:Beauty products | HandCrémeExample 2:Input: J&J Talcum PowderTraining data:Brand:J&JLlama-2 output:Brand:Johnson & JohnsonExample 3:Input: Bluetooth Keyboard for Microsoft Surface 2022Training data:Electronics |Tablet| AccessoriesLlama-2 output:Electronics |Tablets| AccessariesDo you know how to force Llama-2 to stay faithful to the input texts’ spellings?