Language Model Skips entire Sentence
I am messing around with the transformers implementation of HuggingFace to translate strings from english to german. In particular, I try to the following Code.import torch
Test = True

### NLLB GPU
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

print(torch.cuda.mem_get_info())
device = "cuda:0" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M").to(device)
model.eval()
print(torch.cuda.mem_get_info())

def translationPipeline(text):
    input_ids = tokenizer.encode(text, return_tensors="pt").to(device)
    with torch.no_grad():
        outputs = model.generate(input_ids,forced_bos_token_id=tokenizer.lang_code_to_id["deu_Latn"], max_length = 10000)
    decoded = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
    return decoded

# c = df_text.iloc[1,:].body.apply(translationPipeline)
text = "Tech firms’ shares fell after reports that China had barred employees at government-backed agencies and state companies from using iPhones, widening a ban applying to some government staff. Shares in Apple sank by more than 3% on Thursday; its market value has dropped by almost $200bn in the past two days."
text = text
print(tokenizer.encode(text, return_tensors="pt").shape)
with torch.no_grad():
    c = translationPipeline(text)
    
print(torch.cuda.mem_get_info())This returns the following stringDie Aktien von Techfirmen gingen nach Berichten zurück, dass China Mitarbeitern von staatlich unterstützten Agenturen und staatlichen Unternehmen das Verwenden von iPhones verboten hatte und das Verbot für einige Regierungsmitarbeiter erweitert hatte.Therefore, the first input sentence is translated really well. However, the second input sentence is just ignored completely. What is happening there?