Similar to T5, ByT5 was trained on the span-mask denoising task. However, 
since the model works directly on characters, the pretraining task is a bit 
different. Let's corrupt some characters of the 
input sentence "The dog chases a ball in the park." and ask ByT5 to predict them 
for us.
thon