

The first sequence, the "context" used for the question, has all its tokens represented by a 0, whereas the second
sequence, corresponding to the "question", has all its tokens represented by a 1.
Some models, like [XLNetModel] use an additional token represented by a 2.
transfer learning
A technique that involves taking a pretrained model and adapting it to a dataset specific to your task. Instead of training a model from scratch, you can leverage knowledge obtained from an existing model as a starting point. This speeds up the learning process and reduces the amount of training data needed.
transformer
Self-attention based deep learning model architecture.
U
unsupervised learning
A form of model training in which data provided to the model is not labeled. Unsupervised learning techniques leverage statistical information of the data distribution to find patterns useful for the task at hand.
Z
Zero Redundancy Optimizer (ZeRO)
Parallelism technique which performs sharding of the tensors somewhat similar to TensorParallel, 
except the whole tensor gets reconstructed in time for a forward or backward computation, therefore the model doesn't need 
to be modified. This method also supports various offloading techniques to compensate for limited GPU memory. 
Learn more about ZeRO here.