A few other goals:

Expose the models' internals as consistently as possible:

We give access, using a single API, to the full hidden-states and attention weights.

The preprocessing classes and base model APIs are standardized to easily switch between models.

Incorporate a subjective selection of promising tools for fine-tuning and investigating these models:

A simple and consistent way to add new tokens to the vocabulary and embeddings for fine-tuning.

Simple ways to mask and prune Transformer heads.