[Announcement] Model Versioning: Upcoming changes to the model hub
Update: migration is now completed.TL;DRearly next week, we will migrate the models stored on thehuggingface.comodel hub. Accessing models from the library will be transparent and backward-compatible, however theprocess to upload models is going to be different. Please share your feedback!We host more and more of the community’s models which is awesome. To scale this sharing, we need to change the infra to both support more models, and unlock new powerful features.To that effect, we have rebuilt the storage backend that we use for models (currently S3), toour own git repos (using S3 as a git-lfs endpoint for large files), with one model = one repo.The benefits of this switch are:built-in versioning(I mean… it’s git. It’s pretty much what you use for versioning. Versioning in S3 has a ton a limitations)access control(will unlock private models, private datasets, etc)scalability(our usage of S3 to maintain lists of models was starting to bottleneck)Let’s dive in to the actual changes:I. On the websiteYou’ll now see a “Browse files and versions” tab or button on each model page. (design is not final, we’ll make it more prominent/streamlined in the near future)This is what this page looks like:Screenshot 2020-11-06 at 19.23.051239×1060 136 KBHere’s a link to check it out directly in a staging env:https://moon-preprod.huggingface.co/julien-c/EsperBERTo-small/tree/main(disabled now that migration is completed)The UX should look familiar and self-explanatory, but we’ll add more ML-specific features in the future (what cool feature ideas do you have for version control for Machine learning?)You can:see commit histories and diffs of changes made to any text file, like config.json:changes made by the HuggingFace team will be way clearer – we can perform updates to the models to ensure they work well with the library(ies) (you’ll be able to opt out from those changes)Large binary files are stored usinghttps://git-lfs.github.com/which is pretty standard now, and interoperable out of the box with gitAbility to update your text files, like your README.md model card, directly on the website!with instant previewII. In the transformers libraryWe are soliciting feedback on the PR to enable this new storage mode in thetransformerslibrary:https://github.com/huggingface/transformers/pull/8324This PR has two parts:1. changes to the file downloading codeused infrom_pretrained()methods to use the new file URLs.Large files are stored in an S3 bucket and served by Cloudfront so downloads should be as fast as they are right now.In addition, you now have a way to pin a specific version of a model, to a commit hash, tag or branch.For instance:tokenizer = AutoTokenizer.from_pretrained(
  "julien-c/EsperBERTo-small",
  revision="v2.0.1" # tag name, or branch name, or commit hash
)Finally, the networking code is more robust and doesn’t gobble up errors anymore, so in case you have trouble downloading a specific file you’ll know exactly why.2. changes to the model upload CLIto create a model repo then be able to git clone and git push to it.We are intentionally not wrappinggittoo much because we expect most model authors to be familiar with git (and possibly git-lfs), let us know if not the case.To create a repo:transformers-cli repo create your-model-nameThen you’ll get a repo url that you’ll be able to clone:git clone https://huggingface.co/username/your-model-name

# Then commit as usual
cd your-model-name
echo "hello" >> README.md
git add . && git commit -m "Update from $USER"A nice side effect of the new system on the upload side is that file uploading should be more robust for very large files (hello T5!) as git-lfs handles the networking code.By the way, again,every model is its own repo. So you can git clone any public model if you’d like:git clone https://huggingface.co/gpt2But you won’t be able to push unless it’s one of your models (or one of your orgs’).Again, please review this PR if possible:https://github.com/huggingface/transformers/pull/8324III. Backward compatibilityWe intend to merge the PR in transformers next Tuesday morning (November 10).Backward compatibility on model downloads is expected, because even though the new models will be stored in huggingface.co-hosted git repos, we will backport all file changes to S3 automatically.Model uploads using the current system won’t work anymore: you’ll need to upgrade your transformers installation to the next release,v3.5.0, or to build frommaster.Alternatively, in the next week or so we’ll add the ability to create a repo from the website directly so you’ll be able to push even without the transformers library.Please let us know of your feedback! We are super excited about this change, because it’s going to unlock really powerful features in the future.
2020-11-06T19:22:06Z
Awesome new feature!Can’t wait to test it; versioning is really great, especially for fine-tuned models (that can be improved over time).I would love to see an example of how a “tagged” version can be used. E.g. how can a “v2” tag of a model be used in Transformers then - with something like:from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("julien-c/EsperBERTo-small@v2") # specify commit/tagI really like the versioning concept. But how do you “sync” changes between themodel_cardsREADME.md and a specific tagged version of the model?E.g. I normally would open a PR for a model card README.md in the Transformers library. Then later I would update the model to a version 2, “tag” the old model with a “v1” tag and update the model card in Transformers library for version 2. How can I switch back to the version 1 model card, that belongs to the tagged model for v1 UX-wiseIt would be awesome to have a kind of version switcher (for tags) in a more prominant way
2020-11-06T19:31:43Z
Good point about an example of the syntax.You can do with the changes in the PR with:tokenizer = AutoTokenizer.from_pretrained(
  "julien-c/EsperBERTo-small",
  revision="v2" # tag name, or branch name, or commit hash
)
2020-11-06T19:36:01Z
On the model card question: you can push your model card to the model repo itself (that actually is not new, we’ve supported loading model cards from S3 for a while even if most users still push to the transformers repo)See this model for instance:https://huggingface.co/Helsinki-NLP/opus-mt-en-de/blob/main/README.mdThen a specific version of the README.md lives on a branch (then you can push to branchv1orfine-tune-experimentor whatever), or pinned at a specific tag (v1.0.1). Does this solve the use case you’re describing?
2020-11-06T20:09:13Z
This looks great!This last comment made it clearer to me.So basically a model is a repo and a version is a branch.I guess by default everything is on the “main” branch.So:a new model requires a new repo (are they auto-created?)then we just push files to a specific branchA few questions:Who is the repo owner?How is the repo named? Just the model name?
2020-11-06T20:29:17Z
Thanks for the clarification
2020-11-06T22:19:46Z
@stefan-it@borisI added examples of code and commands  directly to the initial post, let me know if it’s clearer.In short:you need to create a repo, then clone itthe repo’s ids are the same as today, i.e.user_name/model_nameororg/model_namedepending on whether you pass an organization flag. Access control rights are the same as now.
2020-11-06T23:14:51Z
This is great! As said by the others this is an excellent addition to keep track of model versions, particularly when you are finetuning your own. Now we can use commit messages to keep track of all model’s changes, too.Thinking about how this will work once people start using it, I’d like to raise two possible concerns. It seems that you can clone other people’s models because they are all public anyway. That is great, but possible concerns are:Will the branching be visually clear? In other words, will it be clear to users which is the original repo?In the search results, the original repo (or most downloaded repo?) should be shown at the top in case of forks, or at least there should be some clarity here. (Imagine that everyone clones bert-base to play with it or to finetune it, and suddenly there are hundreds of bert-bases.)Because of this possible increase in duplication, it might also be fruitful to bring back the discussion about required fields in model cards. I don’t know if that has been addressed yet, but perhaps this is a good time to introduce required fields like license and training data used. I’m aware that that’s not what this PR is about, but it seems related enough.
2020-11-08T04:17:42Z
This sounds and looks great! Definitely looking forward to private and versioned models.Curious why you chose to go with git-lfs overdvc? And, will this change mean git-lfs is an added requirement for publishing and downloading models?
2020-11-09T11:08:22Z
@setu4993There will be no additional requirement for downloading models, it’s still downloading individual files like it’s working now. To publish models, yes – git-lfs is now a requirement (hopefully very widespread and easy to install)Regarding DVC: we did investigate dvc a little bit, and discussed with some in the community. To my understanding, dvc has several “layers”, with the versioning layer being pretty close to (a slight more customizable) LFS in terms of features/concepts. The implementation of a hosting server for the file versioning part of dvc would probably be very similar to that of lfs.For the v1 we figured it was simpler to only do LFS, but we could support DVC-enabled “repos” at some point. What do you think?
2020-11-09T22:41:08Z
Looks really good! I’m excited to try it out.As someone who uses more models than he shares (I’ve uploaded … none) one of the things I’ve been thinking about how I could discover the “best” or “most appropriate” model from the growing family of models available via hugging face. To be honest, I tend to fall back onto BERT very often and don’t feel I have a great way to discover other models that may be better for my needs. I’m sure there are things I have missed!Would be awesome to have a way to search & compare models  against each other. E.g. all models trained on the same dataset, or all models trained on the same downstream task - sorted by a metric that I pick. Just an idea for the future
2020-11-10T12:35:28Z
Worked like a charm!Just uploaded a new model with this new approach:huggingface.codbmdz/electra-base-ukrainian-cased-discriminator at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.Awesome new featureThanks for implementing it
2020-11-10T19:05:34Z
Yes! Everything looks pretty straightforward! Congrats@julien-cand team!!
2020-11-11T06:19:51Z
Thanks for the clarification on requirements, makes sense.I haven’t used git-lfs directly before, but I do use dvc actively. I definitely see what you mean by versioning is only one part of dvc. I’ve come to appreciate versioning with it a lot more than I thought I would, and the simplicity of being able to use it with S3 / external data stores easily, which would be the case here as well. On the negative side, it wouldn’t have as visibility into the files that are already uploaded or their sizes (as in the EsperBERTo example image above).Good to know it is released, looking forward to trying it out soon!
2020-11-11T10:56:36Z
i am try to push from colab but getting this errorfatal: could not read Username for ‘https://huggingface.co’: No such device or addressimage1118×634 22.3 KBany ideas?
2020-11-11T11:35:39Z
Yes: looks like a limitation of Colab. You can add your token when cloning, like this:git clone https://:$TOKEN@huggingface.co/user/model(or update your remote to this)Let me know if this works!
2020-11-11T11:47:45Z
i am getting any error like thisremote: Unauthorizedfatal: Authentication failed for ‘https://:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX@huggingface.co/manishiitg/longformer-recruit-qa-v2/’i have replaced the token with “XXX”
2020-11-11T12:36:31Z
can you try adding your username in front of:? Otherwise you can also userusername:passwordinstead of the token.Let me know if this works
2020-11-11T12:54:05Z
ok it worked finally
No date available
No reply text available