Mike Burnham's picture
3 1

Mike Burnham

mlburnham

AI & ML interests

None yet

Recent Activity

updated a dataset 5 days ago
mlburnham/polnli_hypothesis_stability
published a dataset 5 days ago
mlburnham/polnli_hypothesis_stability
updated a collection 11 days ago
Political DEBATE
View all activity

Organizations

None yet

mlburnham's activity

New activity in answerdotai/ModernBERT-base 13 days ago
replied to MoritzLaurer's post 20 days ago
view reply

Any plans to train a version of your zero-shot models on ModernBERT? I'm finding that ModernBERT is a huge boost in speed, and a slight drop in performance vs. DeBERTa when I tune it. Not sure if the performance drop is because your zero-shot models were such a strong foundation for transfer learning, or the strength of DeBERTa architecture on NLI.

reacted to MoritzLaurer's post with đź‘Ť 28 days ago
view post
Post
2603
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!

This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D

Congrats @answerdotai , @LightOnIO and collaborators like @tomaarsen !

Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
·
New activity in mlburnham/Pol_NLI 5 months ago