MINGYISU commited on
Commit
8976043
·
1 Parent(s): 4b9fddb

fixed some issues

Browse files
Files changed (2) hide show
  1. results.csv +13 -13
  2. utils.py +1 -1
results.csv CHANGED
@@ -1,15 +1,15 @@
1
  Models,Model Size(B),Data Source,Overall,IND,OOD,Classification,VQA,Retrieval,Grounding
2
- CLIP,0.428,unk,37.8,37.1,38.7,42.8,9.1,53.0,51.8
3
- BLIP2,3.74,unk,25.2,25.3,25.1,27.0,4.2,33.9,47.0
4
- SigLIP,0.203,unk,34.8,32.3,38.0,40.3,8.4,31.6,59.5
5
- OpenCLIP,0.632,unk,39.7,39.3,40.2,47.8,10.9,52.3,53.3
6
- UniIR (BLIP_FF),unk,TIGER-Lab,42.8,44.7,40.4,42.1,15.0,60.1,62.2
7
- UniIR (CLIP_SF),unk,TIGER-Lab,44.7,47.1,41.7,44.3,16.2,61.8,65.3
8
- E5-V,8.36,unk,13.3,14.9,11.5,21.8,4.9,11.5,19.0
9
  Magiclens,unk,unk,27.8,31.0,23.7,38.8,8.3,35.4,26.0
10
- CLIP-FFT,0.428,TIGER-Lab,45.4,47.6,42.8,55.2,19.7,53.2,62.2
11
- OpenCLIP-FFT,0.632,TIGER-Lab,47.2,50.5,43.1,56.0,21.9,55.4,64.1
12
- VLM2Vec (Phi-3.5-V-FFT),4.15,TIGER-Lab,55.9,62.8,47.4,52.8,50.3,57.8,72.3
13
- VLM2Vec (Phi-3.5-V-LoRA),4.15,TIGER-Lab,60.1,66.5,52.0,54.8,54.9,62.3,79.5
14
- VLM2Vec (LLaVA-1.6-LoRA-LowRes),7.57,TIGER-Lab,55.0,61.0,47.5,54.7,50.3,56.2,64.0
15
- VLM2Vec (LLaVA-1.6-LoRA-HighRes),7.57,TIGER-Lab,62.9,67.5,57.1,61.2,49.9,67.4,86.1
 
1
  Models,Model Size(B),Data Source,Overall,IND,OOD,Classification,VQA,Retrieval,Grounding
2
+ CLIP,unk,unk,37.8,37.1,38.7,42.8,9.1,53.0,51.8
3
+ BLIP2,unk,unk,25.2,25.3,25.1,27.0,4.2,33.9,47.0
4
+ SigLIP,unk,unk,34.8,32.3,38.0,40.3,8.4,31.6,59.5
5
+ OpenCLIP,unk,unk,39.7,39.3,40.2,47.8,10.9,52.3,53.3
6
+ UniIR (BLIP_FF),unk,unk,42.8,44.7,40.4,42.1,15.0,60.1,62.2
7
+ UniIR (CLIP_SF),unk,unk,44.7,47.1,41.7,44.3,16.2,61.8,65.3
8
+ E5-V,unk,unk,13.3,14.9,11.5,21.8,4.9,11.5,19.0
9
  Magiclens,unk,unk,27.8,31.0,23.7,38.8,8.3,35.4,26.0
10
+ CLIP-FFT,unk,TIGER-Lab,45.4,47.6,42.8,55.2,19.7,53.2,62.2
11
+ OpenCLIP-FFT,unk,unk,47.2,50.5,43.1,56.0,21.9,55.4,64.1
12
+ VLM2Vec (Phi-3.5-V-FFT),unk,TIGER-Lab,55.9,62.8,47.4,52.8,50.3,57.8,72.3
13
+ VLM2Vec (Phi-3.5-V-LoRA),unk,TIGER-Lab,60.1,66.5,52.0,54.8,54.9,62.3,79.5
14
+ VLM2Vec (LLaVA-1.6-LoRA-LowRes),unk,TIGER-Lab,55.0,61.0,47.5,54.7,50.3,56.2,64.0
15
+ VLM2Vec (LLaVA-1.6-LoRA-HighRes),unk,TIGER-Lab,62.9,67.5,57.1,61.2,49.9,67.4,86.1
utils.py CHANGED
@@ -88,7 +88,7 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
88
  "Grounding": 50
89
  }
90
  ```
91
- Please send us an email at {CONTACT_EMAIL}, attaching the file. We will review your submission and update the leaderboard accordingly.
92
  """
93
 
94
  def get_df():
 
88
  "Grounding": 50
89
  }
90
  ```
91
+ Please send us an email at [email protected], attaching the file. We will review your submission and update the leaderboard accordingly.
92
  """
93
 
94
  def get_df():