Update constants.py
Browse files- constants.py +6 -1
constants.py
CHANGED
@@ -52,7 +52,12 @@ SUBMIT_INTRODUCTION = """# Submit on SEED Benchmark Introduction
|
|
52 |
4. For the evaluation dimension, you can choose "All/Image/Video" for SEED-Bench-1 and "L1/L2/L3" for SEED-Bench-2, and the results of dimensions that are not evaluated will be set to zero.
|
53 |
5. After clicking 'Submit Eval', you can click 'Refresh' to obtain the latest result in the leaderboard.
|
54 |
|
55 |
-
|
|
|
|
|
|
|
|
|
|
|
56 |
|
57 |
## Submit Example
|
58 |
For example on SEED-Bench-1, if you want to upload InstructBLIP's result in the leaderboard, you need to:
|
|
|
52 |
4. For the evaluation dimension, you can choose "All/Image/Video" for SEED-Bench-1 and "L1/L2/L3" for SEED-Bench-2, and the results of dimensions that are not evaluated will be set to zero.
|
53 |
5. After clicking 'Submit Eval', you can click 'Refresh' to obtain the latest result in the leaderboard.
|
54 |
|
55 |
+
|
56 |
+
Note: The format of the submitted json file is a dict for each line. This dict contains two keys: question_id and prediction. Specific examples are as follows:
|
57 |
+
```shell
|
58 |
+
{"question_id": "5_0", "prediction": "B"}
|
59 |
+
{"question_id": "3_0", "prediction": "B"}
|
60 |
+
```
|
61 |
|
62 |
## Submit Example
|
63 |
For example on SEED-Bench-1, if you want to upload InstructBLIP's result in the leaderboard, you need to:
|