andthattoo commited on
Commit
a9cc635
·
verified ·
1 Parent(s): 80a328f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -228,3 +228,13 @@ and the MMLU-Pro and DPAB results:
228
  | DPAB (Pythonic, Strict) | 30.0 | 51.0 |
229
 
230
  **\*Note:** The model tends to use Pythonic function calling for a lot of the test cases in STEM-related fields (math, physics, chemistry, etc.) in the MMLU-Pro benchmark, which isn't captured by the evaluation framework and scripts provided in their [Github repository](https://github.com/TIGER-AI-Lab/MMLU-Pro/tree/main). We haven't modified the script for evaluation, and leave it for the future iterations of this model. However, by performing qualitative analysis on the model responses, we suspect that the model's score will increase instead of suffering a ~3% decrease.
 
 
 
 
 
 
 
 
 
 
 
228
  | DPAB (Pythonic, Strict) | 30.0 | 51.0 |
229
 
230
  **\*Note:** The model tends to use Pythonic function calling for a lot of the test cases in STEM-related fields (math, physics, chemistry, etc.) in the MMLU-Pro benchmark, which isn't captured by the evaluation framework and scripts provided in their [Github repository](https://github.com/TIGER-AI-Lab/MMLU-Pro/tree/main). We haven't modified the script for evaluation, and leave it for the future iterations of this model. However, by performing qualitative analysis on the model responses, we suspect that the model's score will increase instead of suffering a ~3% decrease.
231
+
232
+ #### Citation
233
+
234
+ ```
235
+ @misc{Dria-Agent-a,
236
+ url={https://huggingface.co/blog/andthattoo/dria-agent-a},
237
+ title={Dria-Agent-a},
238
+ author={"andthattoo", "Atakan Tekparmak"}
239
+ }
240
+ ```