dataset:keirp/hungarian_national_hs_finals_exam region:us
---
datasets:
- keirp/hungarian_national_hs_finals_exam
---

Raw outputs from running Qwen-72B and deepseek-llm-67b-base on [keirp/hungarian_national_hs_finals_exam](https://huggingface.co/datasets/keirp/hungarian_national_hs_finals_exam).
For more details about the test itself, check that repo.

## Notes
Tests were run with the default settings from the repo, i.e temperature=0.1, top_p=0.95, prompt="few_shot" in float16 on a 4xA100 node with tp=4.

The only possibly relevant change was setting max_model_len=3072, though I don't think it matters since the few shot prompt is ~700 tokens and max_tokens=1024.

## Model grading / results

Keiran was nice enough to do the grading for me. Results can be found in his [twitter post](https://twitter.com/keirp1/status/1730706870149509189).