text
A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
new_line
new_line
text
As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
new_line
text
Dataset 
Dataset 
resource
https://huggingface.co/datasets/MM-UPD/MM-UPD
dataset
MM-UPD/MM-UPD
https://huggingface.co/datasets/MM-UPD/MM-UPD
new_line
text
Paper 
Paper 
resource
https://huggingface.co/papers/2403.20331
paper
2403.20331
https://huggingface.co/papers/2403.20331
Unsolvable Problem Detection: Evaluating Trustworthiness of Vision
  Language Models (2403.20331)
A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓

As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
Dataset https://huggingface.co/datasets/MM-UPD/MM-UPD
Paper https://huggingface.co/papers/2403.20331
👍
KingNish qubvel-hf AlekseiPravdin raincandy-u anldrms