

# Dataset Card for ConflcitQA
## Dataset Description
- **Repository:** https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict
- **Paper:** https://arxiv.org/abs/2305.13300
- **Point of Contact:** Point of Contact: [Jian Xie](mailto:jianx0321@gmail.com)
## Citation
If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries.
```bib
@article{Xie2023KnowledgeConflict,
  title={Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts},
  author={Xie, Jian and Zhang, Kai and Chen, Jiangjie and Lou, Renze and Su, Yu},
  journal={arXiv preprint arXiv:2305.13300},
  year={2023}
}
```

# ConflcitQA

We provide conflictQA-[based large language model], which utilizes large language models guided parametric memory.

```json
{"question": "What is George Rankin's occupation?", "popularity": 142, "ground_truth": ["politician", "political leader", "political figure", "polit.", "pol"], "memory_answer": "George Rankin's occupation is a professional photographer.", "parametric_memory": "As a professional photographer, George Rankin...", "counter_answer": "George Rankin's occupation is political figure.", "counter_memory": "George Rankin has been actively involved in politics for over a decade...", "parametric_memory_aligned_evidence": "George Rankin has a website showcasing his photography portfolio...", "counter_memory_aligned_evidence": "George Rankin Major General George James Rankin..."}
```

```python
#loading dataset
from datasets import load_dataset
# you can choose dataset "ConflictQA-popQA-[PLACEHOLDER]", and the [PLACEHOLDER] is in ["chatgpt","gpt4","palm2","llama2-7b","llama2-70b","qwen7b","vicuna7b","vicuna33b"].
dataset = load_dataset("osunlp/ConflictQA",'ConflictQA-popQA-chatgpt')
```

# Data Fields
- "question": The question in natural language
- "popularity": The monthly page views on Wikipedia for the given question
- "ground_truth": The factual answer to the question, which may include multiple possible answers
- "memory_answer": The answer provided by the LLM to the question
- "parametric_memory": The supportive evidence from LLM's parametric memory for the answer
- "counter_answer": The answer contradicting the "memory_answer"
- "counter_memory": The generation-based evidence supporting the counter_answer
- "parametric_memory_aligned_evidence": Additional evidence supporting the "memory_answer", which could be generated or derived from Wikipedia/human annotation
- "counter_memory_aligned_evidence": Additional evidence supporting the "counter_answer", either generated or sourced from Wikipedia/human annotation



task_categories:question-answering size_categories:10K<n<100K language:en license:apache-2.0 arxiv:2305.13300 region:us
{"language": ["en"], "license": "apache-2.0", "size_categories": ["10K<n<100K"], "task_categories": ["question-answering"], "pretty_name": "conflictQA", "configs": [{"config_name": "ConflictQA-popQA-chatgpt", "data_files": [{"split": "test", "path": "./conflictQA-popQA-chatgpt.json"}]}]}
TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #arxiv-2305.13300 #region-us 



# Dataset Card for ConflcitQA
## Dataset Description
- Repository: URL
- Paper: URL
- Point of Contact: Point of Contact: Jian Xie
If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries.


# ConflcitQA

We provide conflictQA-[based large language model], which utilizes large language models guided parametric memory.





# Data Fields
- "question": The question in natural language
- "popularity": The monthly page views on Wikipedia for the given question
- "ground_truth": The factual answer to the question, which may include multiple possible answers
- "memory_answer": The answer provided by the LLM to the question
- "parametric_memory": The supportive evidence from LLM's parametric memory for the answer
- "counter_answer": The answer contradicting the "memory_answer"
- "counter_memory": The generation-based evidence supporting the counter_answer
- "parametric_memory_aligned_evidence": Additional evidence supporting the "memory_answer", which could be generated or derived from Wikipedia/human annotation
- "counter_memory_aligned_evidence": Additional evidence supporting the "counter_answer", either generated or sourced from Wikipedia/human annotation



# Dataset Card for ConflcitQA ## Dataset Description
- Repository: URL
- Paper: URL
- Point of Contact: Point of Contact: Jian Xie
If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries. # ConflcitQA

We provide conflictQA-[based large language model], which utilizes large language models guided parametric memory. # Data Fields
- "question": The question in natural language
- "popularity": The monthly page views on Wikipedia for the given question
- "ground_truth": The factual answer to the question, which may include multiple possible answers
- "memory_answer": The answer provided by the LLM to the question
- "parametric_memory": The supportive evidence from LLM's parametric memory for the answer
- "counter_answer": The answer contradicting the "memory_answer"
- "counter_memory": The generation-based evidence supporting the counter_answer
- "parametric_memory_aligned_evidence": Additional evidence supporting the "memory_answer", which could be generated or derived from Wikipedia/human annotation
- "counter_memory_aligned_evidence": Additional evidence supporting the "counter_answer", either generated or sourced from Wikipedia/human annotation
TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #arxiv-2305.13300 #region-us 
 # Dataset Card for ConflcitQA ## Dataset Description
- Repository: URL
- Paper: URL
- Point of Contact: Point of Contact: Jian Xie
If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries. # ConflcitQA

We provide conflictQA-[based large language model], which utilizes large language models guided parametric memory. # Data Fields
- "question": The question in natural language
- "popularity": The monthly page views on Wikipedia for the given question
- "ground_truth": The factual answer to the question, which may include multiple possible answers
- "memory_answer": The answer provided by the LLM to the question
- "parametric_memory": The supportive evidence from LLM's parametric memory for the answer
- "counter_answer": The answer contradicting the "memory_answer"
- "counter_memory": The generation-based evidence supporting the counter_answer
- "parametric_memory_aligned_evidence": Additional evidence supporting the "memory_answer", which could be generated or derived from Wikipedia/human annotation
- "counter_memory_aligned_evidence": Additional evidence supporting the "counter_answer", either generated or sourced from Wikipedia/human annotation
passage: TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #arxiv-2305.13300 #region-us 
# Dataset Card for ConflcitQA## Dataset Description
- Repository: URL
- Paper: URL
- Point of Contact: Point of Contact: Jian Xie
If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries.# ConflcitQA

We provide conflictQA-[based large language model], which utilizes large language models guided parametric memory.# Data Fields
- "question": The question in natural language
- "popularity": The monthly page views on Wikipedia for the given question
- "ground_truth": The factual answer to the question, which may include multiple possible answers
- "memory_answer": The answer provided by the LLM to the question
- "parametric_memory": The supportive evidence from LLM's parametric memory for the answer
- "counter_answer": The answer contradicting the "memory_answer"
- "counter_memory": The generation-based evidence supporting the counter_answer
- "parametric_memory_aligned_evidence": Additional evidence supporting the "memory_answer", which could be generated or derived from Wikipedia/human annotation
- "counter_memory_aligned_evidence": Additional evidence supporting the "counter_answer", either generated or sourced from Wikipedia/human annotation