# Dataset Card for "CsFEVERv2"

## Dataset Description

CsFEVERv2 is a dataset for Czech fact-checking developed as part of a bachelor thesis at the Artificial Intelligence Center of the Faculty of Electrical Engineering of 
the Czech technical university in Prague. The dataset consists of an **original** subset, which is only an iteration of CsFEVER with new data and better processing and 
**f1**, **precision**, and **07** subsets filtered using an NLI model and optimized threshold values. The subset **wiki_pages** is a processed Wikipedia dump from 
August 2022 with correct revids. This subset should be used to map evidence from datasets to Wikipedia texts. Additionaly preprocessed datasets **original_nli**, **f1_nli**, **precision_nli**, **07_nli**,
for training of NLI models are included. 

The original subset can be used to generate other filtered datasets by filtering with other thresholds using predicted_label and predicted_score fields.

### Languages

Czech

## Dataset Usage Example

```python
from datasets import load_dataset

#load default (original) subset
dataset = load_dataset("/home/mlynatom/csfever_v2")
dataset = load_dataset("/home/mlynatom/csfever_v2", "original")

#load f1, f1_nli, precision, precision_nli, 07, and 07_nli subsets
dataset = load_dataset("/home/mlynatom/csfever_v2", "f1")

#load wiki_pages subset
dataset = load_dataset("/home/mlynatom/csfever_v2", "wiki_pages")

```

## Dataset Structure

### Data Instances

#### original
An example of 'train' looks as follows.
```json
{'id': 75397,
 'label': 'SUPPORTS',
 'predicted_label': 'SUPPORTS',
 'predicted_score': 0.921731
 'claim': 'Nikolaj Coster-Waldau pracoval pro Fox Broadcasting Company.',
 'evidence': [ [ "Nikolaj Coster-Waldau", "Nikolaj Coster-Waldau" ], [ "Fox Broadcasting Company", "Fox Broadcasting Company" ] ]}
```

#### f1, precision, 07
An example of 'train' looks as follows.
```json
{'id': 75397,
 'label': 'SUPPORTS',
 'claim': 'Nikolaj Coster-Waldau pracoval pro Fox Broadcasting Company.',
 'evidence': [ [ "Nikolaj Coster-Waldau", "Nikolaj Coster-Waldau" ], [ "Fox Broadcasting Company", "Fox Broadcasting Company" ] ]}
```

#### original_nli, f1_nli, precision_nli, 07_nli
An example of 'train' looks as follows.
```json
{'id': 155439,
 'label': 2,
 'claim': 'Newcastle United FC vyhrál pět ligových titulů.',
 'evidence': "Ronnie Simpson. Ronnie Simpson (21. října 1930, Glasgow – 19. dubna 2004, Edinburgh) byl skotský fotbalový brankář..."}
```

#### wiki_pages
An example of 'wiki_pages' looks as follows.
```json
{'id': 80916,
 'revid': 20561555,
 'url': "https://cs.wikipedia.org/wiki?curid=80916",
 'title': "Altruismus",
 'text': "Altruismus (z lat. "alter", druhý, 3. pád "altrui", druhému) je moderní ..."}
```

### Data Fields

#### original

- `id`: a `int32` feature.
- `label`: a `string` feature.
- `predicted_label`: a `string` feature. (label predicted by NLI model)
- `predicted_score`: a `int32` feature. (confidence of predicted_label predicted by NLI model)
- `claim`: a `string` feature.
- `evidence`: a `sequence` feature.

#### f1, precision, 07

- `id`: a `int32` feature.
- `label`: a `string` feature.
- `claim`: a `string` feature.
- `evidence`: a `sequence` feature.

#### original_nli, f1_nli, precision_nli, 07_nli

- `id`: a `int32` feature.
- `label`: a `int32` feature.
- `claim`: a `string` feature.
- `evidence`: a `string` feature.

#### wiki_pages
- `id`: a `int32` feature.
- `revid`: a `int32` feature.
- `url`: a `string` feature.
- `title`: a `string` feature.
- `text`: a `string` feature.


### Data Splits

### Data Splits
#### original
|          |  train | dev  | test  |
|----------|-------:|-----:|------:|
| original | 118950 | 7458 | 7520  |

#### f1

|    | train | dev  | test |
|----|------:|-----:|-----:|
| f1 | 83438 | 5445 | 5328 |

#### precision

|           |  train | dev  | test  |
|-----------|-------:|-----:|------:|
| precision |  60828 | 4288 |  4236 |

#### 07

|    |  train | dev  | test  |
|----|-------:|-----:|------:|
| 07 | 108607 | 6685 | 6623  |

#### wiki_pages
|            | wiki_pages |
|------------|-----------:|
| wiki_pages |    825078  |

# Citation

```bibtex
@article{Ullrich_2023,
	doi = {10.1007/s10579-023-09654-3},
	url = {https://doi.org/10.1007%2Fs10579-023-09654-3},
	year = 2023,
	month = {may},
	publisher = {Springer Science and Business Media {LLC}},
	author = {Herbert Ullrich and Jan Drchal and Martin Rýpar and Hana Vincourová and Václav Moravec},
	title = {{CsFEVER} and {CTKFacts}: acquiring Czech data for fact verification},
	journal = {Language Resources and Evaluation},
    archivePrefix={arXiv},
    eprint={2201.11115},
}
```

```bibtex
@thesis{Mlynar_2023,
  author      = {Mlynář, Tomáš},
  type        = {Bachelor's Thesis}
  title       = {Automated Fact Checking Based on Czech Wikipedia},
  institution = {Czech Technical University in Prague, Faculty of Electrical Engineering},
  date        = {2023},
  url         = {http://hdl.handle.net/10467/109219}
}
```

task_categories:text-classification task_categories:text-retrieval task_ids:natural-language-inference task_ids:document-retrieval multilinguality:monolingual size_categories:100K<n<1M source_datasets:fever language:cs license:cc-by-sa-3.0 Fact-checking arxiv:2201.11115 region:us
{"language": ["cs"], "license": "cc-by-sa-3.0", "multilinguality": "monolingual", "size_categories": ["100K<n<1M"], "source_datasets": "fever", "task_categories": ["text-classification", "text-retrieval"], "task_ids": ["natural-language-inference", "document-retrieval"], "pretty_name": "CsFEVERv2", "tags": ["Fact-checking"]}
TAGS
#task_categories-text-classification #task_categories-text-retrieval #task_ids-natural-language-inference #task_ids-document-retrieval #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-fever #language-Czech #license-cc-by-sa-3.0 #Fact-checking #arxiv-2201.11115 #region-us 

Dataset Card for "CsFEVERv2"
============================


Dataset Description
-------------------


CsFEVERv2 is a dataset for Czech fact-checking developed as part of a bachelor thesis at the Artificial Intelligence Center of the Faculty of Electrical Engineering of
the Czech technical university in Prague. The dataset consists of an original subset, which is only an iteration of CsFEVER with new data and better processing and
f1, precision, and 07 subsets filtered using an NLI model and optimized threshold values. The subset wiki\_pages is a processed Wikipedia dump from
August 2022 with correct revids. This subset should be used to map evidence from datasets to Wikipedia texts. Additionaly preprocessed datasets original\_nli, f1\_nli, precision\_nli, 07\_nli,
for training of NLI models are included.


The original subset can be used to generate other filtered datasets by filtering with other thresholds using predicted\_label and predicted\_score fields.


### Languages


Czech


Dataset Usage Example
---------------------


Dataset Structure
-----------------


### Data Instances


#### original


An example of 'train' looks as follows.


#### f1, precision, 07


An example of 'train' looks as follows.


#### original\_nli, f1\_nli, precision\_nli, 07\_nli


An example of 'train' looks as follows.


#### wiki\_pages


An example of 'wiki\_pages' looks as follows.


### Data Fields


#### original


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'predicted\_label': a 'string' feature. (label predicted by NLI model)
* 'predicted\_score': a 'int32' feature. (confidence of predicted\_label predicted by NLI model)
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature.


#### f1, precision, 07


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature.


#### original\_nli, f1\_nli, precision\_nli, 07\_nli


* 'id': a 'int32' feature.
* 'label': a 'int32' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'string' feature.


#### wiki\_pages


* 'id': a 'int32' feature.
* 'revid': a 'int32' feature.
* 'url': a 'string' feature.
* 'title': a 'string' feature.
* 'text': a 'string' feature.


### Data Splits


### Data Splits


#### original



#### f1



#### precision



#### 07



#### wiki\_pages




### Languages


Czech


Dataset Usage Example
---------------------


Dataset Structure
----------------- ### Data Instances #### original


An example of 'train' looks as follows. #### f1, precision, 07


An example of 'train' looks as follows. #### original\_nli, f1\_nli, precision\_nli, 07\_nli


An example of 'train' looks as follows. #### wiki\_pages


An example of 'wiki\_pages' looks as follows. ### Data Fields #### original


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'predicted\_label': a 'string' feature. (label predicted by NLI model)
* 'predicted\_score': a 'int32' feature. (confidence of predicted\_label predicted by NLI model)
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature. #### f1, precision, 07


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature. #### original\_nli, f1\_nli, precision\_nli, 07\_nli


* 'id': a 'int32' feature.
* 'label': a 'int32' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'string' feature. #### wiki\_pages


* 'id': a 'int32' feature.
* 'revid': a 'int32' feature.
* 'url': a 'string' feature.
* 'title': a 'string' feature.
* 'text': a 'string' feature. ### Data Splits ### Data Splits #### original #### f1 #### precision #### 07 #### wiki\_pages
TAGS
#task_categories-text-classification #task_categories-text-retrieval #task_ids-natural-language-inference #task_ids-document-retrieval #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-fever #language-Czech #license-cc-by-sa-3.0 #Fact-checking #arxiv-2201.11115 #region-us 
 ### Languages


Czech


Dataset Usage Example
---------------------


Dataset Structure
----------------- ### Data Instances #### original


An example of 'train' looks as follows. #### f1, precision, 07


An example of 'train' looks as follows. #### original\_nli, f1\_nli, precision\_nli, 07\_nli


An example of 'train' looks as follows. #### wiki\_pages


An example of 'wiki\_pages' looks as follows. ### Data Fields #### original


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'predicted\_label': a 'string' feature. (label predicted by NLI model)
* 'predicted\_score': a 'int32' feature. (confidence of predicted\_label predicted by NLI model)
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature. #### f1, precision, 07


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature. #### original\_nli, f1\_nli, precision\_nli, 07\_nli


* 'id': a 'int32' feature.
* 'label': a 'int32' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'string' feature. #### wiki\_pages


* 'id': a 'int32' feature.
* 'revid': a 'int32' feature.
* 'url': a 'string' feature.
* 'title': a 'string' feature.
* 'text': a 'string' feature. ### Data Splits ### Data Splits #### original #### f1 #### precision #### 07 #### wiki\_pages
passage: TAGS
#task_categories-text-classification #task_categories-text-retrieval #task_ids-natural-language-inference #task_ids-document-retrieval #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-fever #language-Czech #license-cc-by-sa-3.0 #Fact-checking #arxiv-2201.11115 #region-us 
### Languages


Czech


Dataset Usage Example
---------------------


Dataset Structure
-----------------### Data Instances#### original


An example of 'train' looks as follows.#### f1, precision, 07


An example of 'train' looks as follows.#### original\_nli, f1\_nli, precision\_nli, 07\_nli


An example of 'train' looks as follows.#### wiki\_pages


An example of 'wiki\_pages' looks as follows.### Data Fields#### original


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'predicted\_label': a 'string' feature. (label predicted by NLI model)
* 'predicted\_score': a 'int32' feature. (confidence of predicted\_label predicted by NLI model)
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature.#### f1, precision, 07


* 'id': a 'int32' feature.
* 'label': a 'string' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'sequence' feature.#### original\_nli, f1\_nli, precision\_nli, 07\_nli


* 'id': a 'int32' feature.
* 'label': a 'int32' feature.
* 'claim': a 'string' feature.
* 'evidence': a 'string' feature.