
# Dataset Card for CORE Deduplication

## Dataset Description

- **Homepage:** [https://core.ac.uk/about/research-outputs](https://core.ac.uk/about/research-outputs)
- **Repository:** [https://core.ac.uk/datasets/core_2020-05-10_deduplication.zip](https://core.ac.uk/datasets/core_2020-05-10_deduplication.zip)
- **Paper:** [Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings](http://oro.open.ac.uk/id/eprint/70519)
- **Point of Contact:** [CORE Team](https://core.ac.uk/about#contact)
- **Size of downloaded dataset files:** 204 MB

### Dataset Summary

CORE 2020 Deduplication dataset (https://core.ac.uk/documentation/dataset) contains 100K scholarly documents labeled as duplicates/non-duplicates.

### Languages

The dataset language is English (BCP-47 `en`)

### Citation Information

```
@inproceedings{dedup2020,
  title={Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings},
  author={Gyawali, Bikash and Anastasiou, Lucas and Knoth, Petr},
  booktitle = {Proceedings of 12th Language Resources and Evaluation Conference},
  month = may,
  year = 2020,
  publisher = {France European Language Resources Association},
  pages = {894-903}
}
```

task_categories:other task_ids:natural-language-inference task_ids:semantic-similarity-scoring task_ids:text-scoring annotations_creators:unknown language_creators:unknown multilinguality:monolingual size_categories:100K<n<1M source_datasets:unknown language:en license:mit deduplication region:us
{"annotations_creators": ["unknown"], "language_creators": ["unknown"], "language": ["en"], "license": ["mit"], "multilinguality": ["monolingual"], "size_categories": ["100K<n<1M"], "source_datasets": ["unknown"], "task_categories": ["other"], "task_ids": ["natural-language-inference", "semantic-similarity-scoring", "text-scoring"], "pretty_name": "CORE Deduplication of Scholarly Documents", "tags": ["deduplication"]}
TAGS
#task_categories-other #task_ids-natural-language-inference #task_ids-semantic-similarity-scoring #task_ids-text-scoring #annotations_creators-unknown #language_creators-unknown #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-unknown #language-English #license-mit #deduplication #region-us 


# Dataset Card for CORE Deduplication

## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings
- Point of Contact: CORE Team
- Size of downloaded dataset files: 204 MB

### Dataset Summary

CORE 2020 Deduplication dataset (URL contains 100K scholarly documents labeled as duplicates/non-duplicates.

### Languages

The dataset language is English (BCP-47 'en')





# Dataset Card for CORE Deduplication ## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings
- Point of Contact: CORE Team
- Size of downloaded dataset files: 204 MB ### Dataset Summary

CORE 2020 Deduplication dataset (URL contains 100K scholarly documents labeled as duplicates/non-duplicates. ### Languages

The dataset language is English (BCP-47 'en')
TAGS
#task_categories-other #task_ids-natural-language-inference #task_ids-semantic-similarity-scoring #task_ids-text-scoring #annotations_creators-unknown #language_creators-unknown #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-unknown #language-English #license-mit #deduplication #region-us 
 # Dataset Card for CORE Deduplication ## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings
- Point of Contact: CORE Team
- Size of downloaded dataset files: 204 MB ### Dataset Summary

CORE 2020 Deduplication dataset (URL contains 100K scholarly documents labeled as duplicates/non-duplicates. ### Languages

The dataset language is English (BCP-47 'en')
passage: TAGS
#task_categories-other #task_ids-natural-language-inference #task_ids-semantic-similarity-scoring #task_ids-text-scoring #annotations_creators-unknown #language_creators-unknown #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-unknown #language-English #license-mit #deduplication #region-us 
# Dataset Card for CORE Deduplication## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings
- Point of Contact: CORE Team
- Size of downloaded dataset files: 204 MB### Dataset Summary

CORE 2020 Deduplication dataset (URL contains 100K scholarly documents labeled as duplicates/non-duplicates.### Languages

The dataset language is English (BCP-47 'en')