# Toloker Graph: Interaction of Crowd Annotators

## Dataset Description

- **Repository:** https://github.com/Toloka/TolokerGraph
- **Point of Contact:** research@toloka.ai

### Dataset Summary

This repository contains a graph representing interactions between crowd annotators on a project labeled on the [Toloka](https://toloka.ai/) crowdsourcing platform (see the [Toloka overview](https://toloka.ai/en/docs/guide/concepts/overview) for the details on the used terminology).

The graph contains 11,758 nodes and 519,000 edges. Each node represents an individual annotator; nodes are provided with four numerical and three categorical features. An edge is drawn between a pair of annotators if they annotated the same task. Also, each node is provided with a label showing whether the annotator was banned on this project, or not.

### Nodes

Nodes are stored in the [nodes.tsv](nodes.tsv) file in the TSV format of the following structure:

- `id`: unique identifier of the annotator
- `approved_rate`: percentage of the approved labels of this annotator
- `skipped_rate`: percentage of the skipped tasks of this annotator
- `expired_rate`: percentage of the expired tasks of this annotator
- `rejected_rate`: percentage of the rejected labels of this annotator
- `education`: level of education as self-reported by this annotator (`none`, `basic`, `middle`, `high`)
- `english_profile`: knowledge of English as self-reported by this annotator (`0` for no, `1` for yes)
- `english_tested`: whether the annotator passed the Toloka language test for English (`0` for no, `1` for yes)
- `banned`: whether the annotator was banned on this project (`0` for no, `1` for yes)

The `*_rate` attributes should sum up to 1.

### Edges

Edges are stored in the [edges.tsv](edges.tsv) file in the TSV format of the following structure:

- `source`: source identifier of the annotator
- `target`: target identifier of the annotator

As the graph is undirected, `source` and `target` can be interchanged for the given pair of nodes.

### Citation

* Likhobaba, D., Pavlichenko, N., Ustalov, D. (2023). [Toloker Graph: Interaction of Crowd Annotators](https://doi.org/10.5281/zenodo.7620795). Zenodo. <https://doi.org/10.5281/zenodo.7620795>

```bibtex
@dataset{Tolokers,
  author     = {Likhobaba, Daniil and Pavlichenko, Nikita and Ustalov, Dmitry},
  title      = {{Toloker Graph: Interaction of Crowd Annotators}},
  year       = {2023},
  publisher  = {Zenodo},
  doi        = {10.5281/zenodo.7620795},
  url        = {https://github.com/Toloka/TolokerGraph},
  language   = {english},
}
```

### Copyright

Licensed under the Creative Commons Attribution 4.0 License. See LICENSE file for more details.

task_categories:graph-ml size_categories:10K<n<100K license:cc-by-4.0 toloka graph node-classification region:us
{"license": "cc-by-4.0", "size_categories": ["10K<n<100K"], "task_categories": ["graph-ml"], "pretty_name": "Graph of Crowd Annotators", "tags": ["toloka", "graph", "node-classification"], "dataset_info": {"features": [{"name": "id", "dtype": "int32"}, {"name": "approved_rate", "dtype": "float"}, {"name": "skipped_rate", "dtype": "float"}, {"name": "expired_rate", "dtype": "float"}, {"name": "rejected_rate", "dtype": "float"}, {"name": "education", "dtype": "string"}, {"name": "english_profile", "dtype": "bool"}, {"name": "english_tested", "dtype": "bool"}, {"name": "banned", "dtype": "bool"}]}}
TAGS
#task_categories-graph-ml #size_categories-10K<n<100K #license-cc-by-4.0 #toloka #graph #node-classification #region-us 

# Toloker Graph: Interaction of Crowd Annotators

## Dataset Description

- Repository: URL
- Point of Contact: research@URL

### Dataset Summary

This repository contains a graph representing interactions between crowd annotators on a project labeled on the Toloka crowdsourcing platform (see the Toloka overview for the details on the used terminology).

The graph contains 11,758 nodes and 519,000 edges. Each node represents an individual annotator; nodes are provided with four numerical and three categorical features. An edge is drawn between a pair of annotators if they annotated the same task. Also, each node is provided with a label showing whether the annotator was banned on this project, or not.

### Nodes

Nodes are stored in the URL file in the TSV format of the following structure:

- 'id': unique identifier of the annotator
- 'approved_rate': percentage of the approved labels of this annotator
- 'skipped_rate': percentage of the skipped tasks of this annotator
- 'expired_rate': percentage of the expired tasks of this annotator
- 'rejected_rate': percentage of the rejected labels of this annotator
- 'education': level of education as self-reported by this annotator ('none', 'basic', 'middle', 'high')
- 'english_profile': knowledge of English as self-reported by this annotator ('0' for no, '1' for yes)
- 'english_tested': whether the annotator passed the Toloka language test for English ('0' for no, '1' for yes)
- 'banned': whether the annotator was banned on this project ('0' for no, '1' for yes)

The '*_rate' attributes should sum up to 1.

### Edges

Edges are stored in the URL file in the TSV format of the following structure:

- 'source': source identifier of the annotator
- 'target': target identifier of the annotator

As the graph is undirected, 'source' and 'target' can be interchanged for the given pair of nodes.

* Likhobaba, D., Pavlichenko, N., Ustalov, D. (2023). Toloker Graph: Interaction of Crowd Annotators. Zenodo. <URL



### Copyright

Licensed under the Creative Commons Attribution 4.0 License. See LICENSE file for more details.

# Toloker Graph: Interaction of Crowd Annotators ## Dataset Description

- Repository: URL
- Point of Contact: research@URL ### Dataset Summary

This repository contains a graph representing interactions between crowd annotators on a project labeled on the Toloka crowdsourcing platform (see the Toloka overview for the details on the used terminology).

The graph contains 11,758 nodes and 519,000 edges. Each node represents an individual annotator; nodes are provided with four numerical and three categorical features. An edge is drawn between a pair of annotators if they annotated the same task. Also, each node is provided with a label showing whether the annotator was banned on this project, or not. ### Nodes

Nodes are stored in the URL file in the TSV format of the following structure:

- 'id': unique identifier of the annotator
- 'approved_rate': percentage of the approved labels of this annotator
- 'skipped_rate': percentage of the skipped tasks of this annotator
- 'expired_rate': percentage of the expired tasks of this annotator
- 'rejected_rate': percentage of the rejected labels of this annotator
- 'education': level of education as self-reported by this annotator ('none', 'basic', 'middle', 'high')
- 'english_profile': knowledge of English as self-reported by this annotator ('0' for no, '1' for yes)
- 'english_tested': whether the annotator passed the Toloka language test for English ('0' for no, '1' for yes)
- 'banned': whether the annotator was banned on this project ('0' for no, '1' for yes)

The '*_rate' attributes should sum up to 1. ### Edges

Edges are stored in the URL file in the TSV format of the following structure:

- 'source': source identifier of the annotator
- 'target': target identifier of the annotator

As the graph is undirected, 'source' and 'target' can be interchanged for the given pair of nodes.

* Likhobaba, D., Pavlichenko, N., Ustalov, D. (2023). Toloker Graph: Interaction of Crowd Annotators. Zenodo. <URL ### Copyright

Licensed under the Creative Commons Attribution 4.0 License. See LICENSE file for more details.
TAGS
#task_categories-graph-ml #size_categories-10K<n<100K #license-cc-by-4.0 #toloka #graph #node-classification #region-us 
 # Toloker Graph: Interaction of Crowd Annotators ## Dataset Description

- Repository: URL
- Point of Contact: research@URL ### Dataset Summary

This repository contains a graph representing interactions between crowd annotators on a project labeled on the Toloka crowdsourcing platform (see the Toloka overview for the details on the used terminology).

The graph contains 11,758 nodes and 519,000 edges. Each node represents an individual annotator; nodes are provided with four numerical and three categorical features. An edge is drawn between a pair of annotators if they annotated the same task. Also, each node is provided with a label showing whether the annotator was banned on this project, or not. ### Nodes

Nodes are stored in the URL file in the TSV format of the following structure:

- 'id': unique identifier of the annotator
- 'approved_rate': percentage of the approved labels of this annotator
- 'skipped_rate': percentage of the skipped tasks of this annotator
- 'expired_rate': percentage of the expired tasks of this annotator
- 'rejected_rate': percentage of the rejected labels of this annotator
- 'education': level of education as self-reported by this annotator ('none', 'basic', 'middle', 'high')
- 'english_profile': knowledge of English as self-reported by this annotator ('0' for no, '1' for yes)
- 'english_tested': whether the annotator passed the Toloka language test for English ('0' for no, '1' for yes)
- 'banned': whether the annotator was banned on this project ('0' for no, '1' for yes)

The '*_rate' attributes should sum up to 1. ### Edges

Edges are stored in the URL file in the TSV format of the following structure:

- 'source': source identifier of the annotator
- 'target': target identifier of the annotator

As the graph is undirected, 'source' and 'target' can be interchanged for the given pair of nodes.

* Likhobaba, D., Pavlichenko, N., Ustalov, D. (2023). Toloker Graph: Interaction of Crowd Annotators. Zenodo. <URL ### Copyright

Licensed under the Creative Commons Attribution 4.0 License. See LICENSE file for more details.