
The dataset for training classification model of pixiv artworks by my preference.

## Schema

* tag_probs: List of probabilities for each tag. Preprocessed by [RF5/danbooru-pretrained](https://github.com/RF5/danbooru-pretrained) model. The index of each probability corresponds to the index of the tag in the [class_names_6000.json](https://github.com/RF5/danbooru-pretrained/blob/master/config/class_names_6000.json) file.
* class:
  * not_bookmarked (0): Generated from images randomly-sampled from [animelover/danbooru2022](https://huggingface.co/datasets/animelover/danbooru2022) dataset. The images are filtered in advance to the post with pixiv source.
  * bookmarked_public (1): Generated from publicly bookmarked images of [hakatashi](https://twitter.com/hakatashi).
  * bookmarked_private (2): Generated from privately bookmarked images of [hakatashi](https://twitter.com/hakatashi).

## Stats

train:test:validation = 6:2:2

* not_bookmarked (0): 202,290 images
* bookmarked_public (1): 73,587 images
* bookmarked_private (2): 22,659 images

## Usage

```
>>> from datasets import load_dataset

>>> dataset = load_dataset("hakatashi/hakatashi-pixiv-bookmark-deepdanbooru")
>>> dataset
DatasetDict({
    test: Dataset({
        features: ['tag_probs', 'class'],
        num_rows: 59707
    })
    train: Dataset({
        features: ['tag_probs', 'class'],
        num_rows: 179121
    })
    validation: Dataset({
        features: ['tag_probs', 'class'],
        num_rows: 59708
    })
})
>>> dataset['train'].features
{'tag_probs': Sequence(feature=Value(dtype='float32', id=None), length=-1, id=None),
 'class': ClassLabel(names=['not_bookmarked', 'bookmarked_public', 'bookmarked_private'], id=None)}
```
task_categories:image-classification task_categories:tabular-classification size_categories:100K<n<1M art region:us
{"size_categories": ["100K<n<1M"], "task_categories": ["image-classification", "tabular-classification"], "configs": [{"config_name": "default", "data_files": [{"split": "train", "path": "data/train-*"}, {"split": "test", "path": "data/test-*"}, {"split": "validation", "path": "data/validation-*"}]}], "dataset_info": {"features": [{"name": "tag_probs", "sequence": "float32"}, {"name": "class", "dtype": {"class_label": {"names": {"0": "not_bookmarked", "1": "bookmarked_public", "2": "bookmarked_private"}}}}], "splits": [{"name": "train", "num_bytes": 4301053452, "num_examples": 179121}, {"name": "test", "num_bytes": 1433684484, "num_examples": 59707}, {"name": "validation", "num_bytes": 1433708496, "num_examples": 59708}], "download_size": 7351682183, "dataset_size": 7168446432}, "tags": ["art"]}
TAGS
#task_categories-image-classification #task_categories-tabular-classification #size_categories-100K<n<1M #art #region-us 


The dataset for training classification model of pixiv artworks by my preference.

## Schema

* tag_probs: List of probabilities for each tag. Preprocessed by RF5/danbooru-pretrained model. The index of each probability corresponds to the index of the tag in the class_names_6000.json file.
* class:
  * not_bookmarked (0): Generated from images randomly-sampled from animelover/danbooru2022 dataset. The images are filtered in advance to the post with pixiv source.
  * bookmarked_public (1): Generated from publicly bookmarked images of hakatashi.
  * bookmarked_private (2): Generated from privately bookmarked images of hakatashi.

## Stats

train:test:validation = 6:2:2

* not_bookmarked (0): 202,290 images
* bookmarked_public (1): 73,587 images
* bookmarked_private (2): 22,659 images

## Usage


## Schema

* tag_probs: List of probabilities for each tag. Preprocessed by RF5/danbooru-pretrained model. The index of each probability corresponds to the index of the tag in the class_names_6000.json file.
* class:
  * not_bookmarked (0): Generated from images randomly-sampled from animelover/danbooru2022 dataset. The images are filtered in advance to the post with pixiv source.
  * bookmarked_public (1): Generated from publicly bookmarked images of hakatashi.
  * bookmarked_private (2): Generated from privately bookmarked images of hakatashi. ## Stats

train:test:validation = 6:2:2

* not_bookmarked (0): 202,290 images
* bookmarked_public (1): 73,587 images
* bookmarked_private (2): 22,659 images ## Usage
TAGS
#task_categories-image-classification #task_categories-tabular-classification #size_categories-100K<n<1M #art #region-us 
 ## Schema

* tag_probs: List of probabilities for each tag. Preprocessed by RF5/danbooru-pretrained model. The index of each probability corresponds to the index of the tag in the class_names_6000.json file.
* class:
  * not_bookmarked (0): Generated from images randomly-sampled from animelover/danbooru2022 dataset. The images are filtered in advance to the post with pixiv source.
  * bookmarked_public (1): Generated from publicly bookmarked images of hakatashi.
  * bookmarked_private (2): Generated from privately bookmarked images of hakatashi. ## Stats

train:test:validation = 6:2:2

* not_bookmarked (0): 202,290 images
* bookmarked_public (1): 73,587 images
* bookmarked_private (2): 22,659 images ## Usage