# Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to [this repo ](https://github.com/ServiceNow/broad) to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above.

## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the [ImageNet](https://www.image-net.org/index.php) distribution.

### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition.

### Included Splits

- **Clean** is comprised of 36157 images from the original validation set of ILSVRC2012. They are used as in-distribution in BROAD.
- **Adversarial Autoattack Resnet**, **Adversarial Autoattack ViT**, **Adversarial PGD Resnet** and **Adversarial PGD ViT** are splits each comprised of 5,000 adversarial perturbations of clean validation images, using a perturbation budget of 0.05 with the L-infinity norm. These attacks are computed against a trained ResNet-50 and a trained ViT-b/16. PGD uses 40 iterations and for Autoattack, only the attack model achieving the most confident misclassification is kept.
- **Synthetic Gan** and **Synthetic Diffusion** are each comprised of 25,000 synthetic images generated to imitate the ImageNet distribution. For Synthetic Gan, a conditional BigGan architecture was used to generate 25 artificial samples from each ImageNet class. For Synthetic diffusion, we leveraged stable diffusion models to generate 25 artificial samples per class using the prompt "High quality image of a {class_name}".
- **CoComageNet** is a novel split from the [CoCo](https://cocodataset.org/#home) dataset comprised of 2000 images, each featuring multiple distinct classes of ImageNet. Each image of CoComageNet thus features multiple objects, at least two of which have distinct ImageNet labels. More details on the construction of CoComageNet can be found in the paper.
- **CoComageNet-mono** is built similarly to CoComageNet, except each image only has one object with ImageNet label. It is designed as an ablation, to isolate the effect of having instances of multiple labels from other distributional shifts in CoComageNet.

### External Splits

- **iNaturalist** is a split of the original [iNaturalist2017 dataset](https://github.com/visipedia/inat_comp/tree/master/2017) designed for OOD detection with ImageNet as in-distribution. It was introduced in [MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space](https://arxiv.org/pdf/2105.01879.pdf) and can be downloaded [here](http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz).
- **ImageNet-O** was introduced in [Natural Adversarial Examples](https://arxiv.org/pdf/1907.07174.pdf) and is comprised of natural examples that were selected for their high classification confidence by CNNs. It can be downloaded [here](https://people.eecs.berkeley.edu/~hendrycks/imagenet-o.tar).
- **OpenImage-O** is a subset of the OpenImage dataset that was built similarly to ImageNet-O in [ViM: Out-Of-Distribution with Virtual-logit Matching](https://arxiv.org/pdf/2203.10807.pdf). The file list can be accessed [here](https://github.com/haoqiwang/vim/blob/master/datalists/openimage_o.txt).
- **Defocus blur**, **Gaussian noise**, **Snow** and **Brightness** are all existing splits of the [ImageNet-C dataset](https://github.com/hendrycks/robustness). For BROAD, only the highest strength of corruption (5/5) is used.

### LICENSE

This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/deed.en_US">Creative Commons Attribution 4.0 Unported License</a>.
size_categories:10K<n<100K license:cc-by-4.0 imagenet OOD detection distribution shift arxiv:2105.01879 arxiv:1907.07174 arxiv:2203.10807 region:us
{"license": "cc-by-4.0", "size_categories": ["10K<n<100K"], "pretty_name": "BROAD", "dataset_info": {"features": [{"name": "date_captured", "dtype": "string"}, {"name": "coco_url", "dtype": "string"}, {"name": "license_name", "dtype": "string"}, {"name": "license_url", "dtype": "string"}, {"name": "coco_id", "dtype": "string"}, {"name": "image", "dtype": "image"}, {"name": "label", "dtype": "int64"}, {"name": "flickr_url", "dtype": "string"}], "splits": [{"name": "clean", "num_bytes": 333801279.747, "num_examples": 36157}, {"name": "cocomagenet", "num_bytes": 306403223, "num_examples": 2000}, {"name": "cocomagenet_mono", "num_bytes": 18956338, "num_examples": 2000}, {"name": "synthetic_gan", "num_bytes": 242598071.454, "num_examples": 24999}, {"name": "synthetic_diffusion", "num_bytes": 283705025, "num_examples": 25000}, {"name": "adversarial_autoattack_resnet", "num_bytes": 40058245, "num_examples": 5000}, {"name": "adversarial_autoattack_vit", "num_bytes": 35610460, "num_examples": 5000}, {"name": "adversarial_pgd_resnet", "num_bytes": 65806170, "num_examples": 5000}, {"name": "adversarial_pgd_vit", "num_bytes": 51803590, "num_examples": 5000}], "download_size": 1432934722, "dataset_size": 1378742402.201}, "tags": ["imagenet", "OOD detection", "distribution shift"]}
TAGS
#size_categories-10K<n<100K #license-cc-by-4.0 #imagenet #OOD detection #distribution shift #arxiv-2105.01879 #arxiv-1907.07174 #arxiv-2203.10807 #region-us 

# Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to this repo  to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above.

## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the ImageNet distribution.

### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition.

### Included Splits

- Clean is comprised of 36157 images from the original validation set of ILSVRC2012. They are used as in-distribution in BROAD.
- Adversarial Autoattack Resnet, Adversarial Autoattack ViT, Adversarial PGD Resnet and Adversarial PGD ViT are splits each comprised of 5,000 adversarial perturbations of clean validation images, using a perturbation budget of 0.05 with the L-infinity norm. These attacks are computed against a trained ResNet-50 and a trained ViT-b/16. PGD uses 40 iterations and for Autoattack, only the attack model achieving the most confident misclassification is kept.
- Synthetic Gan and Synthetic Diffusion are each comprised of 25,000 synthetic images generated to imitate the ImageNet distribution. For Synthetic Gan, a conditional BigGan architecture was used to generate 25 artificial samples from each ImageNet class. For Synthetic diffusion, we leveraged stable diffusion models to generate 25 artificial samples per class using the prompt "High quality image of a {class_name}".
- CoComageNet is a novel split from the CoCo dataset comprised of 2000 images, each featuring multiple distinct classes of ImageNet. Each image of CoComageNet thus features multiple objects, at least two of which have distinct ImageNet labels. More details on the construction of CoComageNet can be found in the paper.
- CoComageNet-mono is built similarly to CoComageNet, except each image only has one object with ImageNet label. It is designed as an ablation, to isolate the effect of having instances of multiple labels from other distributional shifts in CoComageNet.

### External Splits

- iNaturalist is a split of the original iNaturalist2017 dataset designed for OOD detection with ImageNet as in-distribution. It was introduced in MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space and can be downloaded here.
- ImageNet-O was introduced in Natural Adversarial Examples and is comprised of natural examples that were selected for their high classification confidence by CNNs. It can be downloaded here.
- OpenImage-O is a subset of the OpenImage dataset that was built similarly to ImageNet-O in ViM: Out-Of-Distribution with Virtual-logit Matching. The file list can be accessed here.
- Defocus blur, Gaussian noise, Snow and Brightness are all existing splits of the ImageNet-C dataset. For BROAD, only the highest strength of corruption (5/5) is used.

### LICENSE

This work is licensed under a <a rel="license" href="URL Commons Attribution 4.0 Unported License</a>.
# Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to this repo  to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above. ## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the ImageNet distribution. ### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition. ### Included Splits

- Clean is comprised of 36157 images from the original validation set of ILSVRC2012. They are used as in-distribution in BROAD.
- Adversarial Autoattack Resnet, Adversarial Autoattack ViT, Adversarial PGD Resnet and Adversarial PGD ViT are splits each comprised of 5,000 adversarial perturbations of clean validation images, using a perturbation budget of 0.05 with the L-infinity norm. These attacks are computed against a trained ResNet-50 and a trained ViT-b/16. PGD uses 40 iterations and for Autoattack, only the attack model achieving the most confident misclassification is kept.
- Synthetic Gan and Synthetic Diffusion are each comprised of 25,000 synthetic images generated to imitate the ImageNet distribution. For Synthetic Gan, a conditional BigGan architecture was used to generate 25 artificial samples from each ImageNet class. For Synthetic diffusion, we leveraged stable diffusion models to generate 25 artificial samples per class using the prompt "High quality image of a {class_name}".
- CoComageNet is a novel split from the CoCo dataset comprised of 2000 images, each featuring multiple distinct classes of ImageNet. Each image of CoComageNet thus features multiple objects, at least two of which have distinct ImageNet labels. More details on the construction of CoComageNet can be found in the paper.
- CoComageNet-mono is built similarly to CoComageNet, except each image only has one object with ImageNet label. It is designed as an ablation, to isolate the effect of having instances of multiple labels from other distributional shifts in CoComageNet. ### External Splits

- iNaturalist is a split of the original iNaturalist2017 dataset designed for OOD detection with ImageNet as in-distribution. It was introduced in MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space and can be downloaded here.
- ImageNet-O was introduced in Natural Adversarial Examples and is comprised of natural examples that were selected for their high classification confidence by CNNs. It can be downloaded here.
- OpenImage-O is a subset of the OpenImage dataset that was built similarly to ImageNet-O in ViM: Out-Of-Distribution with Virtual-logit Matching. The file list can be accessed here.
- Defocus blur, Gaussian noise, Snow and Brightness are all existing splits of the ImageNet-C dataset. For BROAD, only the highest strength of corruption (5/5) is used. ### LICENSE

This work is licensed under a <a rel="license" href="URL Commons Attribution 4.0 Unported License</a>.
TAGS
#size_categories-10K<n<100K #license-cc-by-4.0 #imagenet #OOD detection #distribution shift #arxiv-2105.01879 #arxiv-1907.07174 #arxiv-2203.10807 #region-us 
 # Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to this repo  to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above. ## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the ImageNet distribution. ### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition. ### Included Splits

- Clean is comprised of 36157 images from the original validation set of ILSVRC2012. They are used as in-distribution in BROAD.
- Adversarial Autoattack Resnet, Adversarial Autoattack ViT, Adversarial PGD Resnet and Adversarial PGD ViT are splits each comprised of 5,000 adversarial perturbations of clean validation images, using a perturbation budget of 0.05 with the L-infinity norm. These attacks are computed against a trained ResNet-50 and a trained ViT-b/16. PGD uses 40 iterations and for Autoattack, only the attack model achieving the most confident misclassification is kept.
- Synthetic Gan and Synthetic Diffusion are each comprised of 25,000 synthetic images generated to imitate the ImageNet distribution. For Synthetic Gan, a conditional BigGan architecture was used to generate 25 artificial samples from each ImageNet class. For Synthetic diffusion, we leveraged stable diffusion models to generate 25 artificial samples per class using the prompt "High quality image of a {class_name}".
- CoComageNet is a novel split from the CoCo dataset comprised of 2000 images, each featuring multiple distinct classes of ImageNet. Each image of CoComageNet thus features multiple objects, at least two of which have distinct ImageNet labels. More details on the construction of CoComageNet can be found in the paper.
- CoComageNet-mono is built similarly to CoComageNet, except each image only has one object with ImageNet label. It is designed as an ablation, to isolate the effect of having instances of multiple labels from other distributional shifts in CoComageNet. ### External Splits

- iNaturalist is a split of the original iNaturalist2017 dataset designed for OOD detection with ImageNet as in-distribution. It was introduced in MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space and can be downloaded here.
- ImageNet-O was introduced in Natural Adversarial Examples and is comprised of natural examples that were selected for their high classification confidence by CNNs. It can be downloaded here.
- OpenImage-O is a subset of the OpenImage dataset that was built similarly to ImageNet-O in ViM: Out-Of-Distribution with Virtual-logit Matching. The file list can be accessed here.
- Defocus blur, Gaussian noise, Snow and Brightness are all existing splits of the ImageNet-C dataset. For BROAD, only the highest strength of corruption (5/5) is used. ### LICENSE

This work is licensed under a <a rel="license" href="URL Commons Attribution 4.0 Unported License</a>.
passage: TAGS
#size_categories-10K<n<100K #license-cc-by-4.0 #imagenet #OOD detection #distribution shift #arxiv-2105.01879 #arxiv-1907.07174 #arxiv-2203.10807 #region-us 
# Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to this repo  to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above.## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the ImageNet distribution.### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition.