
Preprocessed training and evaluation data from KiloGram.

KiloGram dataset and code repo: https://github.com/lil-lab/kilogram

---
# File Formats
## Training Set
Texts: `train_*.json` are all in the format of `{tangramName: list(annotations)}`.

Images: Colored images with parts (under `/color`) are named in the format of `tangramName_{idx}.png`, where `idx` corresponds to the index of the annotation in the text file.

## Validation, Development, Heldout Set

Texts: `{whole, part}_{black, color}.json` are in the format of `{"targets": list(imageFileNames), "images": list(imageFileNames), "texts": list(annotations)}`. We flattened all the contexts and concatenated them into one list for each entry. 

E.g. the first 10 elements in `"targets"` are the image file name of the target of the first context repeated 10 times; the first 10 of `"images"` are the image file names in that context; and the first 10 of `"texts"` are the corresponding 10 annotations in that context.

`/controlled` contains experiments with constrained contexts controlled for number of parts, and `/random` contains ones without. (See Appendix A.8 in paper)

`/development/texts/augmented/aug_dev.json` and `images/augmented.tar.bz2` are experiments in the same format as above used to evaluate the effect of adding part information. 


Intermediate files:

`*/text/controlled/eval_batch_data.json` are in the format of 
`{tangramName: {numOfParts: list({"target": [tangramName_{idx}, annotation], "distractors": list(list([tangramName_{idx}, annotation]))})}}`, used to generate controlled experiment jsons. Note: annotations are descriptions concatenated by "#" instead of in natural English.
annotations_creators:crowdsourced multilinguality:monolingual size_categories:1K<n<10K source_datasets:original language:en tangrams reference-games vision-language region:us
{"annotations_creators": ["crowdsourced"], "language": ["en"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["original"], "pretty_name": "KiloGram", "tags": ["tangrams", "reference-games", "vision-language"], "viewer": false}
TAGS
#annotations_creators-crowdsourced #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #tangrams #reference-games #vision-language #region-us 


Preprocessed training and evaluation data from KiloGram.

KiloGram dataset and code repo: URL

---
# File Formats
## Training Set
Texts: 'train_*.json' are all in the format of '{tangramName: list(annotations)}'.

Images: Colored images with parts (under '/color') are named in the format of 'tangramName_{idx}.png', where 'idx' corresponds to the index of the annotation in the text file.

## Validation, Development, Heldout Set

Texts: '{whole, part}_{black, color}.json' are in the format of '{"targets": list(imageFileNames), "images": list(imageFileNames), "texts": list(annotations)}'. We flattened all the contexts and concatenated them into one list for each entry. 

E.g. the first 10 elements in '"targets"' are the image file name of the target of the first context repeated 10 times; the first 10 of '"images"' are the image file names in that context; and the first 10 of '"texts"' are the corresponding 10 annotations in that context.

'/controlled' contains experiments with constrained contexts controlled for number of parts, and '/random' contains ones without. (See Appendix A.8 in paper)

'/development/texts/augmented/aug_dev.json' and 'images/URL.bz2' are experiments in the same format as above used to evaluate the effect of adding part information. 


Intermediate files:

'*/text/controlled/eval_batch_data.json' are in the format of 
'{tangramName: {numOfParts: list({"target": [tangramName_{idx}, annotation], "distractors": list(list([tangramName_{idx}, annotation]))})}}', used to generate controlled experiment jsons. Note: annotations are descriptions concatenated by "#" instead of in natural English.
# File Formats ## Training Set
Texts: 'train_*.json' are all in the format of '{tangramName: list(annotations)}'.

Images: Colored images with parts (under '/color') are named in the format of 'tangramName_{idx}.png', where 'idx' corresponds to the index of the annotation in the text file. ## Validation, Development, Heldout Set

Texts: '{whole, part}_{black, color}.json' are in the format of '{"targets": list(imageFileNames), "images": list(imageFileNames), "texts": list(annotations)}'. We flattened all the contexts and concatenated them into one list for each entry. 

E.g. the first 10 elements in '"targets"' are the image file name of the target of the first context repeated 10 times; the first 10 of '"images"' are the image file names in that context; and the first 10 of '"texts"' are the corresponding 10 annotations in that context.

'/controlled' contains experiments with constrained contexts controlled for number of parts, and '/random' contains ones without. (See Appendix A.8 in paper)

'/development/texts/augmented/aug_dev.json' and 'images/URL.bz2' are experiments in the same format as above used to evaluate the effect of adding part information. 


Intermediate files:

'*/text/controlled/eval_batch_data.json' are in the format of 
'{tangramName: {numOfParts: list({"target": [tangramName_{idx}, annotation], "distractors": list(list([tangramName_{idx}, annotation]))})}}', used to generate controlled experiment jsons. Note: annotations are descriptions concatenated by "#" instead of in natural English.
TAGS
#annotations_creators-crowdsourced #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #tangrams #reference-games #vision-language #region-us 
 # File Formats ## Training Set
Texts: 'train_*.json' are all in the format of '{tangramName: list(annotations)}'.

Images: Colored images with parts (under '/color') are named in the format of 'tangramName_{idx}.png', where 'idx' corresponds to the index of the annotation in the text file. ## Validation, Development, Heldout Set

Texts: '{whole, part}_{black, color}.json' are in the format of '{"targets": list(imageFileNames), "images": list(imageFileNames), "texts": list(annotations)}'. We flattened all the contexts and concatenated them into one list for each entry. 

E.g. the first 10 elements in '"targets"' are the image file name of the target of the first context repeated 10 times; the first 10 of '"images"' are the image file names in that context; and the first 10 of '"texts"' are the corresponding 10 annotations in that context.

'/controlled' contains experiments with constrained contexts controlled for number of parts, and '/random' contains ones without. (See Appendix A.8 in paper)

'/development/texts/augmented/aug_dev.json' and 'images/URL.bz2' are experiments in the same format as above used to evaluate the effect of adding part information. 


Intermediate files:

'*/text/controlled/eval_batch_data.json' are in the format of 
'{tangramName: {numOfParts: list({"target": [tangramName_{idx}, annotation], "distractors": list(list([tangramName_{idx}, annotation]))})}}', used to generate controlled experiment jsons. Note: annotations are descriptions concatenated by "#" instead of in natural English.
passage: TAGS
#annotations_creators-crowdsourced #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #tangrams #reference-games #vision-language #region-us 
# File Formats## Training Set
Texts: 'train_*.json' are all in the format of '{tangramName: list(annotations)}'.

Images: Colored images with parts (under '/color') are named in the format of 'tangramName_{idx}.png', where 'idx' corresponds to the index of the annotation in the text file.