Image Retrieval with Text and Sketch
This code is for our 2022 ECCV paper [A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch]
folder structure
|---model/ : Contain the trained model*
|---sketches/ : Contain example query sketch
|---images/ : Contain 100 randomly sampled images from COCO TBIR benchmark
|---notebooks/ : Contain the demo ipynb notebook (can run via Colab)
|---code/
|---training/model_configs/ : Contain model config file for the network
|---clip/ : Contain source code for running the notebook
*model can be downloaded from https://patsorn.me/projects/tsbir/data/tsbir_model_final.pt
This repo is based on open_clip implementation from https://github.com/mlfoundations/open_clip
Prerequisites
- Pytorch
Getting Started
Simply run notebooks/Retrieval_Demo.ipynb, you can use your own set of images and sketches by modifying the images/ and sketches/ folder accordingly.
Download Models
Pre-trained models
Citation
If you find it this code useful for your research, please cite:
"A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch"
Patsorn Sangkloy, Wittawat Jitkrittum, Diyi Yang, James Hays in ECCV, 2022.
@article{
tsbir2022,
author = {Patsorn Sangkloy and Wittawat Jitkrittum and Diyi Yang and James Hays},
title = {A Sketch is Worth a Thousand Words: Image Retrieval with Text and Sketch},
journal = {European Conference on Computer Vision, ECCV},
year = {2022},
}