adirik / owlvit-base-patch32

Zero-shot / open vocabulary object detection

  • Public
  • 23K runs
  • GitHub
  • Paper
  • License

Run time and cost

This model costs approximately $0.024 to run on Replicate, or 41 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 108 seconds. The predict time for this model varies significantly based on the inputs.

Readme

OWL-ViT

OWL-ViT uses CLIP and vision transformers backbones to enable open-vocabulary object detection. See the paper, original repository and Hugging Face implementation for details.

Using the API

You can use OWL-ViT to query images with text descriptions of any object. To use it, simply upload an image and enter comma separated text descriptions of objects you want to query the image for. You can also use the score threshold slider to set a threshold to filter out low probability predictions.

OWL-ViT is trained on text templates, hence you can get better predictions by querying the image with text templates used in training the original model: “photo of a star-spangled banner”, “image of a shoe”. Refer to the CLIP paper to see the full list of text templates used to augment the training data.

References

@article{minderer2022simple,
  title={Simple Open-Vocabulary Object Detection with Vision Transformers},
  author={Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai, Thomas Kipf, Neil Houlsby},
  journal={ECCV},
  year={2022},
}