cjwbw / cutie

Video Object Segmentation, combined with SAM and ProPainter

  • Public
  • 260 runs
  • GitHub
  • Paper
  • License
Run with an API
Playground API Examples README Versions

Examples

View more

Run time and cost

This model costs approximately $0.091 to run on Replicate, or 10 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 93 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Putting the Object Back into Video Object Segmentation

Highlight

Cutie is a video object segmentation framework – a follow-up work of XMem with better consistency, robustness, and speed. This repository contains code for standard video object segmentation and a GUI tool for interactive video segmentation. The GUI tool additionally contains the “permanent memory” (from XMem++) option for better controllability.

overview

Citation

@inproceedings{cheng2023putting,
  title={Putting the Object Back into Video Object Segmentation},
  author={Cheng, Ho Kei and Oh, Seoung Wug and Price, Brian and Lee, Joon-Young and Schwing, Alexander},
  booktitle={arXiv},
  year={2023}
}

References

  • The GUI tools uses RITM for interactive image segmentation. This repository also contains a redistribution of their code in gui/ritm. That part of code follows RITM’s license.

  • For automatic video segmentation/integration with external detectors, see DEVA.

  • The interactive demo is developed upon IVS, MiVOS, and XMem.

  • We used ProPainter in our video inpainting demo.

  • Thanks to RTIM and XMem++ for making this possible.