Join us at Uncanny Spaces, a series of talks about ML and creativity. 🚀

kvfrans/clipdraw

Public
Synthesize drawings to match a text prompt
2,533 runs

Performance

This model runs predictions on Nvidia T4 GPU hardware.

80% of predictions complete within 5 minutes. The predict time for this model varies significantly based on the inputs.

Readme

CLIPDraw: Synthesize drawings to match a text prompt!

This work presents CLIPDraw, an algorithm that synthesizes novel drawings based on natural language input. CLIPDraw does not require any training; rather a pre-trained CLIP language-image encoder is used as a metric for maximizing similarity between the given description and a generated drawing. Crucially, CLIPDraw operates over vector strokes rather than pixel images, a constraint that biases drawings towards simpler human-recognizable shapes. Results compare between CLIPDraw and other synthesis-through-optimization methods, as well as highlight various interesting behaviors of CLIPDraw, such as satisfying ambiguous text in multiple ways, reliably producing drawings in diverse artistic styles, and scaling from simple to complex visual representations as stroke count is increased.

Replicate