pollinations / lucid-sonic-dreams-xl

Lucid Sonic Dreams syncs StyleGAN XL -generated visuals to music

  • Public
  • 2.5K runs
  • GitHub
  • Paper
Run with an API
Playground API Examples README Versions

😵 Uh oh! This model can't be run on Replicate because it was built with a version of Cog or Python that is no longer supported. Consider opening an issue on the model's GitHub repository to see if it can be updated to use a recent version of Cog. If you need any help, please hop into our Discord channel or Contact us about it.

Examples

View more

Run time and cost

This model costs approximately $0.19 to run on Replicate, or 5 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 15 minutes. The predict time for this model varies significantly based on the inputs.

Readme

Lucid Sonic Dreams (modified to work with StyleGAN XL )

Lucid Sonic Dreams syncs GAN-generated visuals to music. By default, it uses NVLabs StyleGAN2, with pre-trained models lifted from Justin Pinkney’s consolidated repository. Custom weights and other GAN architectures can be used as well.

Sample output can be found on YouTube and Instagram.

Installation

This implementation has been teston on Python 3.6 and 3.7. As per NVLabs’ TensorFlow implementation of StyleGAN2, TensorFlow 1.15 is required. TensorFlow 2.x is not supported.

To install, simply run:

pip install lucidsonicdreams

Usage

You may refer to the Lucid Sonic Dreams Tutorial Notebook for full parameter descriptions and sample code templates. A basic visualization snippet is also found below.

Basic Visualization

from lucidsonicdreams import LucidSonicDream


L = LucidSonicDream(song = 'song.mp3',
                    style = 'abstract photos')

L.hallucinate(file_name = 'song.mp4')