zsxkib / animatediff-prompt-travel
π¨AnimateDiff Prompt Travelπ§ Seamlessly Navigate and Animate Between Text-to-Image Prompts for Dynamic Visual Narratives
Prediction
zsxkib/animatediff-prompt-travel:1b8a8f2725c03b1ff4a0b960079899131c384149d1feba45e4ef43653deb3b5fID2z4ra7db7lh3egx5xhiv5e33kyStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedInput
- seed
- -1
- steps
- 25
- width
- 256
- frames
- 128
- height
- 384
- context
- 16
- clip_skip
- 2
- scheduler
- k_dpmpp_sde
- base_model
- majicmixRealistic_v5Preview
- prompt_map
- 0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off
- head_prompt
- masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky
- tail_prompt
- dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere
- output_format
- mp4
- guidance_scale
- 7.5
- negative_prompt
- (worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island
- prompt_fixed_ratio
- 0.5
- custom_base_model_url
- playback_frames_per_second
- 8
{ "seed": -1, "steps": 25, "width": 256, "frames": 128, "height": 384, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off", "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 }
Install Replicateβs Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; import fs from "node:fs"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "zsxkib/animatediff-prompt-travel:1b8a8f2725c03b1ff4a0b960079899131c384149d1feba45e4ef43653deb3b5f", { input: { seed: -1, steps: 25, width: 256, frames: 128, height: 384, context: 16, clip_skip: 2, scheduler: "k_dpmpp_sde", base_model: "majicmixRealistic_v5Preview", prompt_map: "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off", head_prompt: "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", tail_prompt: "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", output_format: "mp4", guidance_scale: 7.5, negative_prompt: "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island", prompt_fixed_ratio: 0.5, custom_base_model_url: "", playback_frames_per_second: 8 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicateβs Python client library:pip install replicate
Import the client:import replicate
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "zsxkib/animatediff-prompt-travel:1b8a8f2725c03b1ff4a0b960079899131c384149d1feba45e4ef43653deb3b5f", input={ "seed": -1, "steps": 25, "width": 256, "frames": 128, "height": 384, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off", "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "zsxkib/animatediff-prompt-travel:1b8a8f2725c03b1ff4a0b960079899131c384149d1feba45e4ef43653deb3b5f", "input": { "seed": -1, "steps": 25, "width": 256, "frames": 128, "height": 384, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off", "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship\'s lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicateβs HTTP API reference docs.
Output
{ "completed_at": "2023-10-19T14:48:20.515199Z", "created_at": "2023-10-19T14:44:39.293493Z", "data_removed": false, "error": null, "id": "2z4ra7db7lh3egx5xhiv5e33ky", "input": { "seed": -1, "steps": 25, "width": 256, "frames": 128, "height": 384, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: ship steadily moving,((waves crashing against the ship:1.0)) | 32: (((lightning strikes))), distant thunder, ship rocked by waves | 64: ship silhouette,(((heavy rain))),wind howling, waves rising higher | 96: ship navigating through the storm, rain easing off", "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 }, "logs": "--------------------------------------------------------------------------------\n{\n\"name\": \"sample\",\n\"path\": \"share/Stable-diffusion/majicmixRealistic_v5Preview.safetensors\",\n\"motion_module\": \"models/motion-module/mm_sd_v15_v2.ckpt\",\n\"compile\": false,\n\"seed\": [\n-1\n],\n\"scheduler\": \"k_dpmpp_sde\",\n\"steps\": 25,\n\"guidance_scale\": 7.5,\n\"clip_skip\": 2,\n\"prompt_fixed_ratio\": 0.5,\n\"head_prompt\": \"masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky\",\n\"prompt_map\": {\n\"0\": \"ship steadily moving,((waves crashing against the ship:1.0))\", \"32\": \"(((lightning strikes))), distant thunder, ship rocked by waves\", \"64\": \"ship silhouette,(((heavy rain))),wind howling, waves rising higher\", \"96\": \"ship navigating through the storm, rain easing off\"\n},\n\"tail_prompt\": \"dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere\",\n\"n_prompt\": [\n\"(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island\"\n],\n\"output\":{\n\"format\" : \"mp4\",\n\"fps\" : 8,\n\"encode_param\":{\n\"crf\": 10\n}\n}\n}\n--------------------------------------------------------------------------------\nRunning command: animatediff generate -c config/prompts/custom_prompt_travel.json -W 256 -H 384 -L 128 -C 16\n14:46:30 INFO Using generation config: cli.py:288\nconfig/prompts/custom_prompt_travel.json\n14:46:31 INFO is_v2=True util.py:360\nINFO Using base model: runwayml/stable-diffusion-v1-5 cli.py:306\nINFO Will save outputs to cli.py:314\n./output/2023-10-19T14-46-31-sample-majicmixrealist\nic_v5preview\nINFO Checking motion module... generate.py:321\nINFO Loading tokenizer... generate.py:335\nINFO Loading text encoder... generate.py:337\n14:46:32 INFO Loading VAE... generate.py:339\nINFO Loading UNet... generate.py:341\n14:46:41 INFO Loaded 453.20928M-parameter motion module unet.py:578\nINFO Using scheduler \"k_dpmpp_sde\" generate.py:353\n(DPMSolverSinglestepScheduler)\nINFO Loading weights from generate.py:358\n/src/data/share/Stable-diffusion/majicmixReali\nstic_v5Preview.safetensors\n14:46:45 INFO Merging weights into UNet... generate.py:375\nINFO Creating AnimationPipeline... generate.py:421\nINFO No TI embeddings found ti.py:102\nINFO Sending pipeline to device \"cuda\" pipeline.py:22\n14:46:46 INFO Selected data types: unet_dtype=torch.float16, device.py:90\ntenc_dtype=torch.float16,\nvae_dtype=torch.bfloat16\nINFO Using channels_last memory format for UNet and device.py:111\nVAE\n14:46:48 INFO Saving prompt config to output directory cli.py:367\nINFO Initialization complete! cli.py:375\nINFO Generating 1 animations cli.py:376\nINFO Running generation 1 of 1 cli.py:386\nINFO Generation seed: 2091610190647688252 cli.py:392\n100% ββββββββββββββββββββββββββββββββββ 275/275 [ 0:01:17 < 0:00:00 , 4 it/s ]\n14:48:10 INFO Generation complete, saving... generate.py:846\nSaving frames to 00-2091610190647688252 98% βββββΈ 126/β¦ [ 0:0β¦ < 0:00β¦ , 26 ]\nit/s\n14:48:15 INFO Creating ffmpeg encoder... generate.py:773\nINFO Encoding interpolated frames with ffmpeg... generate.py:783\n14:48:17 INFO Saved sample to generate.py:857\noutput/2023-10-19T14-46-31-sample-majicmixreal\nistic_v5preview/00_2091610190647688252_masterp\niece_best-quality_a-haunting-and-detailed-d\nINFO Generation complete! cli.py:451\nINFO Done, exiting... cli.py:457\nError: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\nDownloading (β¦)lve/main/config.json: 0%| | 0.00/4.52k [00:00<?, ?B/s]\nDownloading (β¦)lve/main/config.json: 100%|ββββββββββ| 4.52k/4.52k [00:00<00:00, 12.7MB/s]\nDownloading (β¦)okenizer_config.json: 0%| | 0.00/905 [00:00<?, ?B/s]\nDownloading (β¦)okenizer_config.json: 100%|ββββββββββ| 905/905 [00:00<00:00, 5.39MB/s]\nDownloading (β¦)olve/main/vocab.json: 0%| | 0.00/961k [00:00<?, ?B/s]\nDownloading (β¦)olve/main/vocab.json: 100%|ββββββββββ| 961k/961k [00:00<00:00, 4.15MB/s]\nDownloading (β¦)olve/main/vocab.json: 100%|ββββββββββ| 961k/961k [00:00<00:00, 4.13MB/s]\nDownloading (β¦)olve/main/merges.txt: 0%| | 0.00/525k [00:00<?, ?B/s]\nDownloading (β¦)olve/main/merges.txt: 100%|ββββββββββ| 525k/525k [00:00<00:00, 10.1MB/s]\nDownloading (β¦)cial_tokens_map.json: 0%| | 0.00/389 [00:00<?, ?B/s]\nDownloading (β¦)cial_tokens_map.json: 100%|ββββββββββ| 389/389 [00:00<00:00, 2.47MB/s]\nDownloading (β¦)/main/tokenizer.json: 0%| | 0.00/2.22M [00:00<?, ?B/s]\nDownloading (β¦)/main/tokenizer.json: 100%|ββββββββββ| 2.22M/2.22M [00:00<00:00, 24.5MB/s]\nffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers\nbuilt with gcc 11 (Ubuntu 11.2.0-19ubuntu1)\nconfiguration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared\nlibavutil 56. 70.100 / 56. 70.100\nlibavcodec 58.134.100 / 58.134.100\nlibavformat 58. 76.100 / 58. 76.100\nlibavdevice 58. 13.100 / 58. 13.100\nlibavfilter 7.110.100 / 7.110.100\nlibswscale 5. 9.100 / 5. 9.100\nlibswresample 3. 9.100 / 3. 9.100\nlibpostproc 55. 9.100 / 55. 9.100\nInput #0, image2, from '/src/output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00-2091610190647688252/%08d.png':\nDuration: 00:00:16.00, start: 0.000000, bitrate: N/A\nStream #0:0: Video: png, rgb24(pc), 256x384, 8 fps, 8 tbr, 8 tbn, 8 tbc\nStream mapping:\nStream #0:0 (png) -> fps\nfps -> Stream #0:0 (libx265)\nPress [q] to stop, [?] for help\nx265 [info]: HEVC encoder version 3.5+1-f0c1022b6\nx265 [info]: build info [Linux][GCC 11.2.0][64 bit] 8bit+10bit+12bit\nx265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2\nx265 [info]: Main profile, Level-2 (Main tier)\nx265 [info]: Thread pool 0 using 64 threads on numa nodes 0,1,2,3,4,5,6,7\nx265 [info]: Thread pool 1 using 32 threads on numa nodes 0,1,2,3,4,5,6,7\nx265 [info]: Slices : 1\nx265 [info]: frame threads / pool features : 5 / wpp(6 rows)\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nx265 [warning]: Source height < 720p; disabling lookahead-slices\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nx265 [info]: Coding QT: max CU size, min CU size : 64 / 8\nx265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra\nx265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3\nx265 [info]: Keyframe min / max / scenecut / bias : 8 / 250 / 40 / 5.00\nx265 [info]: Lookahead / bframes / badapt : 20 / 6 / 2\nx265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0\nx265 [info]: References / ref-limit cu / depth : 3 / off / on\nx265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 0.4 / 32 / 1\nx265 [info]: Rate Control / qCompress : CRF-10.0 / 0.60\nx265 [info]: tools: rd=3 psy-rd=0.40 early-skip rskip mode=1 signhide tmvp\nx265 [info]: tools: b-intra strong-intra-smoothing deblock(tC=1:B=1) sao\nOutput #0, mp4, to '/src/output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00_2091610190647688252_masterpiece_best-quality_a-haunting-and-detailed-d.mp4':\nMetadata:\nencoder : Lavf58.76.100\nStream #0:0: Video: hevc (hev1 / 0x31766568), yuv420p(tv, progressive), 256x384, q=2-31, 8 fps, 16384 tbn\nMetadata:\nencoder : Lavc58.134.100 libx265\nSide data:\ncpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A\nframe= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x\nframe= 65 fps=0.0 q=12.7 size= 256kB time=00:00:03.62 bitrate= 578.6kbits/s speed=6.88x\nframe= 108 fps=103 q=12.4 size= 512kB time=00:00:09.00 bitrate= 466.1kbits/s speed=8.62x\nframe= 128 fps= 79 q=12.4 Lsize= 1206kB time=00:00:15.62 bitrate= 632.0kbits/s speed= 9.7x\nvideo:1201kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.384719%\nx265 [info]: frame I: 2, Avg QP:9.70 kb/s: 1038.56\nx265 [info]: frame P: 26, Avg QP:9.67 kb/s: 821.73\nx265 [info]: frame B: 100, Avg QP:12.11 kb/s: 552.28\nx265 [info]: Weighted P-Frames: Y:7.7% UV:7.7%\nx265 [info]: consecutive B-frames: 7.1% 0.0% 7.1% 35.7% 17.9% 28.6% 3.6%\nencoded 128 frames in 1.56s (81.93 fps), 614.61 kb/s, Avg QP:11.58\nIdentifying the GIF path from the generated outputs...\nIdentified directory: output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview\nIdentified Media Path: output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00_2091610190647688252_masterpiece_best-quality_a-haunting-and-detailed-d.mp4", "metrics": { "predict_time": 114.151252, "total_time": 221.221706 }, "output": "https://pbxt.replicate.delivery/2xPrJEruqw6DAJSDUYNMzWT2m0g3qMCJdzSuqXOFzi6sz7bE/00_2091610190647688252_masterpiece_best-quality_a-haunting-and-detailed-d.mp4", "started_at": "2023-10-19T14:46:26.363947Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/2z4ra7db7lh3egx5xhiv5e33ky", "cancel": "https://api.replicate.com/v1/predictions/2z4ra7db7lh3egx5xhiv5e33ky/cancel" }, "version": "1b8a8f2725c03b1ff4a0b960079899131c384149d1feba45e4ef43653deb3b5f" }
Generated in-------------------------------------------------------------------------------- { "name": "sample", "path": "share/Stable-diffusion/majicmixRealistic_v5Preview.safetensors", "motion_module": "models/motion-module/mm_sd_v15_v2.ckpt", "compile": false, "seed": [ -1 ], "scheduler": "k_dpmpp_sde", "steps": 25, "guidance_scale": 7.5, "clip_skip": 2, "prompt_fixed_ratio": 0.5, "head_prompt": "masterpiece, best quality, a haunting and detailed depiction of a ship at sea, battered by waves, ominous,((dark clouds:1.3)),distant lightning, rough seas, rain, silhouette of the ship against the stormy sky", "prompt_map": { "0": "ship steadily moving,((waves crashing against the ship:1.0))", "32": "(((lightning strikes))), distant thunder, ship rocked by waves", "64": "ship silhouette,(((heavy rain))),wind howling, waves rising higher", "96": "ship navigating through the storm, rain easing off" }, "tail_prompt": "dark horizon, flashes of lightning illuminating the ship, sailors working hard, ship's lanterns flickering, eerie, mysterious, sails flapping loudly, stormy atmosphere", "n_prompt": [ "(worst quality, low quality:1.4), black and white, b&w, sunny, clear skies, calm seas, beach, daytime, ((bright colors)), cartoonish, modern ships, sketchy, unfinished, modern buildings, trees, island" ], "output":{ "format" : "mp4", "fps" : 8, "encode_param":{ "crf": 10 } } } -------------------------------------------------------------------------------- Running command: animatediff generate -c config/prompts/custom_prompt_travel.json -W 256 -H 384 -L 128 -C 16 14:46:30 INFO Using generation config: cli.py:288 config/prompts/custom_prompt_travel.json 14:46:31 INFO is_v2=True util.py:360 INFO Using base model: runwayml/stable-diffusion-v1-5 cli.py:306 INFO Will save outputs to cli.py:314 ./output/2023-10-19T14-46-31-sample-majicmixrealist ic_v5preview INFO Checking motion module... generate.py:321 INFO Loading tokenizer... generate.py:335 INFO Loading text encoder... generate.py:337 14:46:32 INFO Loading VAE... generate.py:339 INFO Loading UNet... generate.py:341 14:46:41 INFO Loaded 453.20928M-parameter motion module unet.py:578 INFO Using scheduler "k_dpmpp_sde" generate.py:353 (DPMSolverSinglestepScheduler) INFO Loading weights from generate.py:358 /src/data/share/Stable-diffusion/majicmixReali stic_v5Preview.safetensors 14:46:45 INFO Merging weights into UNet... generate.py:375 INFO Creating AnimationPipeline... generate.py:421 INFO No TI embeddings found ti.py:102 INFO Sending pipeline to device "cuda" pipeline.py:22 14:46:46 INFO Selected data types: unet_dtype=torch.float16, device.py:90 tenc_dtype=torch.float16, vae_dtype=torch.bfloat16 INFO Using channels_last memory format for UNet and device.py:111 VAE 14:46:48 INFO Saving prompt config to output directory cli.py:367 INFO Initialization complete! cli.py:375 INFO Generating 1 animations cli.py:376 INFO Running generation 1 of 1 cli.py:386 INFO Generation seed: 2091610190647688252 cli.py:392 100% ββββββββββββββββββββββββββββββββββ 275/275 [ 0:01:17 < 0:00:00 , 4 it/s ] 14:48:10 INFO Generation complete, saving... generate.py:846 Saving frames to 00-2091610190647688252 98% βββββΈ 126/β¦ [ 0:0β¦ < 0:00β¦ , 26 ] it/s 14:48:15 INFO Creating ffmpeg encoder... generate.py:773 INFO Encoding interpolated frames with ffmpeg... generate.py:783 14:48:17 INFO Saved sample to generate.py:857 output/2023-10-19T14-46-31-sample-majicmixreal istic_v5preview/00_2091610190647688252_masterp iece_best-quality_a-haunting-and-detailed-d INFO Generation complete! cli.py:451 INFO Done, exiting... cli.py:457 Error: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Downloading (β¦)lve/main/config.json: 0%| | 0.00/4.52k [00:00<?, ?B/s] Downloading (β¦)lve/main/config.json: 100%|ββββββββββ| 4.52k/4.52k [00:00<00:00, 12.7MB/s] Downloading (β¦)okenizer_config.json: 0%| | 0.00/905 [00:00<?, ?B/s] Downloading (β¦)okenizer_config.json: 100%|ββββββββββ| 905/905 [00:00<00:00, 5.39MB/s] Downloading (β¦)olve/main/vocab.json: 0%| | 0.00/961k [00:00<?, ?B/s] Downloading (β¦)olve/main/vocab.json: 100%|ββββββββββ| 961k/961k [00:00<00:00, 4.15MB/s] Downloading (β¦)olve/main/vocab.json: 100%|ββββββββββ| 961k/961k [00:00<00:00, 4.13MB/s] Downloading (β¦)olve/main/merges.txt: 0%| | 0.00/525k [00:00<?, ?B/s] Downloading (β¦)olve/main/merges.txt: 100%|ββββββββββ| 525k/525k [00:00<00:00, 10.1MB/s] Downloading (β¦)cial_tokens_map.json: 0%| | 0.00/389 [00:00<?, ?B/s] Downloading (β¦)cial_tokens_map.json: 100%|ββββββββββ| 389/389 [00:00<00:00, 2.47MB/s] Downloading (β¦)/main/tokenizer.json: 0%| | 0.00/2.22M [00:00<?, ?B/s] Downloading (β¦)/main/tokenizer.json: 100%|ββββββββββ| 2.22M/2.22M [00:00<00:00, 24.5MB/s] ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, image2, from '/src/output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00-2091610190647688252/%08d.png': Duration: 00:00:16.00, start: 0.000000, bitrate: N/A Stream #0:0: Video: png, rgb24(pc), 256x384, 8 fps, 8 tbr, 8 tbn, 8 tbc Stream mapping: Stream #0:0 (png) -> fps fps -> Stream #0:0 (libx265) Press [q] to stop, [?] for help x265 [info]: HEVC encoder version 3.5+1-f0c1022b6 x265 [info]: build info [Linux][GCC 11.2.0][64 bit] 8bit+10bit+12bit x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 x265 [info]: Main profile, Level-2 (Main tier) x265 [info]: Thread pool 0 using 64 threads on numa nodes 0,1,2,3,4,5,6,7 x265 [info]: Thread pool 1 using 32 threads on numa nodes 0,1,2,3,4,5,6,7 x265 [info]: Slices : 1 x265 [info]: frame threads / pool features : 5 / wpp(6 rows) set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted x265 [warning]: Source height < 720p; disabling lookahead-slices set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted x265 [info]: Coding QT: max CU size, min CU size : 64 / 8 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3 x265 [info]: Keyframe min / max / scenecut / bias : 8 / 250 / 40 / 5.00 x265 [info]: Lookahead / bframes / badapt : 20 / 6 / 2 x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0 x265 [info]: References / ref-limit cu / depth : 3 / off / on x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 0.4 / 32 / 1 x265 [info]: Rate Control / qCompress : CRF-10.0 / 0.60 x265 [info]: tools: rd=3 psy-rd=0.40 early-skip rskip mode=1 signhide tmvp x265 [info]: tools: b-intra strong-intra-smoothing deblock(tC=1:B=1) sao Output #0, mp4, to '/src/output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00_2091610190647688252_masterpiece_best-quality_a-haunting-and-detailed-d.mp4': Metadata: encoder : Lavf58.76.100 Stream #0:0: Video: hevc (hev1 / 0x31766568), yuv420p(tv, progressive), 256x384, q=2-31, 8 fps, 16384 tbn Metadata: encoder : Lavc58.134.100 libx265 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x frame= 65 fps=0.0 q=12.7 size= 256kB time=00:00:03.62 bitrate= 578.6kbits/s speed=6.88x frame= 108 fps=103 q=12.4 size= 512kB time=00:00:09.00 bitrate= 466.1kbits/s speed=8.62x frame= 128 fps= 79 q=12.4 Lsize= 1206kB time=00:00:15.62 bitrate= 632.0kbits/s speed= 9.7x video:1201kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.384719% x265 [info]: frame I: 2, Avg QP:9.70 kb/s: 1038.56 x265 [info]: frame P: 26, Avg QP:9.67 kb/s: 821.73 x265 [info]: frame B: 100, Avg QP:12.11 kb/s: 552.28 x265 [info]: Weighted P-Frames: Y:7.7% UV:7.7% x265 [info]: consecutive B-frames: 7.1% 0.0% 7.1% 35.7% 17.9% 28.6% 3.6% encoded 128 frames in 1.56s (81.93 fps), 614.61 kb/s, Avg QP:11.58 Identifying the GIF path from the generated outputs... Identified directory: output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview Identified Media Path: output/2023-10-19T14-46-31-sample-majicmixrealistic_v5preview/00_2091610190647688252_masterpiece_best-quality_a-haunting-and-detailed-d.mp4
Prediction
zsxkib/animatediff-prompt-travel:cd225f9838da14780f2f555ce9dca70f4dcad304b0ded87fd13148c4b74b198dIDe74gjm3bhtxfk7mbbht6fdu7nmStatusSucceededSourceWebHardwareA40 (Large)Total durationCreatedby @fofrInput
- seed
- -1
- steps
- 25
- width
- 512
- frames
- 128
- height
- 768
- context
- 16
- clip_skip
- 2
- scheduler
- k_dpmpp_sde
- base_model
- majicmixRealistic_v5Preview
- prompt_map
- 0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays
- head_prompt
- a photo of a sailboat on a stormy sea
- tail_prompt
- 8k, epic, beautiful
- output_format
- mp4
- guidance_scale
- 7.5
- negative_prompt
- ugly, disfigured, broken
- prompt_fixed_ratio
- 0.5
- custom_base_model_url
- playback_frames_per_second
- 8
{ "seed": -1, "steps": 25, "width": 512, "frames": 128, "height": 768, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays", "head_prompt": "a photo of a sailboat on a stormy sea", "tail_prompt": "8k, epic, beautiful", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "ugly, disfigured, broken", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 }
Install Replicateβs Node.js client library:npm install replicate
Import and set up the client:import Replicate from "replicate"; import fs from "node:fs"; const replicate = new Replicate({ auth: process.env.REPLICATE_API_TOKEN, });
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run( "zsxkib/animatediff-prompt-travel:cd225f9838da14780f2f555ce9dca70f4dcad304b0ded87fd13148c4b74b198d", { input: { seed: -1, steps: 25, width: 512, frames: 128, height: 768, context: 16, clip_skip: 2, scheduler: "k_dpmpp_sde", base_model: "majicmixRealistic_v5Preview", prompt_map: "0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays", head_prompt: "a photo of a sailboat on a stormy sea", tail_prompt: "8k, epic, beautiful", output_format: "mp4", guidance_scale: 7.5, negative_prompt: "ugly, disfigured, broken", prompt_fixed_ratio: 0.5, custom_base_model_url: "", playback_frames_per_second: 8 } } ); // To access the file URL: console.log(output.url()); //=> "http://example.com" // To write the file to disk: fs.writeFile("my-image.png", output);
To learn more, take a look at the guide on getting started with Node.js.
Install Replicateβs Python client library:pip install replicate
Import the client:import replicate
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run( "zsxkib/animatediff-prompt-travel:cd225f9838da14780f2f555ce9dca70f4dcad304b0ded87fd13148c4b74b198d", input={ "seed": -1, "steps": 25, "width": 512, "frames": 128, "height": 768, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays", "head_prompt": "a photo of a sailboat on a stormy sea", "tail_prompt": "8k, epic, beautiful", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "ugly, disfigured, broken", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 } ) print(output)
To learn more, take a look at the guide on getting started with Python.
Run zsxkib/animatediff-prompt-travel using Replicateβs API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN" \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "version": "zsxkib/animatediff-prompt-travel:cd225f9838da14780f2f555ce9dca70f4dcad304b0ded87fd13148c4b74b198d", "input": { "seed": -1, "steps": 25, "width": 512, "frames": 128, "height": 768, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays", "head_prompt": "a photo of a sailboat on a stormy sea", "tail_prompt": "8k, epic, beautiful", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "ugly, disfigured, broken", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 } }' \ https://api.replicate.com/v1/predictions
To learn more, take a look at Replicateβs HTTP API reference docs.
Output
{ "completed_at": "2023-10-19T14:12:55.966851Z", "created_at": "2023-10-19T14:06:27.696368Z", "data_removed": false, "error": null, "id": "e74gjm3bhtxfk7mbbht6fdu7nm", "input": { "seed": -1, "steps": 25, "width": 512, "frames": 128, "height": 768, "context": 16, "clip_skip": 2, "scheduler": "k_dpmpp_sde", "base_model": "majicmixRealistic_v5Preview", "prompt_map": "0: waves crashing against the ship | 32: lightning strikes, ship rocked by waves | 64: water calming, clouds clearing | 96: blue skies and sun rays", "head_prompt": "a photo of a sailboat on a stormy sea", "tail_prompt": "8k, epic, beautiful", "output_format": "mp4", "guidance_scale": 7.5, "negative_prompt": "ugly, disfigured, broken", "prompt_fixed_ratio": 0.5, "custom_base_model_url": "", "playback_frames_per_second": 8 }, "logs": "--------------------------------------------------------------------------------\n{\n\"name\": \"sample\",\n\"path\": \"share/Stable-diffusion/majicmixRealistic_v5Preview.safetensors\",\n\"motion_module\": \"models/motion-module/mm_sd_v15_v2.ckpt\",\n\"compile\": false,\n\"seed\": [\n-1\n],\n\"scheduler\": \"k_dpmpp_sde\",\n\"steps\": 25,\n\"guidance_scale\": 7.5,\n\"clip_skip\": 2,\n\"prompt_fixed_ratio\": 0.5,\n\"head_prompt\": \"a photo of a sailboat on a stormy sea\",\n\"prompt_map\": {\n\"0\": \"waves crashing against the ship\", \"32\": \"lightning strikes, ship rocked by waves\", \"64\": \"water calming, clouds clearing\", \"96\": \"blue skies and sun rays\"\n},\n\"tail_prompt\": \"8k, epic, beautiful\",\n\"n_prompt\": [\n\"ugly, disfigured, broken\"\n],\n\"output\":{\n\"format\" : \"mp4\",\n\"fps\" : 8,\n\"encode_param\":{\n\"crf\": 10\n}\n}\n}\n--------------------------------------------------------------------------------\nRunning command: animatediff generate -c config/prompts/custom_prompt_travel.json -W 512 -H 768 -L 128 -C 16\n14:06:31 INFO Using generation config: cli.py:288\nconfig/prompts/custom_prompt_travel.json\n14:06:32 INFO is_v2=True util.py:360\nINFO Using base model: runwayml/stable-diffusion-v1-5 cli.py:306\nINFO Will save outputs to cli.py:314\n./output/2023-10-19T14-06-32-sample-majicmixrealist\nic_v5preview\nINFO Checking motion module... generate.py:321\nINFO Loading tokenizer... generate.py:335\nINFO Loading text encoder... generate.py:337\n14:06:33 INFO Loading VAE... generate.py:339\nINFO Loading UNet... generate.py:341\n14:06:41 INFO Loaded 453.20928M-parameter motion module unet.py:578\nINFO Using scheduler \"k_dpmpp_sde\" generate.py:353\n(DPMSolverSinglestepScheduler)\nINFO Loading weights from generate.py:358\n/src/data/share/Stable-diffusion/majicmixReali\nstic_v5Preview.safetensors\n14:06:43 INFO Merging weights into UNet... generate.py:375\n14:06:44 INFO Creating AnimationPipeline... generate.py:421\nINFO No TI embeddings found ti.py:102\nINFO Sending pipeline to device \"cuda\" pipeline.py:22\nINFO Selected data types: unet_dtype=torch.float16, device.py:90\ntenc_dtype=torch.float16,\nvae_dtype=torch.bfloat16\nINFO Using channels_last memory format for UNet and device.py:111\nVAE\n14:06:46 INFO Saving prompt config to output directory cli.py:367\nINFO Initialization complete! cli.py:375\nINFO Generating 1 animations cli.py:376\nINFO Running generation 1 of 1 cli.py:386\nINFO Generation seed: 1236748965417470028 cli.py:392\n100% ββββββββββββββββββββββββββββββββββ 275/275 [ 0:05:27 < 0:00:00 , 1 it/s ]\n14:12:32 INFO Generation complete, saving... generate.py:846\nSaving frames to 00-1236748965417470028 100% βββββ 128β¦ [ 0:00β¦ < 0:0β¦ , 8 ]\nit/s\n14:12:49 INFO Creating ffmpeg encoder... generate.py:773\nINFO Encoding interpolated frames with ffmpeg... generate.py:783\n14:12:53 INFO Saved sample to generate.py:857\noutput/2023-10-19T14-06-32-sample-majicmixreal\nistic_v5preview/00_1236748965417470028_a-photo\n-of-a-sailboat-on-a-stormy-sea_waves-crashi\nINFO Generation complete! cli.py:451\nINFO Done, exiting... cli.py:457\nError: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\nffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers\nbuilt with gcc 11 (Ubuntu 11.2.0-19ubuntu1)\nconfiguration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared\nlibavutil 56. 70.100 / 56. 70.100\nlibavcodec 58.134.100 / 58.134.100\nlibavformat 58. 76.100 / 58. 76.100\nlibavdevice 58. 13.100 / 58. 13.100\nlibavfilter 7.110.100 / 7.110.100\nlibswscale 5. 9.100 / 5. 9.100\nlibswresample 3. 9.100 / 3. 9.100\nlibpostproc 55. 9.100 / 55. 9.100\nInput #0, image2, from '/src/output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00-1236748965417470028/%08d.png':\nDuration: 00:00:16.00, start: 0.000000, bitrate: N/A\nStream #0:0: Video: png, rgb24(pc), 512x768, 8 fps, 8 tbr, 8 tbn, 8 tbc\nStream mapping:\nStream #0:0 (png) -> fps\nfps -> Stream #0:0 (libx265)\nPress [q] to stop, [?] for help\nx265 [info]: HEVC encoder version 3.5+1-f0c1022b6\nx265 [info]: build info [Linux][GCC 11.2.0][64 bit] 8bit+10bit+12bit\nx265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2\nx265 [info]: Main profile, Level-3 (Main tier)\nx265 [info]: Thread pool 0 using 64 threads on numa nodes 0,1,2,3,4,5,6,7\nx265 [info]: Thread pool 1 using 32 threads on numa nodes 0,1,2,3,4,5,6,7\nx265 [info]: Slices : 1\nx265 [info]: frame threads / pool features : 5 / wpp(12 rows)\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nset_mempolicy: Operation not permitted\nx265 [info]: Coding QT: max CU size, min CU size : 64 / 8\nx265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra\nx265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3\nx265 [info]: Keyframe min / max / scenecut / bias : 8 / 250 / 40 / 5.00\nx265 [info]: Lookahead / bframes / badapt : 20 / 6 / 2\nx265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0\nx265 [info]: References / ref-limit cu / depth : 3 / off / on\nx265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 0.4 / 32 / 1\nx265 [info]: Rate Control / qCompress : CRF-10.0 / 0.60\nx265 [info]: tools: rd=3 psy-rd=0.40 early-skip rskip mode=1 signhide tmvp\nx265 [info]: tools: b-intra strong-intra-smoothing lslices=4 deblock(tC=1:B=1)\nx265 [info]: tools: sao\nOutput #0, mp4, to '/src/output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00_1236748965417470028_a-photo-of-a-sailboat-on-a-stormy-sea_waves-crashi.mp4':\nMetadata:\nencoder : Lavf58.76.100\nStream #0:0: Video: hevc (hev1 / 0x31766568), yuv420p(tv, progressive), 512x768, q=2-31, 8 fps, 16384 tbn\nMetadata:\nencoder : Lavc58.134.100 libx265\nSide data:\ncpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A\nframe= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x\nframe= 42 fps=0.0 q=12.6 size= 256kB time=00:00:00.75 bitrate=2796.4kbits/s speed=1.26x\nframe= 62 fps= 55 q=12.5 size= 1280kB time=00:00:03.25 bitrate=3226.4kbits/s speed=2.88x\nframe= 82 fps= 50 q=12.7 size= 2048kB time=00:00:05.75 bitrate=2917.8kbits/s speed=3.48x\nframe= 102 fps= 47 q=12.5 size= 2816kB time=00:00:08.25 bitrate=2796.2kbits/s speed=3.78x\nframe= 121 fps= 45 q=12.4 size= 3328kB time=00:00:10.62 bitrate=2565.9kbits/s speed=3.95x\nframe= 128 fps= 34 q=12.5 Lsize= 4969kB time=00:00:15.62 bitrate=2605.1kbits/s speed= 4.2x\nvideo:4964kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.097421%\nx265 [info]: frame I: 1, Avg QP:9.76 kb/s: 4418.05\nx265 [info]: frame P: 32, Avg QP:9.73 kb/s: 3211.26\nx265 [info]: frame B: 95, Avg QP:12.01 kb/s: 2295.86\nx265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%\nx265 [info]: consecutive B-frames: 3.0% 0.0% 9.1% 81.8% 6.1% 0.0% 0.0%\nencoded 128 frames in 3.69s (34.69 fps), 2541.29 kb/s, Avg QP:11.42\nIdentifying the GIF path from the generated outputs...\nIdentified directory: output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview\nIdentified Media Path: output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00_1236748965417470028_a-photo-of-a-sailboat-on-a-stormy-sea_waves-crashi.mp4", "metrics": { "predict_time": 388.310027, "total_time": 388.270483 }, "output": "https://pbxt.replicate.delivery/flKSNJS9hv2aHafBc7aePA3W49o8ktL3GnxqgfOtOg3b26eNC/00_1236748965417470028_a-photo-of-a-sailboat-on-a-stormy-sea_waves-crashi.mp4", "started_at": "2023-10-19T14:06:27.656824Z", "status": "succeeded", "urls": { "get": "https://api.replicate.com/v1/predictions/e74gjm3bhtxfk7mbbht6fdu7nm", "cancel": "https://api.replicate.com/v1/predictions/e74gjm3bhtxfk7mbbht6fdu7nm/cancel" }, "version": "cd225f9838da14780f2f555ce9dca70f4dcad304b0ded87fd13148c4b74b198d" }
Generated in-------------------------------------------------------------------------------- { "name": "sample", "path": "share/Stable-diffusion/majicmixRealistic_v5Preview.safetensors", "motion_module": "models/motion-module/mm_sd_v15_v2.ckpt", "compile": false, "seed": [ -1 ], "scheduler": "k_dpmpp_sde", "steps": 25, "guidance_scale": 7.5, "clip_skip": 2, "prompt_fixed_ratio": 0.5, "head_prompt": "a photo of a sailboat on a stormy sea", "prompt_map": { "0": "waves crashing against the ship", "32": "lightning strikes, ship rocked by waves", "64": "water calming, clouds clearing", "96": "blue skies and sun rays" }, "tail_prompt": "8k, epic, beautiful", "n_prompt": [ "ugly, disfigured, broken" ], "output":{ "format" : "mp4", "fps" : 8, "encode_param":{ "crf": 10 } } } -------------------------------------------------------------------------------- Running command: animatediff generate -c config/prompts/custom_prompt_travel.json -W 512 -H 768 -L 128 -C 16 14:06:31 INFO Using generation config: cli.py:288 config/prompts/custom_prompt_travel.json 14:06:32 INFO is_v2=True util.py:360 INFO Using base model: runwayml/stable-diffusion-v1-5 cli.py:306 INFO Will save outputs to cli.py:314 ./output/2023-10-19T14-06-32-sample-majicmixrealist ic_v5preview INFO Checking motion module... generate.py:321 INFO Loading tokenizer... generate.py:335 INFO Loading text encoder... generate.py:337 14:06:33 INFO Loading VAE... generate.py:339 INFO Loading UNet... generate.py:341 14:06:41 INFO Loaded 453.20928M-parameter motion module unet.py:578 INFO Using scheduler "k_dpmpp_sde" generate.py:353 (DPMSolverSinglestepScheduler) INFO Loading weights from generate.py:358 /src/data/share/Stable-diffusion/majicmixReali stic_v5Preview.safetensors 14:06:43 INFO Merging weights into UNet... generate.py:375 14:06:44 INFO Creating AnimationPipeline... generate.py:421 INFO No TI embeddings found ti.py:102 INFO Sending pipeline to device "cuda" pipeline.py:22 INFO Selected data types: unet_dtype=torch.float16, device.py:90 tenc_dtype=torch.float16, vae_dtype=torch.bfloat16 INFO Using channels_last memory format for UNet and device.py:111 VAE 14:06:46 INFO Saving prompt config to output directory cli.py:367 INFO Initialization complete! cli.py:375 INFO Generating 1 animations cli.py:376 INFO Running generation 1 of 1 cli.py:386 INFO Generation seed: 1236748965417470028 cli.py:392 100% ββββββββββββββββββββββββββββββββββ 275/275 [ 0:05:27 < 0:00:00 , 1 it/s ] 14:12:32 INFO Generation complete, saving... generate.py:846 Saving frames to 00-1236748965417470028 100% βββββ 128β¦ [ 0:00β¦ < 0:0β¦ , 8 ] it/s 14:12:49 INFO Creating ffmpeg encoder... generate.py:773 INFO Encoding interpolated frames with ffmpeg... generate.py:783 14:12:53 INFO Saved sample to generate.py:857 output/2023-10-19T14-06-32-sample-majicmixreal istic_v5preview/00_1236748965417470028_a-photo -of-a-sailboat-on-a-stormy-sea_waves-crashi INFO Generation complete! cli.py:451 INFO Done, exiting... cli.py:457 Error: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, image2, from '/src/output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00-1236748965417470028/%08d.png': Duration: 00:00:16.00, start: 0.000000, bitrate: N/A Stream #0:0: Video: png, rgb24(pc), 512x768, 8 fps, 8 tbr, 8 tbn, 8 tbc Stream mapping: Stream #0:0 (png) -> fps fps -> Stream #0:0 (libx265) Press [q] to stop, [?] for help x265 [info]: HEVC encoder version 3.5+1-f0c1022b6 x265 [info]: build info [Linux][GCC 11.2.0][64 bit] 8bit+10bit+12bit x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 x265 [info]: Main profile, Level-3 (Main tier) x265 [info]: Thread pool 0 using 64 threads on numa nodes 0,1,2,3,4,5,6,7 x265 [info]: Thread pool 1 using 32 threads on numa nodes 0,1,2,3,4,5,6,7 x265 [info]: Slices : 1 x265 [info]: frame threads / pool features : 5 / wpp(12 rows) set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted set_mempolicy: Operation not permitted x265 [info]: Coding QT: max CU size, min CU size : 64 / 8 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3 x265 [info]: Keyframe min / max / scenecut / bias : 8 / 250 / 40 / 5.00 x265 [info]: Lookahead / bframes / badapt : 20 / 6 / 2 x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0 x265 [info]: References / ref-limit cu / depth : 3 / off / on x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 0.4 / 32 / 1 x265 [info]: Rate Control / qCompress : CRF-10.0 / 0.60 x265 [info]: tools: rd=3 psy-rd=0.40 early-skip rskip mode=1 signhide tmvp x265 [info]: tools: b-intra strong-intra-smoothing lslices=4 deblock(tC=1:B=1) x265 [info]: tools: sao Output #0, mp4, to '/src/output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00_1236748965417470028_a-photo-of-a-sailboat-on-a-stormy-sea_waves-crashi.mp4': Metadata: encoder : Lavf58.76.100 Stream #0:0: Video: hevc (hev1 / 0x31766568), yuv420p(tv, progressive), 512x768, q=2-31, 8 fps, 16384 tbn Metadata: encoder : Lavc58.134.100 libx265 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x frame= 42 fps=0.0 q=12.6 size= 256kB time=00:00:00.75 bitrate=2796.4kbits/s speed=1.26x frame= 62 fps= 55 q=12.5 size= 1280kB time=00:00:03.25 bitrate=3226.4kbits/s speed=2.88x frame= 82 fps= 50 q=12.7 size= 2048kB time=00:00:05.75 bitrate=2917.8kbits/s speed=3.48x frame= 102 fps= 47 q=12.5 size= 2816kB time=00:00:08.25 bitrate=2796.2kbits/s speed=3.78x frame= 121 fps= 45 q=12.4 size= 3328kB time=00:00:10.62 bitrate=2565.9kbits/s speed=3.95x frame= 128 fps= 34 q=12.5 Lsize= 4969kB time=00:00:15.62 bitrate=2605.1kbits/s speed= 4.2x video:4964kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.097421% x265 [info]: frame I: 1, Avg QP:9.76 kb/s: 4418.05 x265 [info]: frame P: 32, Avg QP:9.73 kb/s: 3211.26 x265 [info]: frame B: 95, Avg QP:12.01 kb/s: 2295.86 x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0% x265 [info]: consecutive B-frames: 3.0% 0.0% 9.1% 81.8% 6.1% 0.0% 0.0% encoded 128 frames in 3.69s (34.69 fps), 2541.29 kb/s, Avg QP:11.42 Identifying the GIF path from the generated outputs... Identified directory: output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview Identified Media Path: output/2023-10-19T14-06-32-sample-majicmixrealistic_v5preview/00_1236748965417470028_a-photo-of-a-sailboat-on-a-stormy-sea_waves-crashi.mp4
Want to make some of these yourself?
Run this model