predict Loaded v3.x HD model. /tmp/tmpn1xr0wpiacidtho.mp4, 99.0 frames in total, 24.08905651195328FPS to 96.35622604781312FPS The audio will be merged after interpolation process 0%| | 0/99.0 [00:00 #0:0 (copy) Press [q] to stop, [?] for help size= 66kB time=00:00:04.10 bitrate= 131.9kbits/s speed= 199x video:0kB audio:64kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.068213% ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/interpolated_noaudio.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2mp41 encoder : Lavf58.35.100 Duration: 00:00:04.08, start: 0.000000, bitrate: 3148 kb/s Stream #0:0(und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 3143 kb/s, 96.36 fps, 96.36 tbr, 96356 tbn, 24089 tbc (default) Metadata: handler_name : VideoHandler Input #1, matroska,webm, from './temp/audio.mkv': Metadata: COMPATIBLE_BRANDS: qt MAJOR_BRAND : qt MINOR_VERSION : 0 ENCODER : Lavf58.29.100 Duration: 00:00:04.16, start: 0.000000, bitrate: 130 kb/s Stream #1:0: Audio: aac (LC), 44100 Hz, stereo, fltp (default) Metadata: HANDLER_NAME : Core Media Audio DURATION : 00:00:04.157000000 Output #0, mp4, to '/tmp/interpolated.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2mp41 encoder : Lavf58.29.100 Stream #0:0(und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=2-31, 3143 kb/s, 96.36 fps, 96.36 tbr, 96356 tbn, 96356 tbc (default) Metadata: handler_name : VideoHandler Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp (default) Metadata: HANDLER_NAME : Core Media Audio DURATION : 00:00:04.157000000 Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #1:0 -> #0:1 (copy) Press [q] to stop, [?] for help frame= 393 fps=0.0 q=-1.0 Lsize= 1635kB time=00:00:04.15 bitrate=3223.2kbits/s speed= 578x video:1565kB audio:64kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.379001% ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/interpolated.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2mp41 encoder : Lavf58.29.100 Duration: 00:00:04.18, start: 0.000000, bitrate: 3205 kb/s Stream #0:0(und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 3143 kb/s, 96.36 fps, 96.36 tbr, 96356 tbn, 24089 tbc (default) Metadata: handler_name : VideoHandler Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default) Metadata: handler_name : Core Media Audio Stream mapping: Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help [libx264 @ 0x55b57e0935c0] using SAR=1/1 [libx264 @ 0x55b57e0935c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x55b57e0935c0] profile High, level 3.1 [libx264 @ 0x55b57e0935c0] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=11 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to '/tmp/interpolated_2x.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2mp41 encoder : Lavf58.29.100 Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 96.36 fps, 24089 tbn, 96.36 tbc (default) Metadata: handler_name : VideoHandler encoder : Lavc58.54.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1 Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : Core Media Audio encoder : Lavc58.54.100 aac frame= 111 fps=0.0 q=30.0 size= 0kB time=00:00:01.20 bitrate= 0.3kbits/s speed=2.37x frame= 218 fps=215 q=30.0 size= 0kB time=00:00:02.32 bitrate= 0.2kbits/s speed=2.29x frame= 315 fps=207 q=30.0 size= 256kB time=00:00:03.32 bitrate= 631.7kbits/s speed=2.19x [mp4 @ 0x55b57e0a8dc0] Starting second pass: moving the moov atom to the beginning of the file frame= 393 fps=179 q=-1.0 Lsize= 449kB time=00:00:04.17 bitrate= 880.1kbits/s speed= 1.9x video:372kB audio:68kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.012863% [libx264 @ 0x55b57e0935c0] frame I:2 Avg QP:21.75 size: 7820 [libx264 @ 0x55b57e0935c0] frame P:129 Avg QP:22.82 size: 2025 [libx264 @ 0x55b57e0935c0] frame B:262 Avg QP:24.02 size: 395 [libx264 @ 0x55b57e0935c0] consecutive B-frames: 10.2% 2.5% 0.8% 86.5% [libx264 @ 0x55b57e0935c0] mb I I16..4: 29.4% 59.5% 11.1% [libx264 @ 0x55b57e0935c0] mb P I16..4: 4.2% 9.1% 0.2% P16..4: 38.6% 7.3% 3.4% 0.0% 0.0% skip:37.1% [libx264 @ 0x55b57e0935c0] mb B I16..4: 0.5% 0.7% 0.0% B16..8: 34.6% 0.8% 0.0% direct: 0.2% skip:63.2% L0:45.8% L1:52.5% BI: 1.7% [libx264 @ 0x55b57e0935c0] 8x8 transform intra:65.6% inter:75.4% [libx264 @ 0x55b57e0935c0] direct mvs spatial:97.7% temporal:2.3% [libx264 @ 0x55b57e0935c0] coded y,uvDC,uvAC intra: 30.3% 44.1% 3.3% inter: 2.4% 5.1% 0.1% [libx264 @ 0x55b57e0935c0] i16 v,h,dc,p: 22% 50% 19% 9% [libx264 @ 0x55b57e0935c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 28% 45% 2% 1% 1% 1% 1% 3% [libx264 @ 0x55b57e0935c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 35% 14% 2% 2% 2% 3% 2% 2% [libx264 @ 0x55b57e0935c0] i8c dc,h,v,p: 39% 37% 21% 3% [libx264 @ 0x55b57e0935c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x55b57e0935c0] ref P L0: 63.1% 10.0% 17.2% 5.2% 3.7% 0.7% [libx264 @ 0x55b57e0935c0] ref B L0: 80.9% 14.5% 3.7% 0.9% [libx264 @ 0x55b57e0935c0] ref B L1: 93.3% 6.7% [libx264 @ 0x55b57e0935c0] kb/s:746.20 [aac @ 0x55b57e078640] Qavg: 1051.758 total 96 -rw-rw-r-- 1 root root 3276 Jan 19 13:38 Colab_demo.ipynb -rw-rw-r-- 1 root root 1066 Jan 19 13:38 LICENSE -rw-rw-r-- 1 root root 9766 Jan 19 13:38 README.md drwxr-xr-x 1 root root 4096 Jan 20 15:26 __pycache__ drwxrwxr-x 2 root root 4096 Jan 20 15:22 benchmark -rw-rw-r-- 1 root root 860 Jan 19 14:34 cog.yaml -rw-rw-r-- 1 root root 4337 Jan 19 13:38 dataset.py drwxrwxr-x 2 root root 4096 Jan 20 15:22 demo drwxrwxr-x 2 root root 4096 Jan 20 15:22 docker -rw-rw-r-- 1 root root 4187 Jan 19 13:38 inference_img.py -rw-rw-r-- 1 root root 11184 Jan 20 14:15 inference_video.py drwxrwxr-x 5 root root 4096 Jan 20 15:22 model -rw-rw-r-- 1 root root 1191 Jan 20 15:23 predict.py -rw-rw-r-- 1 root root 110 Jan 19 13:38 requirements.txt -rw-rw-r-- 1 root root 7217 Jan 19 13:38 train.py drwxr-xr-x 1 root root 4096 Jan 20 15:22 train_log total 4464 -rw-r--r-- 1 root root 40181 Jan 19 21:10 cog-0.0.1.dev-py3-none-any.whl -rw-r--r-- 1 root root 1674656 Jan 20 15:26 interpolated.mp4 -rw-r--r-- 1 root root 459824 Jan 20 15:26 interpolated_2x.mp4 -rw-r--r-- 1 root root 918913 Jan 19 21:12 python-build.20230119211018.206.log -rw------- 1 root root 14560 Jan 19 21:10 python-patch.HnY0l8 -rw-r--r-- 1 root root 129 Jan 19 21:10 requirements.txt -rw------- 1 root root 265969 Jan 19 21:12 tmpa24cmjz_ -rw------- 1 root root 915373 Jan 20 15:26 tmpn1xr0wpiacidtho.mov -rw------- 1 root root 265969 Jan 19 21:12 tmpp3wa944t

Examples

Run time and cost