camenduru / emage

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

  • Public
  • 39 runs
  • GitHub
  • Paper
  • License

Input

Output

Run time and cost

This model runs on Nvidia A40 GPU hardware.

Readme

🐣 Please follow me for new updates https://twitter.com/camenduru
🔥 Please join our discord server https://discord.gg/k5BwmmvJJU
🥳 Please join my patreon community https://patreon.com/camenduru

🕸 Replicate

https://replicate.com/camenduru/minigpt4-video

📋 Tutorial

  • Register at https://smpl-x.is.tue.mpg.de and download the SMPL-X for Blender add-on. The ZIP release (https://drive.google.com/drive/folders/1ukbifhHc85qWTzspEgvAxCXwn9mK4ifr) file (smplx_blender_addon_20230921.zip) will include the required SMPL-X+FLAME model which is not included in the code repository.
  • Blender>Edit>Preferences>Add-ons>Install
  • Select downloaded SMPL-X for Blender add-on ZIP file (smplx_blender_addon-YYYYMMDD.zip) and install
  • Enable SMPL-X for Blender add-on
  • Enable sidebar in 3D Viewport>View>Sidebar
  • SMPL-X tool will show up in sidebar

🧬 Code

https://github.com/PantoMatrix/PantoMatrix/tree/main/scripts/EMAGE_2024

📄 Paper

https://arxiv.org/abs/2401.00374

🌐 Page

https://pantomatrix.github.io/EMAGE/

https://replicate.com