ttsds / amphion_valle

The VALL-E models by Amphion. (Updated 4 months, 2 weeks ago)

  • Public
  • 616 runs
  • L40S
  • GitHub
  • Weights
  • Paper
  • License
Iterate in playground

Input

Video Player is loading.
Current Time 00:00:000
Duration 00:00:000
Loaded: 0%
Stream Type LIVE
Remaining Time 00:00:000
 
1x
*string

An enumeration.

*string
Shift + Return to add a new line
*file
*string
Shift + Return to add a new line

Output

Video Player is loading.
Current Time 00:00:000
Duration 00:00:000
Loaded: 0%
Stream Type LIVE
Remaining Time 00:00:000
 
1x
Generated in

This output was created using a different version of the model, ttsds/amphion_valle:b533d19a.

Run time and cost

This model costs approximately $0.0062 to run on Replicate, or 161 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 7 seconds.

Readme

VALLE-v1 and VALLE-v2 by Amphion

Three models of the VALL-E family as replicated by the authors of the Amphion project

Citation

@article{wang2023neural,
  title={Neural codec language models are zero-shot text to speech synthesizers},
  author={Wang, Chengyi and Chen, Sanyuan and Wu, Yu and Zhang, Ziqiang and Zhou, Long and Liu, Shujie and Chen, Zhuo and Liu, Yanqing and Wang, Huaming and Li, Jinyu and others},
  journal={arXiv preprint arXiv:2301.02111},
  year={2023}
}

@inproceedings{amphion,
    author={Xueyao Zhang and Liumeng Xue and Yicheng Gu and Yuancheng Wang and Jiaqi Li and Haorui He and Chaoren Wang and Ting Song and Xi Chen and Zihao Fang and Haopeng Chen and Junan Zhang and Tze Ying Tang and Lexiao Zou and Mingxuan Wang and Jun Han and Kai Chen and Haizhou Li and Zhizheng Wu},
    title={Amphion: An Open-Source Audio, Music and Speech Generation Toolkit},
    booktitle={{IEEE} Spoken Language Technology Workshop, {SLT} 2024},
    year={2024}
}

Disclaimer

This is an unofficial implementation. Please refer to the Amphion project for the original code and details. By using this model, you agree to the terms stated at the link above, which could change at any time. Make sure to comply to these terms before using the model.