zsxkib/realistic-voice-cloning:bbdb9b99 – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

zsxkib /realistic-voice-cloning:bbdb9b99

Playground API

Input

song_input

file

Upload your audio file here.

rvc_model

string

Shift + Return to add a new line

RVC model for a specific voice. If using a custom model, this should match the 'custom_rvc_model_download_name'.

Default: "Squidward"

custom_rvc_model_download_url

string

Shift + Return to add a new line

URL to download a custom RVC model. To use the downloaded model, 'rvc_model' should be set to the same value as 'custom_rvc_model_download_name'.

custom_rvc_model_download_name

string

Shift + Return to add a new line

The name of the custom RVC model. This should match the 'rvc_model' if you want to use the downloaded model.

pitch_change

number

Change pitch of AI vocals in octaves. Set to 0 for no change. Generally, use 1 for male to female conversions and -1 for vice-versa.

Default: 0

keep_files

boolean

Can be added to keep all intermediate audio files generated. e.g. Isolated AI vocals/instrumentals. Leave out to save space.

Default: false

index_rate

number

Control how much of the AI's accent to leave in the vocals. 0 <= INDEX_RATE <= 1.

Default: 0.5

filter_radius

integer

If >=3: apply median filtering median filtering to the harvested pitch results. 0 <= FILTER_RADIUS <= 7.

Default: 3

rms_mix_rate

number

Control how much to use the original vocal's loudness (0) or a fixed loudness (1). 0 <= RMS_MIX_RATE <= 1.

Default: 0.25

pitch_detection_algorithm

string

Best option is rmvpe (clarity in vocals), then mangio-crepe (smoother vocals).

Default: "rmvpe"

crepe_hop_length

integer

When `pitch_detection_algo` is set to `mangio-crepe`, this controls how often it checks for pitch changes in milliseconds. Lower values lead to longer conversions and higher risk of voice cracks, but better pitch accuracy.

Default: 128

protect

number

Control how much of the original vocals' breath and voiceless consonants to leave in the AI vocals. Set 0.5 to disable. 0 <= PROTECT <= 0.5.

Default: 0.33

main_vocals_volume_change

number

Control volume of main AI vocals. Use -3 to decrease the volume by 3 decibels, or 3 to increase the volume by 3 decibels.

Default: 0

backup_vocals_volume_change

number

Control volume of backup AI vocals.

Default: 0

instrumental_volume_change

number

Control volume of the background music/instrumentals.

Default: 0

pitch_change_all

number

Change pitch/key of background music, backup vocals and AI vocals in semitones. Reduces sound quality slightly.

Default: 0

reverb_size

number

(minimum: 0, maximum: 1)

The larger the room, the longer the reverb time. 0 <= REVERB_SIZE <= 1.

Default: 0.15

reverb_wetness

number

(minimum: 0, maximum: 1)

Level of AI vocals with reverb. 0 <= REVERB_WETNESS <= 1.

Default: 0.2

reverb_dryness

number

(minimum: 0, maximum: 1)

Level of AI vocals without reverb. 0 <= REVERB_DRYNESS <= 1.

Default: 0.8

reverb_damping

number

(minimum: 0, maximum: 1)

Absorption of high frequencies in the reverb. 0 <= REVERB_DAMPING <= 1.

Default: 0.7

output_format

string

wav for best quality and large file size, mp3 for decent quality and small file size.

Default: "mp3"

Run this model in Node.js with one line of code:

npx create-replicate --model=zsxkib/realistic-voice-cloning

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run zsxkib/realistic-voice-cloning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "zsxkib/realistic-voice-cloning:bbdb9b991f627b94d5fa92f0a3eb4037f224bb79cf56ff46b70afe84ce7bb646",
  {
    input: {
      protect: 0.33,
      rvc_model: "Squidward",
      index_rate: 0.5,
      keep_files: false,
      reverb_size: 0.15,
      pitch_change: 0,
      rms_mix_rate: 0.25,
      filter_radius: 3,
      output_format: "mp3",
      reverb_damping: 0.7,
      reverb_dryness: 0.8,
      reverb_wetness: 0.2,
      crepe_hop_length: 128,
      pitch_change_all: 0,
      main_vocals_volume_change: 0,
      pitch_detection_algorithm: "rmvpe",
      instrumental_volume_change: 0,
      backup_vocals_volume_change: 0
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run zsxkib/realistic-voice-cloning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "zsxkib/realistic-voice-cloning:bbdb9b991f627b94d5fa92f0a3eb4037f224bb79cf56ff46b70afe84ce7bb646",
    input={
        "protect": 0.33,
        "rvc_model": "Squidward",
        "index_rate": 0.5,
        "keep_files": False,
        "reverb_size": 0.15,
        "pitch_change": 0,
        "rms_mix_rate": 0.25,
        "filter_radius": 3,
        "output_format": "mp3",
        "reverb_damping": 0.7,
        "reverb_dryness": 0.8,
        "reverb_wetness": 0.2,
        "crepe_hop_length": 128,
        "pitch_change_all": 0,
        "main_vocals_volume_change": 0,
        "pitch_detection_algorithm": "rmvpe",
        "instrumental_volume_change": 0,
        "backup_vocals_volume_change": 0
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run zsxkib/realistic-voice-cloning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "bbdb9b991f627b94d5fa92f0a3eb4037f224bb79cf56ff46b70afe84ce7bb646",
    "input": {
      "protect": 0.33,
      "rvc_model": "Squidward",
      "index_rate": 0.5,
      "keep_files": false,
      "reverb_size": 0.15,
      "pitch_change": 0,
      "rms_mix_rate": 0.25,
      "filter_radius": 3,
      "output_format": "mp3",
      "reverb_damping": 0.7,
      "reverb_dryness": 0.8,
      "reverb_wetness": 0.2,
      "crepe_hop_length": 128,
      "pitch_change_all": 0,
      "main_vocals_volume_change": 0,
      "pitch_detection_algorithm": "rmvpe",
      "instrumental_volume_change": 0,
      "backup_vocals_volume_change": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No output yet! Press "Submit" to start a prediction.