Webhooks

Webhooks provide real-time updates about your prediction. Specify an endpoint when you create a prediction, and Replicate will send HTTP POST requests to that URL when the prediction is created, updated, and finished.

Here are some example scenarios where webhooks are useful:

• Persisting prediction data and files. Input and output (including any files) are automatically deleted after an hour for any predictions created through the API. Webhooks give you a way to receive all the metadata for a completed prediction, so you can store it in a database or save the output files to persistent storage before they're gone.
• Sending notifications when long-running predictions finish. Some predictions like training jobs can take several minutes to run. You can use a webhook handler to send a notification like an email or a Slack message when a prediction completes.
• Creating model pipelines. You can use webhooks to capture the output of one long-running prediction and pipe it into another model as input.

Note: Webhooks are handy, but they're not strictly necessary to use Replicate, and there are other ways to receive updates. You can also poll the predictions API or use server-sent events (SSEs) to check the status of a prediction over time.

Set up webhooks

To use webhooks, specify a webhook URL in the request body when creating a prediction.

Here's an example using the replicate JavaScript client:

await replicate.predictions.create({
  version: "d55b9f2d...",
  input: { prompt: "call me later maybe" },
  webhook: "https://example.com/replicate-webhook",
  webhook_events_filter: ["completed"], // optional
});

In addition to predictions, you can also receive webhooks when fine-tuning models with the training API:

await replicate.trainings.create({
  version: "d55b9f2d...",
  destination: "my-username/my-model",
  input: { training_data: "..." },
  webhook: "https://example.com/replicate-webhook",
});

• See the Node.js client webhook docs.
• See the Python client webhook docs.
• See predictions.create and trainings.create API docs.
• See Scribble Diffusion's codebase for a reference implementation in JavaScript.
• Read our streaming guide to learn how to consume server-sent events (SSEs) from langauge models.

Receiving webhooks

Replicate can send an HTTP POST request to the URL you specified whenever the prediction is created, has new logs, new output, or is completed.

The request body is a prediction object in JSON format. This object has the same structure as the object returned by the get a prediction API. Here's an example of an unfinished prediction:

{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": null,
  "completed_at": null,
  "status": "starting",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {}
}

The prediction's status property will have one of the following values:

• starting: the prediction is starting up. If this status lasts longer than a few seconds, then it's typically because a new worker is being started to run the prediction.
• processing: the model is currently running.
• succeeded: the prediction completed successfully.
• failed: the prediction encountered an error during processing.
• canceled: the prediction was canceled by the user.

Here's an example of a Next.js webhook handler:

// pages/api/replicate-webhook.js
export default async function handler(req, res) {
  console.log("🪝 incoming webhook!", req.body.id);
  const prediction = req.body;
  await saveToMyDatabase(prediction);
  await sendSlackNotification(prediction);
  res.end();
}

By default, Replicate sends requests to your webhook URL whenever there are new outputs or the prediction has finished. You can change which events trigger a webhook using the webhook_events_filter property.

Your endpoint should respond with a 2xx status code within a few seconds, otherwise the webhook might be retried.

Retries

When Replicate sends the final webhook for a prediction (where the status is succeeded, failed or canceled), we check the response status we get. If we can't make the request at all, or if we get a 4xx or 5xx response, we'll retry the webhook. We retry several times on an exponential backoff. The final retry is sent about 1 minute after the prediction completed.

We do not retry any webhooks for intermediate states.

Idempotency

Make webhook handlers idempotent. Identical webhooks can be sent more than once, so you'll need handle potentially duplicate information.

Redirects

When Replicate sends a webhook request to your service, it does not follow redirects. If your endpoint responds with a redirect, the webhook will fail to send. Be sure to specify a webhook URL that will resolve without redirecting.

Testing webhook handlers

When writing the code for your new webhook handler, it's useful to be able to receive real webhooks in your development environment so you can verify your code is handling them as expected.

ngrok is a free reverse proxy tool that can create a secure tunnel to your local machine so you can receive webhooks. If you have Node.js installed, run ngrok directly from the command line using the npx command that's included with Node.js.

npx ngrok http 3000

The command above will generate output that looks like this:

Session Status                online
Session Expires               1 hour, 59 minutes
Version                       2.3.41
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://3e48-20-171-41-18.ngrok.io -> http://localhost:3000
Forwarding                    https://3e48-20-171-41-18.ngrok.io -> http://localhost:3000

The HTTPS URL in the output (http://3e48-20-171-41-18.ngrok.io in the example above) is a temporary URL pointing to your local machine. Copy that URL and use it as the base of your webhook URL.

Here's an example using the replicate JavaScript client:

await replicate.predictions.create({
  version: "d55b9f2d...",
  input: { prompt: "call me later maybe" },
  webhook: "https://3e48-20-171-41-18.ngrok.io/replicate-webhook",
});

Your webhook handler should now receive webhooks from Replicate. Once you've deployed your app, change the value of the webhook URL to your production webhook handler endpoint when creating predictions.

For a real-world example of webhook handling in Next.js, check out Scribble Diffusion's codebase.

Verifying webhooks

To prevent unauthorized requests, Replicate signs every webhook and its metadata with a unique key for each user or organization. You can use this signature to verify the webhook indeed comes from Replicate before you process it.

Why verify webhooks?

A webhook is an HTTP POST from an unknown source. Attackers can impersonate services by simply sending a fake webhook to an endpoint.

Another potential security hole is a replay attack, wherein an attacker intercepts a valid webhook payload (including the signature) and re-transmits it to your endpoint. This payload will pass signature validation, and will therefore be acted upon. To mitigate replay attacks, Replicate includes a timestamp indicating when the webhook attempt occurred.

Client library support

Replicate's official JavaScript client has built-in support for verifying webhooks:

import { validateWebhook } from "replicate";
 
const isValid = validateWebhook(request, secret);

Replicate's official Go client also has built-in support for verifying webhooks:

import (
	"github.com/replicate/replicate-go"
)
 
isValid, err := replicate.ValidateWebhookRequest(req, secret)

Manually validating webhook data

Each webhook delivery includes three HTTP headers with additional information that you can use to verify the authenticity of the request:

• webhook-id: The unique message identifier for the webhook messages. This identifier is unique across all messages but will be the same when a webhook is being resent (e.g. retried).
• webhook-timestamp: timestamp in seconds since epoch.
• webhook-signature: the Base64 encoded list of signatures (space delimited).

Constructing the signed content

As a webhook receiver, you are responsible for constructing this signed content and performing the validation steps. To validate a webhook, the signed data must be constructed into a well-defined structure from the payload data (body), webhook-id, and webhook-timestamp headers.

The content to sign is composed by concatenating the id, timestamp, and data, separated by the full-stop character (.).

In code it will look something like:

const signedContent = `${webhook_id}.${webhook_timestamp}.${body}`

In the example above, body is the raw body of the request. The signature is sensitive to any changes, so even a small change in the body will cause the signature to be completely different. This means that you should not change the body in any way before verifying.

Retrieving the webhook signing key

Replicate provides an API endpoint you can use to retrieve the signing key. The signing key is unique to your user or organization. The endpoint will return only the signing key associated with the API token and its corresponding user or organization.

For optimal performance of the webhook receiver, it is advised to locally cache the signing key. By doing so, you eliminate the need for the receiver to make a request to the Replicate API for validation every time a webhook is received.

GET https://api.replicate.com/v1/webhooks/default/secret

Example cURL request:

curl -s -X GET -H "Authorization: Bearer <paste-your-token-here>" \
https://api.replicate.com/v1/webhooks/default/secret

The response will be a json object with a single key field:

{
    "key": "whsec_C2FVsBQIhrscChlQIMV+b5sSYspob7oD"
}

Determining the expected signature

Replicate uses an HMAC with SHA-256 to sign its webhooks.

To calculate the expected signature, you should HMAC the signed_content from above using the base64 portion of the signing secret (this is the part after the whsec_ prefix) as the key. For example given a secret whsec_C2FVsBQIhrscChlQIMV+b5sSYspob7oD you will want to use C2FVsBQIhrscChlQIMV+b5sSYspob7oD.

For example, this is how you can calculate the signature in Node.js:

const crypto = require('crypto');
 
const signedContent = `${webhook_id}.${webhook_timestamp}.${body}`
const secret = "whsec_C2FVsBQIhrscChlQIMV+b5sSYspob7oD";
 
// Base64 decode the secret
const secretBytes = new Buffer(secret.split('_')[1], "base64");
const signature = crypto
  .createHmac('sha256', secretBytes)
  .update(signedContent)
  .digest('base64');
console.log(signature);

This generated signature should match one of the ones sent in the webhook-signature header.

The webhook-signature header is composed of a list of space-delimited signatures and their corresponding version identifiers. The signature list is most commonly of length one, though there could be any number of signatures. For example:

v1,g0hM9SsE+OTPJTGt/tmIKtSyZlE3uFJELVlNIOLJ1OE= v1,bm9ldHUjKzFob2VudXRob2VodWUzMjRvdWVvdW9ldQo= v2,MzJsNDk4MzI0K2VvdSMjMTEjQEBAQDEyMzMzMzEyMwo=

Make sure to remove the version prefix and delimiter (e.g. v1,) before verifying the signature.

Please note that to compute the signatures it's recommended to use a constant-time string comparison method in order to prevent timing attacks.

An example of how to do this in Node.js:

const expectedSignatures = webhookSignatures.split(' ').map(sig => sig.split(',')[1]);
const isValid = expectedSignatures.some(expectedSignature => expectedSignature === computedSignature);
console.log(isValid);

Verify timestamp

As mentioned above, Replicate also sends the timestamp of the attempt in the webhook-timestamp header. You should compare the timestamp against your system timestamp and make sure it's within your tolerance in order to prevent replay attacks.