Build a robot artist for your Discord server with Stable Diffusion, Replicate, and Fly.io

Soon after VQGAN-CLIP was released last year, somebody made a bot in #the-faraday-cage on Eleuther’s Discord that ran the model. You asked it to !imagine something, and then VQGAN-CLIP responded with its weird interpretation of your request.

It was thrilling. This was the first playground for text to image models, the first place where people could prompt computers to produce art. And, it was hilarious! Instead of the model being locked up in a Colab, it was a hundred people in a room playing with it together. People were riffing off each other’s ideas, patting each other on the back, laughing at each other’s outputs. This is where the “unreal engine”, “artstation”, and other prompt hacks appeared. Somebody saw somebody else do something neat, and then the prompt meme reproduced.

We've come a long way since four-eared faceless rabbits. Earlier this week, we were blessed with the weights from The Great A100 Cluster. So, let's put them to good use.

a screenshot of the stable-diffusion-bot generating a cute rabbit in Discord

This guide will show you how to create your own Discord bot with Stable Diffusion, so you can have a much better robot artist in your own Discord server.

We’ll use Replicate to run the model so you don’t need to have a GPU, and we’ll deploy it to Fly.io so you can keep it running all the time, even when your computer’s off.

Prerequisites

Install Python 3.5 or later.
A command-line shell (e.g. Terminal on Mac, gnome-terminal on Linux, or PowerShell on Windows)
Sign up for an account on Replicate.
Sign up for an account on Discord.
Have a read of the model license so you know what you can use the model for.

Set up a Discord Bot account

Follow discord.py's guide to set up a Discord Bot account. Give it a name like stable-diffusion-bot. At the end of the first section, you'll need to turn on the "message content intent" option on the bot page. That permission is needed for your bot to read the content of messages, to get the prompts!

In the second half of the guide ("Inviting your bot") when it tells you to "tick the permissions required for your bot to function", you'll want to give the bot permission to "Send messages".

Write some code

This section walks you through steps required to set up a codebase.

🐇 Want to skip ahead? If you'd rather not copy and paste all the code below, then you can clone the replicate/discord-bot-tutorial-code GitHub repo to start with a working project template.

Create a project directory

mkdir stable-diffusion-bot
cd stable-diffusion-bot

Define Python dependencies

Next you'll define some of the Python package dependencies needed by your project.

Create a new file called requirements.txt and paste in the following content:

discord.py==2.0.1
packaging==21.3
python-dotenv==0.20.0
replicate==0.0.1a15

Then install the dependencies:

pip install -r requirements.txt

Configure environment and secrets

Create a file named .env. This text file will be used to store secrets for your development environment. Paste in the following:

REPLICATE_API_TOKEN=<your-token>
DISCORD_TOKEN=<your-token>

Visit replicate.com/account to copy and paste your API token.

The Discord token is the token you copied when creating your bot from the Discord guide. If you don't have it, you can generate a new one by visiting discord.com/applications, selecting your bot application, selecting "Bot" from the side menu, and clicking "Reset Token".

✋ The .env file contains secrets, so it should not be shared with anyone. If you're planning to turn your project into a Git repository, be sure to create a .gitignore file and add .env to it.

Write the bot

Create a new file called bot.py and paste the following code into it:

from discord import Intents
from discord.ext import commands
from dotenv import load_dotenv
import os
import replicate
 
load_dotenv()
 
intents = Intents.default()
intents.message_content = True
 
bot = commands.Bot(
    command_prefix="!",
    description="Runs models on Replicate!",
    intents=intents,
)
 
 
@bot.command()
async def dream(ctx, *, prompt):
    """Generate an image from a text prompt using the stable-diffusion model"""
    msg = await ctx.send(f"“{prompt}”\n> Generating...")
 
    model = replicate.models.get("stability-ai/stable-diffusion")
    version = model.versions.get("27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478")
    image = version.predict(prompt=prompt)[0]
 
    await msg.edit(content=f"“{prompt}”\n{image}")
 
 
bot.run(os.environ["DISCORD_TOKEN"])

This file defines the command for your bot (!dream) and how it works. When you want to make changes to your bot later, this will be the file you come back to.

Run your bot locally

Now that you've written the code for your bot, it's time to run it!

Run this command to start your bot locally:

python bot.py

Use the bot

Check your bot is running by typing !help into one of the channels on your Discord server. Your bot should respond with the list of commands it can run, including !dream.

Now try generating an image:

!dream a rabbit in a meadow, trending on artstation

Your bot should write a message saying "Generating...", and then 20–40 seconds later it should swap out that message for the newly generated image. 🥳

Deploy your bot with Fly.io

Running your bot locally is convenient, especially when you're actively working on it. The downside, however, is that it can only run when you're online. If you're building a bot for a server, you probably want it to be online even when you aren't.

There are lots of ways to deploy an application like this. Some of the easiest are services like Vercel, Fly.io or Heroku. In this tutorial, you'll use Fly to deploy the bot to the cloud.

To get started, see Fly's "speedrun" guide to install the flyctl command-line tool and create a Fly account.

Then create a new file called Dockerfile and paste the following code into it:

FROM python:3.10
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code
CMD python bot.py

Then create a new Fly application:

flyctl launch

That command will generate a new file called fly.toml, but it's designed for running web apps so you'll need to make a few changes. Remove the [[services]] block and everything below it. Your modified file should look something like this:

# fly.toml file generated for your-fly-app on 2022-08-25T17:37:31+01:00
app = "your-fly-app"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []
 
[env]
 
[experimental]
  allowed_public_ports = []
  auto_rollback = true

Then configure your Fly app using the secrets from your local .env file:

flyctl secrets set REPLICATE_API_TOKEN=... DISCORD_TOKEN=...

That's it! Your bot is now running in the cloud.

One thing you need to do if you're running the bot in a server is to make people who use it aware of the model's license. You might want to add it to the server rules, or to the channel topic, or something like that.

Make it your own

Now might be a good time to tinker with the bot a bit. Some ideas:

Generate multiple images instead of a single one, and show them all.
Add another command that runs a different model.
Take the image output and run it through another model, like an upscaler.
Turn it into a game. Perhaps a game of telephone, or a game of Pictionary.

Get creative! Share what you've built in our Discord and we'll feature some of our favorite things. 🪄