VoltaML - Fast Stable Diffusion

Stable Diffusion WebUI and API accelerated by TensorRT

Documentation · Report Bug · Request Feature

Table of Contents

• About the Project
  • Screenshots
  • Tech Stack
  • Features
  • Speed comparison
  • Environment Variables
• Getting Started
  • Docker setup
  • Running locally
    • API + Compiled frontend
    • Frontend in development mode
• Contributing
  • Code of Conduct
• License
• Contact

About the Project

Screenshots

Tech Stack

Client

• Typescript
• Vue.js
• NaiveUI
• Ionicons

API

• Python
• FastAPI
• PyTorch
• TensorRT
• xFormers
• WebSockets

Discord Bot

• Discord.py

DevOps

• Docker
• GitHub Actions
• GitHub Pages
• VitePress

Features

• Easy install with Docker
• Clean and simple Web UI
• Supports PyTorch as well as TensorRT for fastest inference
• Support for Windows and Linux (TRT is not officialy supported on Windows if running locally)
• xFormers support
• GPU cluster support with load balancing
• Discord bot

Speed comparison

The below benchmarks have been done for generating a 512x512 image, batch size 1 for 50 iterations.

Model	T4 (it/s)	A10 (it/s)	A100 (it/s)	4090 (it/s)	3090 (it/s)	2080Ti (it/s)
PyTorch	4.3	8.8	15.1	19	11	8
Flash attention xformers	5.5	15.6	27.5	28	15.7	N/A
AITemplate	Not supported	26.7	55	60	N/A	Not supported
VoltaML(TRT-Flash)	11.4	29.2	62.8	85	44.7	26.2

Environment Variables

You need to export following environment variables to run this project locally:

• HUGGINGFACE_TOKEN
• DISCORD_BOT_TOKEN (only if you want to use the discord bot)

When running with docker, you need to set these environment variables in the docker-compose.yml file

Getting Started

Docker setup

Download the docker-compose.yml file (no need to clone the repository):

curl -O https://raw.githubusercontent.com/VoltaML/voltaML-fast-stable-diffusion/master/docker-compose.yml

Please open the file and edit the environment variables.

Now you can start the container with:

docker-compose up -d

-d specifies that the container should run in the background. You can omit this flag if you want to see the logs and have the container stop when you press CTRL+C.

Linux users may need to run this command with sudo.

Running locally

Clone the project

git clone https://github.com/VoltaML/voltaML-fast-stable-diffusion.git

Go to the project directory

cd voltaML-fast-stable-diffusion

API + Compiled frontend

Start up the API, you might be prompted to activate the virtual environment (how will be explained in the message)

python main.py

Frontend in development mode

Change directory to frontend

cd frontend

Install dependencies

yarn

Start the development server

yarn dev

Contributing

Contributions are always welcome!

See contributing.md for ways to get started.

Code of Conduct

Please read the Code of Conduct

License

Distributed under the GPL v3. See LICENSE for more information.

Contact

Feel free to contact us on discord: https://discord.gg/pY5SVyHmWm

Project Link: https://github.com/VoltaML/voltaML-fast-stable-diffusion