Stable Diffusion Webgpu

Overview of Stable Diffusion WebGPU

Stable Diffusion WebGPU is a web-based image generation tool that allows users to create images through a user-friendly interface. It operates on the create-react-app framework and is designed to run in modern web browsers, specifically requiring the latest version of Chrome with JavaScript enabled. Users can access the application for free, making it an accessible option for those interested in image generation.

Technical Requirements and Setup

To use Stable Diffusion WebGPU, users must enable specific experimental features in their Chrome browser, including "Experimental WebAssembly" and "Experimental WebAssembly JavaScript Promise Integration (JSPI)". The application relies on a series of inference steps to generate images, with each step taking approximately one minute, plus an additional ten seconds for the VAE decoder to finalize the image. It is important to note that having the DevTools open can significantly slow down the image generation process.

Image Generation Process

The image generation process utilizes the UNET model, which is optimized to run on the CPU for better performance and accuracy. For optimal results, users are encouraged to perform a minimum of 20 inference steps, although for demonstration purposes, as few as 3 steps can be sufficient. The application caches model files to improve efficiency by eliminating the need for repeated downloads.

Current Limitations

Despite its capabilities, the webgpu implementation in Stable Diffusion WebGPU is still in the early stages, which leads to some operational limitations. Data transfer between the CPU and GPU can impact performance, and multi-threading is not currently supported. Additionally, limitations in WebAssembly restrict the creation of 64-bit memory with SharedArrayBuffer, which may affect performance in certain scenarios. The developer is actively working on addressing these issues through proposed changes and patches.

Community and Development

The source code for Stable Diffusion WebGPU is publicly available on GitHub, allowing users to run the application locally if desired. A patched version of onnxruntime is also provided, which enables the use of large language models with transformers.js, although its reliability may vary. The developer plans to submit a pull request to the onnxruntime repository to enhance functionality and address existing limitations.