Introducing Plethora of Stable Diffusion models: Part 1

In the past year or two, we’ve seen funny memes predicting how our favorite celebrities might look in the future, with wrinkles and white facial hair. This playful trend showcases the rise of technology like generative AI in creating customized images.

Now, what is generative AI?

Generative AI refers to artificial intelligence systems that can create new content, such as text, images, or other media, based on patterns and information learned from existing data. When a piece of code or AI system consistently delivers refined, current, and sought-after outcomes, it can be called as generative AI model. Some of the well known generative AI models are Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Autoencoders, Boltzmann Machines, Stable Diffusion. In this article we are briefly talking about stable diffusion model.

Below, We are producing images from this UI which is taking prompt/command to generate customized image/picture with completely offline setup. This particular generative AI model is called a stable diffusion model. You can have a copyright-free image ready with your preferred style and dimensions within minutes.

Architecture of stable diffusion

The architecture of Stable Diffusion involves a latent diffusion model that generates images from text prompts. While the detailed architecture might vary, here is a general overview:

Latent Diffusion Model: Stable Diffusion operates as a latent diffusion model, working in a lower-dimensional latent space rather than directly in the high-dimensional pixel space of images.
Diffusion Process: The core of Stable Diffusion is a diffusion process. Unlike traditional generative models that apply diffusion directly to pixel images, Stable Diffusion performs diffusion in the latent space, enhancing efficiency.
Text-to-Latent Conversion: To generate images from textual prompts, Stable Diffusion converts input text into a latent image representation. This involves encoding textual information into a format compatible with the latent space.
Noise Prediction with U-Net: In many implementations, Stable Diffusion integrates a U-Net model for noise prediction. The U-Net takes encoded text and a noisy array of numbers as inputs, predicting noise patterns that enhance the latent image.
Integration of Noise: The predicted noise from the U-Net is integrated into the diffusion process. This step refines the latent image representation by incorporating the predicted noise, contributing to the generation of detailed and coherent images.
Image Generation: The final step involves generating high-quality images based on the refined latent image representation and the integrated noise. The generated images align with the provided textual descriptions or prompts.

It’s important to note that Stable Diffusion is a concept with variations in implementation. Different models may utilize unique architectures, and the integration of components like U-Net for noise prediction can vary. Understanding the specific architecture often requires referring to the details provided in the respective research papers, documentation, or articles related to a particular implementation of Stable Diffusion.

Stable Diffusion Models

These are some sites that host thousands of diffusion models easily downloadable for content(text, image, video) generation free of charge. Some sites also provide a way to use them online under a price range or model to model basis.

Installation and Use

In following steps, we will show you how to download and install stability-diffusion UI these pre-trained model are free to run as long as you have a powerful system to run them at full capacity.

Download & Install Stablity Diffusion UI

You need to consider installing the WSL if you don’t have a Linux system. Use this article to get your system prepared for stable diffusion click here. Use the following commands to install the web-UI.

Python

(base) ubuntu@WorkBox:~/newoness$ git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
(base) ubuntu@WorkBox:~/newoness$ cd stable-diffusion-webui/
(base) ubuntu@WorkBox:~/newoness$ explorer.exe .

Following folder is where you will save the models you download. Make sure they have .safetensors file extension.

Download Model

civitai is one of the sites that provides you with pretrained models. Do explore the site there is a lot to generation models than just images. You can generate texts, voices, videos. And you can either use images or text prompts to generate.

Visit following links to find website that host stable diffusion models.

Distinctive Analytics