Advertisement

You're blocking ads, which pay for BlenderNation. Read about other ways to support us.

Blender and Stable Diffusion

6

It's no secret that generative AIs are taking the world by storm, generating both wonder for their technical prowess and disdain for ethical reasons. Stable Diffusion marked a milestone in the rise of AI imagery, being the first open-source state-of-the-art model released, in a relatively compact and performant package able to run on consumer-grade GPUs (whereas the other famous alternatives such as midjourney are closed-source and only available to compute on massive cloud arrays).

And where there is open-source, there is Blender! The Blender community has wasted no time in testing Stable Diffusion as a texture generator, and implemeting it to Blender in various ways. Here are some examples that stood out to me:

CEB Stable Diffusion

Carlos Barreto is an add-on developer known for his great tools which often incorporate an AI component, including for motion capture right in Blender. His rapidly developing CEB Stable Diffusion add-on already has both txt2img (image generation via prompt) and img2img (image-based diffusion) implementations and is just generally a great way to have a complete build of Stable Diffusion on your machine.

Stable Diffusion as a Live Renderer Within Blender

Over on the Blender subreddit, Gorm Labenz shared a video of an add-on he wrote that enables the use of Stable Diffusion as a live renderer, basically reacting to the Blender viewport in realtime and generating an image (img2img) based on it and some prompts that define the style of the result. The whole thing is just incredible to witness and opens up whole new workflows of AI-Assisted look development.

I wrote a plugin that lets you use Stable Diffusion (AI) as a live renderer from blender

AI Suzanne

Still, on the subreddit, user "Renaissance_blender" shares a Stable Diffusion enhanced Suzanne, once again using AI as a renderer to the "basic" image prompt from Blender.

Blender render of Suzanne + Stable Diffusion from blender

Texture Generation

And finally, here is an example of one of the most basic yet useful uses for Stable Diffusion: Texture generation, with infinite variations available just a few words away!

AI text-to-image generators are pretty good at generating images! Here's one I did with Stable Diffusion from blender

About the Author

Mario Hawat

Mario Hawat is a Lebanese 3D artist, writer, and musician currently based in Paris. He is a generalist with a special focus on environments, procedural and generative artworks. Open to freelance work.

6 Comments

  1. Hola, es un trabajo increíble y de suma importancia. Llevo algún tiempo pensando en cuán increíble será cuando podamos aplicar estás tecnologías de IA a nuestros flujos 3D. Cuántos beneficios, cuan ilimitado y eficiente resultaría el proceso creativo de los artistas 3D y gráficos en sentido general. Gracias por eso y felicidades!! Ver qué existen personas con ese nivel de compromiso es lo que hace al resto soñar

Leave A Reply

To add a profile picture to your message, register your email address with Gravatar.com. To protect your email address, create an account on BlenderNation and log in when posting a message.

Advertisement