Creating your own Virtual Avatar with Stable Diffusion
Day 8 / 366
After learning about Stable Diffusion yesterday, the next goal I had in mind was to use it to train a custom model using my images, so that I can create AI-generated images with me in them.
Now I know there are tools out there for creating your own AI images, and I could easily use any one of them to do it. But that’s not what I am after. I want to understand how those tools work so that I can create tools like that of my own.
So I spent the day today looking into this.
Dreambooth
After some research, I found that there are training techniques through which you can create models of your own by using a Stable Diffusion model as the base. The most widely used technique for this is Dreambooth.
The process involves getting several images of the person you want to create the model of. The images should be of good quality and in a variety of poses and angles.
Then you need to come up with a unique word to give that person. This allows the model to associate that name with that word. This must be random and meaningless so that it does not interfere with any of the existing concepts that the model knows.
Apart from this, there are several parameters that you can adjust to improve the quality of your model, but I was not concerned with those. My first goal was to just get a working model. So I needed to figure out a way to run Dreambooth
GPU Resources
The only barrier to entry to run Stable Diffusion on your own is getting your hands on some sweet sweet GPUs. I have a really old 1080 GPU in my PC. With 4GBs of VRAM, it's embarrassingly ill-equipped to handle this training.
While you can generate SD images comfortably with a 12GB graphic card, to train your model you require at least 24 GB of VRAM.
So I started looking at cloud solutions available.
RunDiffusion
I used this yesterday to try out SD for the first time. And playing around with it I was able to find the Dreambooth option in it as well. The pricing was very reasonable. Even if it took me 2 hours to do this, it would cost me just 1 dollar.
I tried for an hour but failed miserably. I kept running into error messages with no documentation. And because of the name — ‘RunDiffusion’ it was really hard to get help from Google as well. At last, I gave up and looked for something else.
Google Collab
Google Collab is a way to run Jupyter Notebooks on Google Cloud. These have access to GPUs as well, so I found lots of resources online about how to use Google Collab to run Dreambooth.
After an hour of debugging, I realized that I had to purchase Google Compute credits to access the good GPUs. The free ones were not strong enough to handle Dreambooth training. It was 10 dollars, and I thought that was reasonable for now.
But luck was just not in my favor today, and none of my cards worked in their payment gateway. I was close to giving up this time.
Huggingface
I looked online for more ways to rent GPUs and found out that HuggingFace offered this as well. I always wanted to try out huggingface so I chose it as my last try.
I had to pay for a GPU here as well, but at last, I was able to get my model made!
Now before I show you the result please keep in mind that I made this with no changes to the default parameters and I used low-quality images that I already have. With that, I made my first model based on me named — Aligixy.
Once I had the model, I could ask it to create images of me (or aligixy to be precise) in various scenarios -
The above is an AI-generated image, from a model that I created using 10 of my old selfies.
There is still a long way to go, and lots of scope for improvement here. But I am glad I didn’t give up today and kept trying. As they say, the beginning is half done. I can’t wait to learn more about SD and get better results