A Complete Guide on How to Use LoRAs
So you have now got the hang of using stable diffusion, you know what models are and how to use them . You have even created some pretty decent AI images, but you are missing one thing. You have no idea what a Lora is and how you should be using one to generate your images with Stable Diffusion.
Well let us help you. LoRA’s are smaller files (around 10-250MB). You use these files in conjunction with other Stable Diffusion model files (check point models files work best). These files help to create character/subject images.
Here is an example. If you wanted images of a hamster 🐹 but couldn’t quite generate them well with a model file. You would use a LoRA that has been trained on hamster images, this could be trained on 1000’s of images to ensure a life like appearance. Buy using this custom made file you are trying to ensures that you get an image that closely resembles a hamster.
These type of model files are hugely popular, one of the main reasons is because they can mimic other peoples art styles, or characters. Some people will use LoRAs to create realistic characters you can train a LoRA on any set of images.
Stable diffusion uses LoRA’s like an artistic aid in a way. It’s not just subjects but you can also get LoRA model files that are trained on art styles too. The AI model community is huge, chances are someone somewhere has already created a custom made model file just waiting for you to download. All you need to do is to know where to look for it.
In this article we are going to show you how to use LoRAs with stable diffusion, we will help you create the images you want to create with these custom made files.
LoRAs Are Compatible With All Stable Diffusion Models
Yes you heard us right, you can literally use a LoRA with any stable diffusion model. Of course some of them are specifically created to be used with a certain model, but that doesn’t stop you playing around a little.
When downloading these LoRA files be sure to check out the creators description. Sometimes it does have some key information in regards of how to use the file, what models it should be used with and if any licence agreements come with the download of the file.
Now if you don’t know where to begin or are worried that your main model might throw up some strange images or cause some kind of conflict. You can always use the AnyLoRA file. This is a model that has been created to be as neutral as possible so it should work well with any LoRA.
If you want to get the best results using LoRAs you might want to consider entering specific prompts related to the LoRA. So if your file was about Anime Mange Art you might want to use “Anime, Manga, Art” in your prompt. Again the creator may make you aware of this.
How to Use LoRAs With Stable Diffusion
So now you know what LoRAs are, we are now going to show you how to use them. First off you need to ensure you have installed stable diffusion locally either on windows or mac.
Make sure you are running the AUTOMATIC1111 web UI version, this is the version we are going to be using throughout this guide. Please note this version of stable diffusion is free to use.
Download the Necessary Files
For this guide you will need to download two files. You will need a checkpoint model file, these are the large model files a few Gigabytes in size and you will need a LoRA file. We will be using the following files:
Model Checkpoint: DreamShaper Our base model. Great for creating artistic images.
LoRA File: C4ttitude a Lora file trained to make cats in clothing.
Yes, we know it’s a strange combo but fun, well feel it shows the creativity of stable diffusion. You are more than welcome to download your own files from https://civitai.com/ – a huge database of models can be found here. But if you want to play along, you should download the two files above.
Choose your two files? we let us begin.
Time to Install Our Files
When installing files for the web ui version of stable diffusion, you can install them while it’s running. But we prefer to do it while it’s switched off to stop any conflicts or issues.
First off we are going to head over to our stable diffusion folder: “Stable-diffusion-webui\models\Stable-diffusion” we will place our Model file here so grab your DreamShaper model or your own custom model file and place it in this folder. “stable-diffusion-webui\models\Lora“. You can place multiple LoRA’s and Model files in these folders.
All done? files placed in the correct folders? now let’s start up the WebUI. You might be familiar with some of the parameters stable diffusion uses but you might have neglected or not noticed some of it’s features.
If you notice under the generate button on the far right there is a few buttons, you may or may not have used these before. The button we want to click on is the button shown in the image below. This is the “Show Extra Networks” button.
Once you click on the above button you will see a new menu on this new menu click the LoRAs tab. This will show a list of all the LoRA files you have place in the “stable-diffusion-webui/models/Lora” folder. If you can’t see them just hit the refresh button or restart the web UI.
If you want to use any one of your LoRA files just click on it and it will show up in your prompt. You can actually use as many LoRAs as you like in the same prompt. One of the main aspects of using stable diffusion is trial and error. This is why we created an article on how you can create your own characters.
If you want to enter the LoRA keyword manually you can this manually too. You will need to enter it in the following format:
<lora:LORA-FILENAME:WEIGHT>
LORA-FILENAME is the filename of the LoRA model you are wanting to use. Do not put the file extension name here. You need to ensure the file name is correct or else it won’t work. If you have copied the prompt from someone else ensure both filenames match or again it wont work. This is why we recommend you do the above steps to insert it into your prompt.
WEIGHT This is the strength of the LoRA file. 1 being the default and the maximum number, use 0 to switch it off. A lot of people tend to use around 0.5 – 0.8 for best results. Using 1 can make things look a little messy at times. But it really is dependant on which LoRA file you are using so test it out a little, see what works for you.
Here is our example:
<lora:c4ttitudeflexver:0.8>
It’s best to check the notes of the creator of any models or LoRAs you download. It can sometime contain vital information on how use the files and what the terms of use are in regards to the files.
Here is our prompt for our generation, we kept it simple and easy to understand. We didn’t go full on prompt engineer mode for this one. Just something basic.
Parameters:
- Sampler: Euler A
- CFG Scale: 7
- Size: 512 x 768
- Steps: 50
- Seed: Random
Prompt:
<lora:c4ttitudeflexver:0.8> c4ttitude, wearing warrior armor, side portrait Negative Prompt:
easynegativePlease note we used the easynegative file to help generate our images. It’s good when you are creating drawings. To install this you need to place it in the “\stable-diffusion-webui\embeddings” folder.
Using the above prompt we got the following images. We set our batch size to 6. Here is the result of our generated image.
Popular LoRAs and where to find them
If you are looking for places to download LoRAs there are two places we would recommend visiting:
- Civitai most popular source of stable diffusion files. Seriously the list is extensive but please be warned some of it is NSFW!
- HuggingFace – Another source for LoRAs. Have a look around see if you find anything you like the look of.
If NSFW models are your think then Civitai is the number one place for it, you will need to create an account in order to see it, we are not here to judge.
Style/Aesthetic LoRAs
Anime Lineart
This LoRA helps you create lineart. Think of manga/anime. You can use it for 3D models too but it’s best job is creating 2D black and white art. Anime LineArt Download.
He-Man & Masters Of The Universe Style LoRA
This LoRA is an example that you can copy someone elses artwork. This is why Japan have brought new rules in regards to using AI tools. Can you see why some people might be annoyed by the use of AI technology. Hey for personal use it’s great, don’t go monetizing it though! Download here.
EPI Noise Offset
Want to increase contrast or make your images darker, well this LoRA is perfect for that. Highly recommended if you want to add that extra style to your generated images. Download Here.
These are just a few of the examples of types of LoRA you can download and use for free with stable diffusion. Don’t forget you can even mix multiple files together to create some crazy images. This is why Stable diffusion is our go to tool when creating AI generated images. It’s unique customization tools makes it beat it’s competition hands down.
Video Example of How to Use LoRA with Stable Diffusion
Conclusion
We do hope that this guide has helped you learn how to use LoRAs with stable diffusion. We find that creating the perfect prompt is only half the battle. If you twin this with a LoRA that compliments your image, you can really create something special.
What LoRAs would be interested in using? why not let us know in the comments below.
