0:00 welcome to episode four of this comfy UI Series today I want to show you the image to image workflow and how you can download and use Laura models in comy UI this is a basic sdxl text to image workflow I called it sdxl because it uses an sdxl Bas stable diffusion model you should be familiar with this workflow by now if not go back to the previous episodes we talked about how it still needs an image in this case a latent image in order to generate something but what if I don't want the starting point to be an empty image what if I want the image I choose to be the starting point well we can do that I'll delete this node we want to load our own image so we search for load and find a
0:46 node called load image it provides a basic drawing image but we can choose our own image to upload I'll select the bunny in the forest image when we try to connect it we see that there's an image output point but the case sampler requires a latent input so what do we do in this case we're working in pixel mode and we need to convert it to a format that stable diffusion understands which is the latent mode to do this we can use the vae encode node and now we have a latent that we can connect to the case sampler don't forget to connect the vae for my model the vae is included so I'll connect it to where it says vae
1:32 now when I cue The Prompt I see that the bunny didn't affect the image why is that it's because of the Deno value which is set to one we talked about the case sampler in the last episode but I didn't cover Deno yet and now is the perfect time to explain it the denoising strength determines how much the output image differs from the input image low denoising strength like 0.1 adds a small amount of noise to the input image meaning the output image will be very similar to the input with only slight modifications high denoising strength like value of one adds a significant amount of noise transforming the input image into a nearly unrecognizable state of noise this allows for a completely new image to be generated based on the
2:19 prompt with minimal influence from the original input the process behind how stable diffusion works is complex but here's a simplified version to help you understand in the first case with low denoid strength it was easier to guess what's in the image because you can still see the original content in the second case with high denoise strength it's harder to guess what's in the image because the noise has obscured the original content as a result the output could be anything like a robot a bunny or a house making it look like a completely new image think of it as adjusting the opacity of tracing paper overlay a low denying strength is like the tracing paper is very transparent so
3:05 the original drawing underneath is clearly visible and the changes are minor if is high denoising strength then the tracing paper is almost opaque allowing you to draw a completely new image with little reference to the original the term Den noise is used instead of add noise because the process starts by adding noise to the image but the main focus is on removing that noise to produce a clear high quality output back to our example I usually start with a value like 0.6 so that it is a little different from the original and the prompt has enough power to make changes to the image then I play around with lower or higher values this is the result for
3:50 0.5 if I lower the value even more to 0.4 it is different but not much and the prompt with robot doesn't have much impact if I set a value too high like 0.9 the result is completely different in this case if I want a robot bunny I use a value like 0.65 and make sure to include robot bunny in the prompt as you can see the bunny is in the same position and composition but now it's a robot and set in the environment I chose right now the image is generated with the width and height of the loaded image however we know that depending on the the model it prefers to work with certain sizes my loaded image was 1,024
4:37 pixels because I work with an sdxl model so something around that value produces good results but what happens if I load a larger image instead let's try it I'll choose a different image that I've upscaled four times now when I cue The Prompt it looks like it's going to take forever to generate if I open the command window and check what's happening you can see it says it ran out of memory and is retrying with a tiled vae so clearly that image is way too big so I'll go to VQ to see the running jobs and then click cancel job but if you're wondering what would have happened if I let it run well I would have ended up with an image that looks like this instead of taking 4 seconds it took 200 seconds it's the same size as the loaded
5:24 image but clearly not what I wanted so the solution is to make the image smaller before uploading it using software like Photoshop or we can do it uh right here if I create a little space between nodes and search for image I can find a node called upscale image when I select that we can see width and height options just like we had in the text to image workflow with latent images now we can control the exact size my image is square so I'll choose values close to that but if you have a landscape or portrait image just make sure the sizes are close close to 1,24 pixels and preferably the values are divisible by 64 because computers work better with
6:11 those numbers I'm deleting the link between the loaded image and the vae en code node to make room for the upscale image node then I'll connect them by matching the image output with the image input even if you're not sure how it works it's fairly intuitive to connect inputs and outputs with the same color and name now when I cue The Prompt again you can see it's quite fast if I look in the command window I see it took only 5 Seconds instead of 200 the resulting image size matches the dimensions you set in this box you can also enable cropping if you want the result to have a different ratio than the uploaded image but what if I want this generated image to be the loaded image without
6:56 saving it and loading it again you can do that by right clicking on the image and choosing copy image then go to the load image node rightclick and although I expected a paste image option it seems to be missing however you can still paste it using the control+v shortcut I'll do that now when I cue The Prompt I get another image of the robot based on this new input even if I delete the load image node you can still use any image copied to the clipboard whether it's from Photoshop a screenshot or elsewhere when you use control+v it will create a load image node with the pasted image this way you can easily connect that node and run it to get your Generations
7:43 faster without having to repeatedly save and upload uh images let's move on to Laura now I'm loading the basic sdxl text to image workflow again you can quickly see that this one loads an empty latent image but first let's talk a bit about what a Laura is Laura stands for low rank adaptation it is a method used to fine-tune large pre-trained neural network models efficiently Laura fine-tunes large models by updating only a small targeted part of the model this makes it quicker and less resource intensive to adapt the model to specific tasks or new data sets with Laura you can find tuna model to generate specific
8:30 objects like cars or gas gadgets that it may not have been able to generate before this helps the model understand and create images of things it previously didn't know how to render you can also adapt a model to generate images that accurately depict a specific person capturing their unique features and style you can also train a model to create images in a particular art style such as impressionism surrealism or a specific artist's style imagine you have a toolbox with many tools if you need to do a specific job like tightening a unique type of screw you could add a special attachment to one of your screwdrivers this attachment Laura lets you do the job without needing a whole
9:19 new toolbox similarly Laura tweaks a small part of a big model to make it better at a specific task such as creating images of a particular object depicting a specific person or emulating an art style I know this might sound a bit basic and boring but it will make more sense once we test a Laura Laura is a model you can download similar to the stable diffusion model however it's smaller and faster to create there are various tutorials online on how to train aura but in this video I'll show you how to download one and use it if we go to the civit AI website and click on the models tab we can see all kinds of models by going to the filters we can choose Laura as the model type and
10:04 you'll also need to know which base model you're using the most popular ones are v1.5 but in my case I have an sdxl model so I'll select that for the time period I'll choose year to get more models you can then click on a Model to see what types of images it can generate for this video I'll show you two examples search for The Ether Cloud Model you'll see that it lists the base model as sdxl which is what I want also make sure to note the trigger words listed you need to use these exact words in the prompt you can read more information here about what weight to use and other details then click the download button to download um the model
10:50 after that place it in the comy UI folder navigate to the models folder and then to the luras folder where you should save the downloaded model while it's downloading let's search for another one called ether fire check to make sure it uses the same base model sdxl you'll see that it has different trigger words that you need to use in the prompt then look for more information to see if there's anything else you should know after that download it and place it in the same Loris folder once you have downloaded the models it should look like this you'll see even has a file that says put luras here now
11:38 go to comfy UI if it was open when you downloaded the files just click the refresh button then we can see how to add the models to the workflow um we need a Laura so let's search for the word Laura to see what we can find uh the load Laura option looks like something we need um you'll see that from the drop- down menu you can select the downloaded models uh I'll select the the cloud model first but where do we place this node let's think uh logically we have inputs and outputs labeled model so we need to position it somewhere between the load checkpoint and the K sampler as only those currently have the right connection points we have Clips on
12:23 the input and output One Clip will definitely go to the load checkpoint but where does the output clip go since the K sampler doesn't have have a clip only positive and negative logically the node should be placed between the load checkpoint and the prompts so let's do that connect the clip from the low checkpoint to the Laura and then connect the clip from the Laura to the prompts next we want the model link to go through the Laura so I'll delete the existing link and recreate it connect from the load checkpoint to the Laura and then from the Laura to the K sampler it's like making a little detour to pass through the the Laura node why is the Laura placed between the checkpoint and the prompt by placing the Laura node
13:09 before the prompt encoding the model is first adapted with the Laura adjustments this ensures that these adjustments are considered when interpreting the prompts in other words the modified model is used to understand and generate images based on the given prompts so let's add a prompt making sure to use the trigger word specified on the model page that trigger words are Cloud that look like and I'll add a bullhead blue sky photo after them I will also set the C to be fixed so we can compare the results later I'll change randomize to fixed then I'll bypass the Laura node to see what we get without using the Laura for that prompt when I run it you'll see that I get a cloud but it doesn't look
13:57 like a bull as I want so let's enable the Lura node Now by using the bypass option to activate it now when I cue The Prompt we get something closer to what we had in mind as you've likely noticed this Laura is specialized in making clouds resemble different subjects you also have an option called strength model which determines how strongly the Laura adjustments are applied to the main model it adjusts the impact of the fine-tuning with a higher value making the adjustments more influential and a lower value making them less noticeable I usually set it between the 0.3 and one if you use a value that's too high like two for example it can make the image
14:44 look worse so try to use a value closer to the recommended settings which is often around one uh play around with different prompts just remember to include those trigger words look a cloud that looks like a bunny let's try the second Laura the one with the fire used the trigger words fire that looks like and when you run it you'll get a fire that looks like a bull let's change the seed back to random and try a few generations to see what we can get you might be wondering if this will work with an image you have like if you want to get a version of your bunny in fire yes you can do that too let's convert this workflow into an image to image one
15:29 delete the empty lat an image node and add a load image node then use an encode vae node to encode the image and remember to connect the vae to ensure the image isn't too large and works well with stable diffusion use the image upscale node Place between the load image and vae and code nodes for the prompt I usually add the trigger words first so I don't forget them then I include the rest of the details like a bunny in the forest it's important to remember to reduce the Deno value so um the result looks more similar to your image you can play with different values until you get something you like in my case a value of 0.5 made the bunny look
16:16 like it was ignited in fire to recap the process begins by loading a pre-trained model Juggernaut X which is an sdxl model in this case next Alura model called e clouds is loaded which fine-tunes the base model to make it better at creating clouds that resemble different subjects then the positive and negative text prompts are encoded so stable diffusion can understand them an image of a bunny is loaded and scaled to a size that works well with our model after scaling the image is encoded together with the information from the model Laura and prompts it goes to the samper which performs the image generation finally the image is decoded and saved to our computer computer I've
17:01 tried to explain everything as clearly as possible with my understanding uh in a simplified way even if it's not the most accurate I hope it helps make the concepts easier to understand and that's all for today's episode if you found something useful please leave a like or a comment thank you for watching have a great day [Music]
