Fine-tuning Stable Diffusion XL for custom face and thumbs up gesture

Colab Notebook implementation of LoRA based Dreambooth technique, utilizing Huggingface diffusers, for fine-tuning the model to reliably predict thumbs up gesture with a custom face.

Datasets

Use the following links to download and view the thumbs-up and selfie images respectively

  1. Thumbs Up
  2. Selfies

Checkpoints

  1. lora-trained-thumbs-up-pw-0.5-steps-800 contains the weights for the model finetuned for the thumbs-up gesture
  2. lora-trained-thumbs-up-pw-0.5-steps-800-uttam-pw-0.5-steps-1600 contains the weights for the model finetuned for the custom face

Results

Here are some of the quantitative results trigger keyword: a photo of ug2146 giving thumbs up

trigger keyword: + in a joker costume