🎭 Emotion Puppeteer - Version 2 🎭

Choose LoRA, Inpaint, or Original

• Photoreal faces with exact control of facial expressions
• Use the LoRA to add emotion control to any other model
• Use the inpaint-model to change the expression of an existing output
• Use the original for strongestest expressions and merging/blending

⬇️   ⬇️   Read Important Instructions Below Samples   ⬇️   ⬇️

• Scroll down for important instructions about how to use this model

Sample Output (Raw Output)

🚨   🚨   Import Instructions   🚨   🚨

To get what you want, you should read this.

Use this prompt format for best results

• "{A} face, {B} eyes, {C} mouth, {D}, {anything else}"
• e.g. "plain face, seething eyes, screaming mouth, closeup portrait, looking away, old man with white hair"
• CFG = 5 is recommended

Explanation of the keyword bullets below

• caption used in training, # instances of caption in the training data, % images with caption, visual decription of images
• a caption found more frequency in training data results in a keyword that generates more reliable results

A - Face keywords

• plain, 125, 49%, unattractive face - my subjective opinion - biased towards extreme emotions
• cute, 112, 44%, attractive face - my subjective opinion - biased towards pleasant emotions
• adorable, 18, 7%, most attractive face - my subjective opinion - strongly biased towards pleasant emotions

B - Eye keywords

• seething, 42, 17%, >: shaped brows - scrunched brows
• grieving, 38, 15%, <: shaped brows - scrunched brows
• widening, 37, 15%, eyelids raised very high - brows arched
• glaring, 34, 13%, lowered brows - eyelids lowered
• pleasing, 34, 13%, neutral brows - eyes crecent shaped and smile crinkled
• piercing, 31, 12%, neutral brows - eyelids neutral
• squinting, 22, 9%, lowered brows - eyelids squeezed closed or almost closed
• winking, 19, 7%, right eye open, left eye closes

C - Mouth keywords

• breathing, 28, 11%, mouth relaxed and slightly open
• resting, 25, 10%, neutral or slight relaxed frown - lips together
• smiling, 25, 10%, smiling with lips parted - teeth together or slightly apart
• screaming, 20, 9%, mouth wide open - lower-teeth visible or chin scrunched
• sneering, 18, 7%, nose scrunched - left upper-lip raised - upper-teeth not bared
• singing, 17, 7%, mouth wide open and oval-shaped
• delighting, 17, 7%, smiling with mouth wide open
• kissing, 16, 6%, puckered lips
• gnashing, 15, 6%, nose scrunched - upper-lip evenly raised - teeth together - lower-teeth bared
• gasping, 14, 5%, mouth open and frowning-bean-shaped - chin relaxed
• relaxing, 14, 5%, slight relaxed smile - lips together
• frowning, 12, 5%, chin and or nose scrunched - lips together
• snarling, 10, 4%, nose scrunched - upper-lip evenly raised - lower-teeth not bared
• pouting, 10, 4%, slight frown - lower-lip extended
• grimacing, 9, 4%, lips apart and stretched wide - lips apart - teeth may be visible

D - Other keywords

• extreme closeup portrait, 141, 55%, cropped at neck or closer
• closeup portrait, 91, 36%, cropped at shoulders
• looking straight, 131, 51%, head facing forward and eyes looking into the camera
• lookway away, 46, 18%, head turned to three-quarters, regardless of eye direction

More image info

• (n/a), 255, 100%, all images were 512x512 and featured a single face without hands or glasses
• (n/a), 78, 31%, head facing forward while eyes are looking away - SD can't understand this a keyword for this
• (n/a), 23, 9%, cropped at waist or chest
• (n/a), 231, 91%, female
• (n/a), 24, 9%, male
• (n/a), 221, 87%, caucasian or ambiguous race
• (n/a), 34, 13%, unambiguously non-caucasian

FAQ

Help, the emotions are too extreme!

• Breath and just let it out. Oh, did you mean the model? Try adding "cute" to the prompt, or de-weighting the keyword, or using both the intense keword with a neutral one, or using an alternating prompt, e.g. in Auto111, "[(grieving eyes:0.5)|(piercing eyes:0.5)]"

How do I get a neutral expression?

• Use "pleasing eyes, relaxing mouth" or "piercing eyes, resting mouth"

How do I get stronger expressions?

• First, try removing words from your negative prompt. Then try weighting the keyword or using a neutral keyword in the negative prompt. Keep trying different seeds.

How do I change the expression, while keeping a consitent identity?

• Use an alternating prompt that combines 3 celebrity names, e.g. with Auto1111 "[(beyonce:1.0)|(bjork:1.0)|(adele:1.0)]"

I don't want / I do want NSFW

• No training images were NSFW or even racey, so if you're getting unwanted NSFW, it's in your prompt. If you want it, get more creative with the keywords and your prompts.

Why is most of the training data caucasian females?

• Because it's very hard to find good images, and the internet overall is biased towards images of caucasian females. If I can find more diverse images in the future, I'll retrain the model.

Can I have your training images?

• No. But the license allows you to fine-tune or merge the model.

How was the model trained?

• Using Everydream2, SD-1-5-vae as the base model, hand captioned images, and no classifiers/prior-preservation