Digit-Spatial-Diffusion's Hugging Face Models

This repository contains several machine learning models developed as part of my project, "Spatial Knowledge Injection into Diffusion-based Image Generation".

Project Overview

The project focused on the integration of spatial relations knowledge into diffusion-based image generation models. The main objective was to enhance the realism and context-awareness of the generated images. I experimented with four different methods to control the image generation process: baseline, handcrafted approach, typed control, and control dots.

Models

This repository includes the following models:

  1. Baseline - Stable Diffusion model trained with digits on our custom dataset
  2. Handcrafted - ControlNet with Handcrafted tesnor.
  3. Typed - ControlNet with typed digits as control signals.
  4. Typed dot - ControlNet with dots as control signals.

How to Use

Here's a basic example of how to use these models:

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Krishan3168/Digit-Spatial-Diffusion")
model = AutoModel.from_pretrained("Krishan3168/Digit-Spatial-Diffusion")

inputs = tokenizer("Example input", return_tensors="pt")
outputs = model(**inputs)