ai_or_not sumsub image_classification sumsubaiornot aiornot deepfake synthetic generated pytorch

For Fake's Sake: a set of models for detecting generated and synthetic images

Many people on the internet have recently been tricked by fake images of Pope Francis wearing a coat or of Donald Trump's arrest. To help combat this issue, we provide detectors for such images generated by popular tools like Midjourney and Stable Diffusion.

Image1 Image2 Image3

Model Details

Model Description

Demo

The demo page can be found here.

How to Get Started with the Model & Model Sources

Use the code below to get started with the model:

git lfs install
git clone https://huggingface.co/Sumsub/Sumsub-ffs-synthetic-1.0_mj_5 sumsub_synthetic_mj_5

from sumsub_synthetic_mj_5.pipeline import PreTrainedPipeline
from PIL import Image

pipe = PreTrainedPipeline("sumsub_synthetic_mj_5/")

img = Image.open("sumsub_synthetic_mj_5/images/2.jpg")

result = pipe(img)
print(result) #[{'label': 'by AI', 'score': 0.201515331864357}, {'label': 'by human', 'score': 0.7984846234321594}]

You may need these prerequsites installed:

pip install -r requirements.txt
pip install "git+https://github.com/rwightman/pytorch-image-models"
pip install "git+https://github.com/huggingface/huggingface_hub"

Training Details

Training Data

The models were trained on the following datasets:

Midjourney datasets:

Training Procedure

To improve the performance metrics, we used data augmentations such as rotation, crop, Mixup and CutMix. Each model was trained for 30 epochs using early stopping with batch size equal to 32.

Evaluation

For evaluation we used the following datasets:

Midjourney datasets:

Realistic images:

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Model Dataset Accuracy
midjourney_5M Kaggle Midjourney 2022-250k 0.852
midjourney_5M Kaggle Midjourney v5.1 0.875
midjourney_5M MS COCO 0.822

Limitations

Image1

Citation

If you find this useful, please cite as:

@misc{sumsubaiornot, 
    publisher = {Sumsub},
    url       = {https://huggingface.co/Sumsub/Sumsub-ffs-synthetic-1.0_mj_5},
    year      = {2023},
    author    = {Savelyev, Alexander and Toropov, Alexey and Goldman-Kalaydin, Pavel and Samarin, Alexey},
    title     = {For Fake's Sake: a set of models for detecting deepfakes, generated images and synthetic images}
}

References