UPB's Multi-task Learning model for AuTexTification

This is a model for classifying text as human- or LLM-generated.

This model was trained for one of University Politehnica of Bucharest's (UPB) submissions to the AuTexTification shared task.

This model was trained using multi-task learning to predict whether a text document was written by a human or a large language model, and whether it was written in English or Spanish.

The model outputs a score/probability for each task, but it also makes a binary prediction for detecting synthetic text, based on a threshold.

Training data

The model was trained on approximately 33,845 English documents and 32,062 Spanish documents, covering five different domains, such as legal or social media. The dataset is available on Zenodo (more instructions here).

Evaluation results

These results were computed as part of the AuTexTification shared task:

Using the model

You can load the model and its tokenizer using AutoModel and AutoTokenizer.

This is an example of using the model for inference:

import torch
from transformers import AutoModel, AutoTokenizer

checkpoint = "pandrei7/autextification-upb-mtl"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModel.from_pretrained(checkpoint, trust_remote_code=True)

texts = ["Enter your text here."]
tokenized_batch = tokenizer(
    texts,
    padding=True,
    truncation=True,
    max_length=512,
    return_tensors="pt",
)

model.eval()
with torch.no_grad():
    preds = model(tokenized_batch)

print("Bot?\t", preds["is_bot"][0].item())
print("Bot score\t", preds["bot_prob"][0].item())
print("English score\t", preds["english_prob"][0].item())