This is a model for text summarization in Spanish. It has been trained on the Spanish portion of mlsum, finetuning the mt5-base model.

We used the following set of hyperparameters:

    {
      "learning_rate": 2e-5,
      "num_train_epochs": 8,
      "per_device_train_batch_size": 1,
      "per_device_eval_batch_size": 1,
      "gradient_accumulation_steps": 256,
      "fp16": False,
      "weight_decay": 0.01,
    }

The model was finetuned to predict the concatenation of the title and the summary of each item in the dataset. The results that we show below correspond to the set split of mlsum. The metrics for the concatenation of titles and summaries are:

{'rouge1': 26.946, 'rouge2': 10.7271, 'rougeL': 21.4591, 'rougeLsum': 24.5001, 'gen_len': 18.9628}

On the other hand, the metrics for just the summaries are:

{'rouge1': 21.9788, 'rouge2': 6.5249, 'rougeL': 17.7444, 'rougeLsum': 18.9783, 'gen_len': 18.9628}

This model is really easy to use, and with the following lines of code you can just start summarizing your documents in Spanish:

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

text = "Hola esto es un ejemplo de texto a resumir. Poco hay que resumir aquí, pero es sólo de muestra."
model_str = "IIC/mt5-spanish-mlsum"
tokenizer = AutoTokenizer.from_pretrained(model_str)
model = AutoModelForSeq2SeqLM.from_pretrained(model_str)

input_ids = tokenizer(text, return_tensors="pt").input_ids

output_ids = model.generate(input_ids)[0]
print(tokenizer.decode(output_ids, skip_special_tokens=True))

Contributions

Thanks to @avacaondata, @alborotis, @albarji, @Dabs, @GuillemGSubies for adding this model.