Model Card for SpaBERT-enpt for Semantic Role Labeling (cased)

This model is fine-tuned on a version of multilingual BERT which is pre-trained on the SRL task for English and Portuguese, and is one of 24 models introduced as part of this project.

Model Details

Model Description

SpaBERT-enpt for Semantic Role Labeling (SRL) is a transformers model, leveraging mBERT's extensive pretraining on 104 languages to achieve better SRL predictions for Spanish. This model is additionally pre-trained on the SRL task for English and Portuguese. It was fine-tuned on Spanish with the following objectives:

• Identify up to 16 verbal roots within a sentence.
• Identify available arguments and thematic roles for each verbal root.

Labels are formatted as: r#:tag, where r# links the token to a specific verbal root of index #, and tag identifies the token as the verbal root (root) or an individual argument (arg0/arg1/arg2/arg3/argM) and it's thematic role (adv/agt/atr/ben/cau/cot/des/efi/ein/exp/ext/fin/ins/loc/mnr/ori/pat/src/tem/tmp)

• Developed by: Micaella Bruton
• Model type: Transformers
• Language(s) (NLP): Spanish (es), English (en), Portuguese (pt)
• License: Apache 2.0
• Finetuned from model: English & Portuguese pre-trained multilingual BERT

Model Sources

• Repository: GalicianSRL
• Paper: To be updated

Uses

This model is intended to be used to develop and improve natural language processing tools for Spanish.

Bias, Risks, and Limitations

The Spanish training set lacked highly complex sentences and as such, performs much better on sentences of mid- to low-complexity.

Training Details

Training Data

This model was pre-trained on the OntoNotes 5.0 English SRL corpus and PropBank.Br Portuguese SRL corpus. This model was fine-tuned on the "train" portion of the SpanishSRL Dataset produced as part of this same project.

Training Hyperparameters

• Learning Rate: 2e-5
• Batch Size: 16
• Weight Decay: 0.01
• Early Stopping: 10 epochs

Evaluation

Testing Data

This model was tested on the "test" portion of the SpanishSRL Dataset produced as part of this same project.

Metrics

seqeval is a Python framework for sequence labeling evaluation. It can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, and semantic role labeling. It supplies scoring both overall and per label type.

Overall:

• accuracy: the average accuracy, on a scale between 0.0 and 1.0.
• precision: the average precision, on a scale between 0.0 and 1.0.
• recall: the average recall, on a scale between 0.0 and 1.0.
• f1: the average F1 score, which is the harmonic mean of the precision and recall. It also has a scale of 0.0 to 1.0.

Per label type:

• precision: the average precision, on a scale between 0.0 and 1.0.
• recall: the average recall, on a scale between 0.0 and 1.0.
• f1: the average F1 score, on a scale between 0.0 and 1.0.

Results

Citation

BibTeX:

@mastersthesis{bruton-galician-srl-23,
    author = {Bruton, Micaella},
    title = {BERTie Bott's Every Flavor Labels: A Tasty Guide to Developing a Semantic Role Labeling Model for Galician},
    school = {Uppsala University},
    year = {2023},
    type = {Master's thesis},
}