SpanMarker with bert-base-multilingual-cased on xtreme/PAN-X.es
This is a SpanMarker model trained on the xtreme/PAN-X.es dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-multilingual-cased as the underlying encoder.
Model Details
Model Description
- Model Type: SpanMarker
- Encoder: bert-base-multilingual-cased
- Maximum Sequence Length: 512 tokens
- Maximum Entity Length: 8 words
- Training Dataset: xtreme/PAN-X.es
- Languages: es
- License: cc-by-4.0
Model Sources
- Repository: SpanMarker on GitHub
- Thesis: SpanMarker For Named Entity Recognition
Model Labels
| Label | Examples |
|---|---|
| LOC | "Salamanca", "Paris", "Barcelona (España)" |
| ORG | "ONU", "Fútbol Club Barcelona", "Museo Nacional del Prado" |
| PER | "Fray Luis de León", "Leo Messi", "Álvaro Bartolomé" |
Uses
Direct Use for Inference
from span_marker import SpanMarkerModel
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("alvarobartt/bert-base-multilingual-cased-ner-spanish")
# Run inference
entities = model.predict("Marie Curie fue profesora en la Universidad de Paris.")
<!--
Out-of-Scope Use
List how the model may foreseeably be misused and address what users ought not to do with the model. -->
<!--
Bias, Risks and Limitations
What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->
<!--
Recommendations
What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Sentence length | 3 | 6.4642 | 64 |
| Entities per sentence | 1 | 1.2375 | 24 |
Training Hyperparameters
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 2
Training Results
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
|---|---|---|---|---|---|---|
| 0.3998 | 1000 | 0.0388 | 0.8761 | 0.8641 | 0.8701 | 0.9223 |
| 0.7997 | 2000 | 0.0326 | 0.8995 | 0.8740 | 0.8866 | 0.9341 |
| 1.1995 | 3000 | 0.0277 | 0.9076 | 0.9019 | 0.9047 | 0.9424 |
| 1.5994 | 4000 | 0.0261 | 0.9143 | 0.9113 | 0.9128 | 0.9473 |
| 1.9992 | 5000 | 0.0234 | 0.9231 | 0.9143 | 0.9187 | 0.9502 |
Framework Versions
- Python: 3.10.12
- SpanMarker: 1.3.1.dev
- Transformers: 4.33.3
- PyTorch: 2.0.1+cu118
- Datasets: 2.14.5
- Tokenizers: 0.13.3
Citation
BibTeX
@software{Aarsen_SpanMarker,
author = {Aarsen, Tom},
license = {Apache-2.0},
title = {{SpanMarker for Named Entity Recognition}},
url = {https://github.com/tomaarsen/SpanMarkerNER}
}
<!--
Glossary
Clearly define terms in order to be accessible across audiences. -->
<!--
Model Card Authors
Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->
<!--
Model Card Contact
Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->
