SpanMarker with bert-base-cased on DFKI-SLT/few-nerd
This is a SpanMarker model trained on the DFKI-SLT/few-nerd dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-cased as the underlying encoder.
Model Details
Model Description
- Model Type: SpanMarker
- Encoder: bert-base-cased
- Maximum Sequence Length: 256 tokens
- Maximum Entity Length: 8 words
- Training Dataset: DFKI-SLT/few-nerd
- Language: en
- License: cc-by-sa-4.0
Model Sources
- Repository: SpanMarker on GitHub
- Thesis: SpanMarker For Named Entity Recognition
Model Labels
| Label | Examples |
|---|---|
| art | "The Seven Year Itch", "Time", "Imelda de ' Lambertazzi" |
| building | "Boston Garden", "Henry Ford Museum", "Sheremetyevo International Airport" |
| event | "Russian Revolution", "Iranian Constitutional Revolution", "French Revolution" |
| location | "Croatian", "the Republic of Croatia", "Mediterranean Basin" |
| organization | "IAEA", "Church 's Chicken", "Texas Chicken" |
| other | "BAR", "Amphiphysin", "N-terminal lipid" |
| person | "Ellaline Terriss", "Edmund Payne", "Hicks" |
| product | "Phantom", "100EX", "Corvettes - GT1 C6R" |
Evaluation
Metrics
| Label | Precision | Recall | F1 |
|---|---|---|---|
| all | 0.7793 | 0.7634 | 0.7713 |
| art | 0.7608 | 0.7395 | 0.75 |
| building | 0.6095 | 0.6816 | 0.6435 |
| event | 0.6094 | 0.5392 | 0.5721 |
| location | 0.8112 | 0.8599 | 0.8348 |
| organization | 0.7335 | 0.6827 | 0.7072 |
| other | 0.7715 | 0.5822 | 0.6636 |
| person | 0.8635 | 0.9044 | 0.8835 |
| product | 0.7172 | 0.5932 | 0.6494 |
Uses
Direct Use for Inference
from span_marker import SpanMarkerModel
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")
# Run inference
entities = model.predict("Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2–0 by Sheffield United on 28 March 1925.")
Downstream Use
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
from span_marker import SpanMarkerModel, Trainer
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")
# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003
# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
model=model,
train_dataset=dataset["train"],
eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("span_marker_model_id-finetuned")
</details>
<!--
Out-of-Scope Use
List how the model may foreseeably be misused and address what users ought not to do with the model. -->
<!--
Bias, Risks and Limitations
What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->
<!--
Recommendations
What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Sentence length | 1 | 24.4956 | 163 |
| Entities per sentence | 0 | 2.5439 | 35 |
Training Hyperparameters
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
Training Results
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
|---|---|---|---|---|---|---|
| 0.1629 | 200 | 0.0339 | 0.7327 | 0.6104 | 0.6660 | 0.9052 |
| 0.3259 | 400 | 0.0234 | 0.7717 | 0.6954 | 0.7316 | 0.9212 |
| 0.4888 | 600 | 0.0223 | 0.7598 | 0.7447 | 0.7522 | 0.9337 |
| 0.6517 | 800 | 0.0207 | 0.7600 | 0.7625 | 0.7612 | 0.9362 |
| 0.8147 | 1000 | 0.0196 | 0.7847 | 0.7484 | 0.7661 | 0.9369 |
| 0.9776 | 1200 | 0.0192 | 0.7809 | 0.7584 | 0.7695 | 0.9376 |
Framework Versions
- Python: 3.10.12
- SpanMarker: 1.4.0
- Transformers: 4.34.0
- PyTorch: 2.0.1+cu118
- Datasets: 2.14.5
- Tokenizers: 0.14.1
Citation
BibTeX
@software{Aarsen_SpanMarker,
author = {Aarsen, Tom},
license = {Apache-2.0},
title = {{SpanMarker for Named Entity Recognition}},
url = {https://github.com/tomaarsen/SpanMarkerNER}
}
<!--
Glossary
Clearly define terms in order to be accessible across audiences. -->
<!--
Model Card Authors
Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->
<!--
Model Card Contact
Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->
