SpanMarker with bert-base-cased on DFKI-SLT/few-nerd
This is a SpanMarker model trained on the DFKI-SLT/few-nerd dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-cased as the underlying encoder.
Model Details
Model Description
- Model Type: SpanMarker
- Encoder: bert-base-cased
- Maximum Sequence Length: 256 tokens
- Maximum Entity Length: 8 words
- Training Dataset: DFKI-SLT/few-nerd
- Language: en
- License: cc-by-sa-4.0
Model Sources
- Repository: SpanMarker on GitHub
- Thesis: SpanMarker For Named Entity Recognition
Model Labels
| Label | Examples |
|---|---|
| art | "Time", "The Seven Year Itch", "Imelda de ' Lambertazzi" |
| building | "Boston Garden", "Sheremetyevo International Airport", "Henry Ford Museum" |
| event | "French Revolution", "Iranian Constitutional Revolution", "Russian Revolution" |
| location | "Croatian", "the Republic of Croatia", "Mediterranean Basin" |
| organization | "IAEA", "Texas Chicken", "Church 's Chicken" |
| other | "N-terminal lipid", "BAR", "Amphiphysin" |
| person | "Hicks", "Edmund Payne", "Ellaline Terriss" |
| product | "100EX", "Phantom", "Corvettes - GT1 C6R" |
Evaluation
Metrics
| Label | Precision | Recall | F1 |
|---|---|---|---|
| all | 0.7806 | 0.7630 | 0.7717 |
| art | 0.7465 | 0.7395 | 0.7430 |
| building | 0.6027 | 0.7184 | 0.6555 |
| event | 0.6178 | 0.5438 | 0.5784 |
| location | 0.8138 | 0.8547 | 0.8338 |
| organization | 0.7359 | 0.6613 | 0.6966 |
| other | 0.7397 | 0.6166 | 0.6726 |
| person | 0.8845 | 0.9071 | 0.8957 |
| product | 0.7056 | 0.5932 | 0.6446 |
Uses
Direct Use for Inference
from span_marker import SpanMarkerModel
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")
# Run inference
entities = model.predict("Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2–0 by Sheffield United on 28 March 1925.")
Downstream Use
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
from span_marker import SpanMarkerModel, Trainer
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")
# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003
# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
model=model,
train_dataset=dataset["train"],
eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("span_marker_model_id-finetuned")
</details>
<!--
Out-of-Scope Use
List how the model may foreseeably be misused and address what users ought not to do with the model. -->
<!--
Bias, Risks and Limitations
What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->
<!--
Recommendations
What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Sentence length | 1 | 24.4956 | 163 |
| Entities per sentence | 0 | 2.5439 | 35 |
Training Hyperparameters
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
Training Results
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
|---|---|---|---|---|---|---|
| 0.1629 | 200 | 0.0359 | 0.6908 | 0.6298 | 0.6589 | 0.9053 |
| 0.3259 | 400 | 0.0237 | 0.7535 | 0.7018 | 0.7267 | 0.9227 |
| 0.4888 | 600 | 0.0216 | 0.7659 | 0.7438 | 0.7547 | 0.9333 |
| 0.6517 | 800 | 0.0208 | 0.7730 | 0.7550 | 0.7639 | 0.9344 |
| 0.8147 | 1000 | 0.0197 | 0.7805 | 0.7567 | 0.7684 | 0.9372 |
| 0.9776 | 1200 | 0.0194 | 0.7771 | 0.7634 | 0.7702 | 0.9381 |
Framework Versions
- Python: 3.10.12
- SpanMarker: 1.4.0
- Transformers: 4.34.0
- PyTorch: 2.0.1+cu118
- Datasets: 2.14.5
- Tokenizers: 0.14.1
Citation
BibTeX
@software{Aarsen_SpanMarker,
author = {Aarsen, Tom},
license = {Apache-2.0},
title = {{SpanMarker for Named Entity Recognition}},
url = {https://github.com/tomaarsen/SpanMarkerNER}
}
<!--
Glossary
Clearly define terms in order to be accessible across audiences. -->
<!--
Model Card Authors
Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->
<!--
Model Card Contact
Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->
