MetaExtractor

This model extracts metadata from research articles related to Paleoecology.

The entities detected by this model are:

AGE: when historical ages are mentioned such as 1234 AD or 4567 BP (before present)
TAXA: plant or animal taxa names indicating what samples contained
GEOG: geographic coordinates indicating where samples were excavated from, e.g. 12'34"N 34'23"W
SITE: site names for where samples were excavated from
REGION: more general regions to provide context for where sites are located
EMAIL: researcher emails in the articles able to be used for follow-up contact
ALTI: altitudes of sites from where samples were excavated, e.g. 123 m a.s.l (above sea level)

Model Details

Model Description

Developed by: Ty Andrews, Jenit Jain, Shaun Hutchinson, Kelly Wu, and Simon Goring
Shared by: Neotoma Paleocology Database
Model type: Token Classification
Language(s) (NLP): English
License: MIT
Finetuned from model: roberta-base

Model Sources

Repository: https://github.com/NeotomaDB/MetaExtractor
Paper: TBD
Demo: TBD

Uses

This model can be used to extract entities from any text that are Paeleoecology related or tangential. Potential uses include identifying unique SITE names in research papers in other domains.

Direct Use

This model is deployed on the xDD (formerly GeoDeepDive) servers where it is getting fed new research articles relevant to Neotoma and returning the extracted data.

This approach could be adapted to other domains by using the training and development code found github.com/NeotomaDB/MetaExtractor to run similar data extraction for other research domains.

Bias, Risks, and Limitations

This model was trained entirely on English research articles and will likely not perform well on research in other languages. Also, the articles used to train the model were chosen based on being already present in the Neotoma database and therefore may have selection bias as they represent what is already known to be relevant to Neotoma and may not correctly manage new, previously missed articles.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("finding-fossils/metaextractor")
model = AutoModelForTokenClassification.from_pretrained("finding-fossils/metaextractor")
ner_pipe = pipeline('ner', model=model, tokenizer=tokenizer, aggregation_strategy="simple")

ner_pipe("In Northern Canada, the BGC site core was primarily made up of Pinus pollen.")

# Output
[
  {
    "entity_group": "REGION",
    "score": 0.8088379502296448,
    "word": " Northern Canada,",
    "start": 3,
    "end": 19
  },
  {
    "entity_group": "SITE",
    "score": 0.8307041525840759,
    "word": " BGC",
    "start": 24,
    "end": 27
  },
  {
    "entity_group": "TAXA",
    "score": 0.9806344509124756,
    "word": " Pinus",
    "start": 63,
    "end": 68
  }
]

Training Details

Training Data

The model was trained using a set of 39 research articles deemed relevant to the Neotoma Database. All articles were written in English. The entities were labeled by the project team along with using pre-labelling with early models to speed up the labelling process.

A 70/15/15 train/val/test split was used which had the following breakdown of words and entities.

	Train	Validation	Test
Articles	28	6	6
Words	220857	37809	36098
TAXA Entities	3352	650	570
SITE Entities	1228	177	219
REGION Entities	2314	318	258
GEOG Entities	188	37	8
AGE Entities	919	206	153
ALTI Entities	99	24	14
Email Entities	14	4	11

Training Procedure

For full training details please see the GitHub repository and Wiki: github.com/NeotomaDB/MetaExtractor

Results & Metrics

For full model results see the report here: Final Project Report