<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6832
- Answer: {'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}
- Header: {'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119}
- Question: {'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065}
- Overall Precision: 0.7259
- Overall Recall: 0.7812
- Overall F1: 0.7525
- Overall Accuracy: 0.8047
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.8109 | 1.0 | 10 | 1.6140 | {'precision': 0.013268998793727383, 'recall': 0.013597033374536464, 'f1': 0.013431013431013432, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.10519645120405577, 'recall': 0.07793427230046948, 'f1': 0.08953613807982741, 'number': 1065} | 0.0581 | 0.0472 | 0.0521 | 0.3634 |
| 1.468 | 2.0 | 20 | 1.2385 | {'precision': 0.13105413105413105, 'recall': 0.11372064276885044, 'f1': 0.12177365982792852, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4378029079159935, 'recall': 0.5089201877934272, 'f1': 0.4706904038211029, 'number': 1065} | 0.3268 | 0.3181 | 0.3224 | 0.5793 |
| 1.0973 | 3.0 | 30 | 0.9328 | {'precision': 0.41563275434243174, 'recall': 0.41409147095179233, 'f1': 0.41486068111455104, 'number': 809} | {'precision': 0.03125, 'recall': 0.008403361344537815, 'f1': 0.013245033112582781, 'number': 119} | {'precision': 0.6483720930232558, 'recall': 0.6544600938967137, 'f1': 0.6514018691588785, 'number': 1065} | 0.5400 | 0.5183 | 0.5289 | 0.7016 |
| 0.8233 | 4.0 | 40 | 0.7582 | {'precision': 0.6106290672451193, 'recall': 0.695920889987639, 'f1': 0.6504910456383594, 'number': 809} | {'precision': 0.17543859649122806, 'recall': 0.08403361344537816, 'f1': 0.11363636363636363, 'number': 119} | {'precision': 0.6941176470588235, 'recall': 0.72018779342723, 'f1': 0.7069124423963133, 'number': 1065} | 0.6430 | 0.6724 | 0.6573 | 0.7595 |
| 0.6573 | 5.0 | 50 | 0.6894 | {'precision': 0.6411332633788038, 'recall': 0.7552533992583437, 'f1': 0.6935300794551646, 'number': 809} | {'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119} | {'precision': 0.7267080745341615, 'recall': 0.7690140845070422, 'f1': 0.7472627737226277, 'number': 1065} | 0.6704 | 0.7296 | 0.6987 | 0.7838 |
| 0.5504 | 6.0 | 60 | 0.6623 | {'precision': 0.6652675760755509, 'recall': 0.7836835599505563, 'f1': 0.7196367763904654, 'number': 809} | {'precision': 0.19736842105263158, 'recall': 0.12605042016806722, 'f1': 0.15384615384615385, 'number': 119} | {'precision': 0.731626754748142, 'recall': 0.831924882629108, 'f1': 0.7785588752196836, 'number': 1065} | 0.6853 | 0.7702 | 0.7253 | 0.7933 |
| 0.4731 | 7.0 | 70 | 0.6464 | {'precision': 0.6681127982646421, 'recall': 0.761433868974042, 'f1': 0.7117273252455227, 'number': 809} | {'precision': 0.22641509433962265, 'recall': 0.20168067226890757, 'f1': 0.21333333333333335, 'number': 119} | {'precision': 0.7656794425087108, 'recall': 0.8253521126760563, 'f1': 0.7943967464979665, 'number': 1065} | 0.6981 | 0.7622 | 0.7287 | 0.8003 |
| 0.428 | 8.0 | 80 | 0.6407 | {'precision': 0.6865671641791045, 'recall': 0.796044499381953, 'f1': 0.7372638809387521, 'number': 809} | {'precision': 0.22321428571428573, 'recall': 0.21008403361344538, 'f1': 0.21645021645021645, 'number': 119} | {'precision': 0.7692307692307693, 'recall': 0.8262910798122066, 'f1': 0.7967406066093254, 'number': 1065} | 0.7060 | 0.7772 | 0.7399 | 0.8053 |
| 0.3776 | 9.0 | 90 | 0.6475 | {'precision': 0.7108843537414966, 'recall': 0.7750309023485785, 'f1': 0.7415730337078651, 'number': 809} | {'precision': 0.23770491803278687, 'recall': 0.24369747899159663, 'f1': 0.24066390041493776, 'number': 119} | {'precision': 0.7615780445969125, 'recall': 0.8338028169014085, 'f1': 0.796055580457194, 'number': 1065} | 0.7115 | 0.7747 | 0.7418 | 0.8022 |
| 0.3434 | 10.0 | 100 | 0.6694 | {'precision': 0.6895074946466809, 'recall': 0.796044499381953, 'f1': 0.7389558232931727, 'number': 809} | {'precision': 0.2831858407079646, 'recall': 0.2689075630252101, 'f1': 0.27586206896551724, 'number': 119} | {'precision': 0.7693661971830986, 'recall': 0.8206572769953052, 'f1': 0.79418446160836, 'number': 1065} | 0.7100 | 0.7777 | 0.7423 | 0.8007 |
| 0.3082 | 11.0 | 110 | 0.6749 | {'precision': 0.6961206896551724, 'recall': 0.7985166872682324, 'f1': 0.7438111686816349, 'number': 809} | {'precision': 0.2905982905982906, 'recall': 0.2857142857142857, 'f1': 0.288135593220339, 'number': 119} | {'precision': 0.7794779477947795, 'recall': 0.8131455399061033, 'f1': 0.7959558823529411, 'number': 1065} | 0.7171 | 0.7757 | 0.7452 | 0.7985 |
| 0.2933 | 12.0 | 120 | 0.6635 | {'precision': 0.7130242825607064, 'recall': 0.7985166872682324, 'f1': 0.7533527696793003, 'number': 809} | {'precision': 0.28448275862068967, 'recall': 0.2773109243697479, 'f1': 0.28085106382978725, 'number': 119} | {'precision': 0.78125, 'recall': 0.8215962441314554, 'f1': 0.8009153318077803, 'number': 1065} | 0.7255 | 0.7797 | 0.7516 | 0.8056 |
| 0.278 | 13.0 | 130 | 0.6760 | {'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809} | {'precision': 0.3170731707317073, 'recall': 0.3277310924369748, 'f1': 0.32231404958677684, 'number': 119} | {'precision': 0.7897111913357401, 'recall': 0.8215962441314554, 'f1': 0.8053382420616658, 'number': 1065} | 0.7297 | 0.7827 | 0.7553 | 0.8049 |
| 0.2699 | 14.0 | 140 | 0.6824 | {'precision': 0.7041484716157205, 'recall': 0.7972805933250927, 'f1': 0.7478260869565218, 'number': 809} | {'precision': 0.3305785123966942, 'recall': 0.33613445378151263, 'f1': 0.33333333333333337, 'number': 119} | {'precision': 0.7845601436265709, 'recall': 0.8206572769953052, 'f1': 0.8022028453419, 'number': 1065} | 0.7248 | 0.7822 | 0.7524 | 0.8045 |
| 0.2645 | 15.0 | 150 | 0.6832 | {'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809} | {'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119} | {'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065} | 0.7259 | 0.7812 | 0.7525 | 0.8047 |
Framework versions
- Transformers 4.21.3
- Pytorch 1.12.1+cu102
- Datasets 2.4.0
- Tokenizers 0.12.1
