<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6888
- Answer: {'precision': 0.7152391546162402, 'recall': 0.7948084054388134, 'f1': 0.752927400468384, 'number': 809}
- Header: {'precision': 0.32592592592592595, 'recall': 0.3697478991596639, 'f1': 0.3464566929133859, 'number': 119}
- Question: {'precision': 0.7731239092495636, 'recall': 0.831924882629108, 'f1': 0.8014473089099955, 'number': 1065}
- Overall Precision: 0.7216
- Overall Recall: 0.7893
- Overall F1: 0.7539
- Overall Accuracy: 0.8064
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.7613 | 1.0 | 10 | 1.5881 | {'precision': 0.032716927453769556, 'recall': 0.02843016069221261, 'f1': 0.030423280423280425, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2751159196290572, 'recall': 0.1671361502347418, 'f1': 0.20794392523364488, 'number': 1065} | 0.1489 | 0.1009 | 0.1203 | 0.3664 |
| 1.4369 | 2.0 | 20 | 1.2249 | {'precision': 0.16204986149584488, 'recall': 0.1446229913473424, 'f1': 0.15284128020901372, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.46102819237147596, 'recall': 0.5220657276995305, 'f1': 0.4896521356230735, 'number': 1065} | 0.3491 | 0.3377 | 0.3433 | 0.5911 |
| 1.0517 | 3.0 | 30 | 0.9409 | {'precision': 0.49002493765586036, 'recall': 0.4857849196538937, 'f1': 0.4878957169459963, 'number': 809} | {'precision': 0.03333333333333333, 'recall': 0.008403361344537815, 'f1': 0.013422818791946308, 'number': 119} | {'precision': 0.6160409556313993, 'recall': 0.6779342723004694, 'f1': 0.6455073759499329, 'number': 1065} | 0.5569 | 0.5600 | 0.5584 | 0.7085 |
| 0.8128 | 4.0 | 40 | 0.7827 | {'precision': 0.6324152542372882, 'recall': 0.7379480840543882, 'f1': 0.6811180832857958, 'number': 809} | {'precision': 0.14084507042253522, 'recall': 0.08403361344537816, 'f1': 0.10526315789473685, 'number': 119} | {'precision': 0.6759825327510917, 'recall': 0.7267605633802817, 'f1': 0.7004524886877828, 'number': 1065} | 0.6394 | 0.6929 | 0.6651 | 0.7571 |
| 0.6648 | 5.0 | 50 | 0.7231 | {'precision': 0.6456521739130435, 'recall': 0.7342398022249691, 'f1': 0.6871023713128976, 'number': 809} | {'precision': 0.24050632911392406, 'recall': 0.15966386554621848, 'f1': 0.1919191919191919, 'number': 119} | {'precision': 0.695364238410596, 'recall': 0.7887323943661971, 'f1': 0.7391113066432027, 'number': 1065} | 0.6584 | 0.7291 | 0.6919 | 0.7700 |
| 0.5437 | 6.0 | 60 | 0.6741 | {'precision': 0.6892039258451472, 'recall': 0.7812113720642769, 'f1': 0.7323290845886442, 'number': 809} | {'precision': 0.2376237623762376, 'recall': 0.20168067226890757, 'f1': 0.2181818181818182, 'number': 119} | {'precision': 0.6918238993710691, 'recall': 0.8262910798122066, 'f1': 0.7531022678647838, 'number': 1065} | 0.6707 | 0.7707 | 0.7173 | 0.7898 |
| 0.4719 | 7.0 | 70 | 0.6655 | {'precision': 0.7017738359201774, 'recall': 0.7824474660074165, 'f1': 0.7399181765049679, 'number': 809} | {'precision': 0.30927835051546393, 'recall': 0.25210084033613445, 'f1': 0.2777777777777778, 'number': 119} | {'precision': 0.7549956559513467, 'recall': 0.815962441314554, 'f1': 0.7842960288808664, 'number': 1065} | 0.7126 | 0.7687 | 0.7396 | 0.7963 |
| 0.4287 | 8.0 | 80 | 0.6544 | {'precision': 0.701212789415656, 'recall': 0.7861557478368356, 'f1': 0.7412587412587412, 'number': 809} | {'precision': 0.3153153153153153, 'recall': 0.29411764705882354, 'f1': 0.30434782608695654, 'number': 119} | {'precision': 0.7593073593073593, 'recall': 0.8234741784037559, 'f1': 0.7900900900900901, 'number': 1065} | 0.7124 | 0.7767 | 0.7432 | 0.8015 |
| 0.3842 | 9.0 | 90 | 0.6613 | {'precision': 0.7128378378378378, 'recall': 0.7824474660074165, 'f1': 0.7460223924572775, 'number': 809} | {'precision': 0.3162393162393162, 'recall': 0.31092436974789917, 'f1': 0.3135593220338983, 'number': 119} | {'precision': 0.7614917606244579, 'recall': 0.8244131455399061, 'f1': 0.7917042380522993, 'number': 1065} | 0.7173 | 0.7767 | 0.7458 | 0.8046 |
| 0.344 | 10.0 | 100 | 0.6669 | {'precision': 0.7030567685589519, 'recall': 0.796044499381953, 'f1': 0.7466666666666666, 'number': 809} | {'precision': 0.31851851851851853, 'recall': 0.36134453781512604, 'f1': 0.33858267716535434, 'number': 119} | {'precision': 0.7571801566579635, 'recall': 0.8169014084507042, 'f1': 0.7859078590785908, 'number': 1065} | 0.7077 | 0.7812 | 0.7427 | 0.8048 |
| 0.305 | 11.0 | 110 | 0.6761 | {'precision': 0.7134894091415831, 'recall': 0.7911001236093943, 'f1': 0.7502930832356389, 'number': 809} | {'precision': 0.3391304347826087, 'recall': 0.3277310924369748, 'f1': 0.3333333333333333, 'number': 119} | {'precision': 0.781387181738367, 'recall': 0.8356807511737089, 'f1': 0.8076225045372051, 'number': 1065} | 0.7294 | 0.7873 | 0.7572 | 0.8061 |
| 0.2952 | 12.0 | 120 | 0.6823 | {'precision': 0.7130144605116796, 'recall': 0.792336217552534, 'f1': 0.7505854800936768, 'number': 809} | {'precision': 0.3203125, 'recall': 0.3445378151260504, 'f1': 0.33198380566801616, 'number': 119} | {'precision': 0.7775831873905429, 'recall': 0.8338028169014085, 'f1': 0.8047122791119167, 'number': 1065} | 0.7238 | 0.7878 | 0.7544 | 0.8066 |
| 0.2746 | 13.0 | 130 | 0.6829 | {'precision': 0.7212189616252822, 'recall': 0.7898640296662547, 'f1': 0.7539823008849559, 'number': 809} | {'precision': 0.34959349593495936, 'recall': 0.36134453781512604, 'f1': 0.35537190082644626, 'number': 119} | {'precision': 0.7833775419982316, 'recall': 0.831924882629108, 'f1': 0.8069216757741348, 'number': 1065} | 0.7327 | 0.7868 | 0.7588 | 0.8092 |
| 0.2632 | 14.0 | 140 | 0.6901 | {'precision': 0.7150776053215078, 'recall': 0.7972805933250927, 'f1': 0.7539450613676213, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.3865546218487395, 'f1': 0.357976653696498, 'number': 119} | {'precision': 0.7746478873239436, 'recall': 0.8262910798122066, 'f1': 0.7996365288505224, 'number': 1065} | 0.7220 | 0.7883 | 0.7537 | 0.8066 |
| 0.2575 | 15.0 | 150 | 0.6888 | {'precision': 0.7152391546162402, 'recall': 0.7948084054388134, 'f1': 0.752927400468384, 'number': 809} | {'precision': 0.32592592592592595, 'recall': 0.3697478991596639, 'f1': 0.3464566929133859, 'number': 119} | {'precision': 0.7731239092495636, 'recall': 0.831924882629108, 'f1': 0.8014473089099955, 'number': 1065} | 0.7216 | 0.7893 | 0.7539 | 0.8064 |
Framework versions
- Transformers 4.27.2
- Pytorch 1.13.1+cpu
- Datasets 2.11.0
- Tokenizers 0.11.0
