<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
lilt-en-funsd
This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on the funsd-layoutlmv3 dataset. It achieves the following results on the evaluation set:
- Loss: 1.7029
- Answer: {'precision': 0.8619489559164734, 'recall': 0.9094247246022031, 'f1': 0.8850506253722453, 'number': 817}
- Header: {'precision': 0.648936170212766, 'recall': 0.5126050420168067, 'f1': 0.5727699530516431, 'number': 119}
- Question: {'precision': 0.8972046889089269, 'recall': 0.9238625812441968, 'f1': 0.9103385178408051, 'number': 1077}
- Overall Precision: 0.8712
- Overall Recall: 0.8937
- Overall F1: 0.8823
- Overall Accuracy: 0.8004
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2500
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 0.4178 | 10.53 | 200 | 1.1592 | {'precision': 0.8244444444444444, 'recall': 0.9082007343941249, 'f1': 0.8642981945253349, 'number': 817} | {'precision': 0.62, 'recall': 0.5210084033613446, 'f1': 0.5662100456621004, 'number': 119} | {'precision': 0.874561403508772, 'recall': 0.9257195914577531, 'f1': 0.8994136220117277, 'number': 1077} | 0.8416 | 0.8947 | 0.8673 | 0.7757 |
| 0.0458 | 21.05 | 400 | 1.3058 | {'precision': 0.8473988439306358, 'recall': 0.8971848225214198, 'f1': 0.8715814506539833, 'number': 817} | {'precision': 0.5462184873949579, 'recall': 0.5462184873949579, 'f1': 0.5462184873949579, 'number': 119} | {'precision': 0.8808243727598566, 'recall': 0.9127205199628597, 'f1': 0.8964888280893752, 'number': 1077} | 0.8481 | 0.8847 | 0.8660 | 0.7946 |
| 0.0135 | 31.58 | 600 | 1.6170 | {'precision': 0.855188141391106, 'recall': 0.9179926560587516, 'f1': 0.885478158205431, 'number': 817} | {'precision': 0.5737704918032787, 'recall': 0.5882352941176471, 'f1': 0.5809128630705394, 'number': 119} | {'precision': 0.8915223336371924, 'recall': 0.9080779944289693, 'f1': 0.8997240110395583, 'number': 1077} | 0.8578 | 0.8932 | 0.8752 | 0.7978 |
| 0.0081 | 42.11 | 800 | 1.3449 | {'precision': 0.8650602409638555, 'recall': 0.8788249694002448, 'f1': 0.8718882817243473, 'number': 817} | {'precision': 0.6476190476190476, 'recall': 0.5714285714285714, 'f1': 0.6071428571428571, 'number': 119} | {'precision': 0.8675324675324675, 'recall': 0.9303621169916435, 'f1': 0.8978494623655914, 'number': 1077} | 0.8555 | 0.8882 | 0.8716 | 0.8073 |
| 0.0048 | 52.63 | 1000 | 1.4197 | {'precision': 0.8709288299155609, 'recall': 0.8837209302325582, 'f1': 0.8772782503037667, 'number': 817} | {'precision': 0.5925925925925926, 'recall': 0.5378151260504201, 'f1': 0.5638766519823789, 'number': 119} | {'precision': 0.8701298701298701, 'recall': 0.9331476323119777, 'f1': 0.9005376344086022, 'number': 1077} | 0.8561 | 0.8897 | 0.8726 | 0.8155 |
| 0.0021 | 63.16 | 1200 | 1.5708 | {'precision': 0.8366666666666667, 'recall': 0.9216646266829865, 'f1': 0.8771112405358183, 'number': 817} | {'precision': 0.6590909090909091, 'recall': 0.48739495798319327, 'f1': 0.5603864734299517, 'number': 119} | {'precision': 0.8841628959276018, 'recall': 0.9071494893221913, 'f1': 0.8955087076076994, 'number': 1077} | 0.8543 | 0.8882 | 0.8709 | 0.8014 |
| 0.0019 | 73.68 | 1400 | 1.5995 | {'precision': 0.8174946004319654, 'recall': 0.9265605875152999, 'f1': 0.8686173264486519, 'number': 817} | {'precision': 0.5892857142857143, 'recall': 0.5546218487394958, 'f1': 0.5714285714285715, 'number': 119} | {'precision': 0.8880800727934486, 'recall': 0.9062209842154132, 'f1': 0.8970588235294117, 'number': 1077} | 0.8418 | 0.8937 | 0.8670 | 0.8108 |
| 0.0015 | 84.21 | 1600 | 1.6443 | {'precision': 0.8770883054892601, 'recall': 0.8996328029375765, 'f1': 0.8882175226586103, 'number': 817} | {'precision': 0.5945945945945946, 'recall': 0.5546218487394958, 'f1': 0.5739130434782609, 'number': 119} | {'precision': 0.8947368421052632, 'recall': 0.9155060352831941, 'f1': 0.9050022946305646, 'number': 1077} | 0.8713 | 0.8877 | 0.8794 | 0.8002 |
| 0.0011 | 94.74 | 1800 | 1.6845 | {'precision': 0.8622685185185185, 'recall': 0.9118727050183598, 'f1': 0.8863771564544913, 'number': 817} | {'precision': 0.6444444444444445, 'recall': 0.48739495798319327, 'f1': 0.5550239234449761, 'number': 119} | {'precision': 0.8973660308810173, 'recall': 0.9173630454967502, 'f1': 0.9072543617998163, 'number': 1077} | 0.8715 | 0.8897 | 0.8805 | 0.7996 |
| 0.0003 | 105.26 | 2000 | 1.6754 | {'precision': 0.8582949308755761, 'recall': 0.9118727050183598, 'f1': 0.8842729970326408, 'number': 817} | {'precision': 0.5961538461538461, 'recall': 0.5210084033613446, 'f1': 0.5560538116591929, 'number': 119} | {'precision': 0.9006381039197813, 'recall': 0.9173630454967502, 'f1': 0.9089236430542779, 'number': 1077} | 0.8676 | 0.8917 | 0.8795 | 0.8034 |
| 0.0003 | 115.79 | 2200 | 1.6803 | {'precision': 0.8733727810650888, 'recall': 0.9033047735618115, 'f1': 0.888086642599278, 'number': 817} | {'precision': 0.6116504854368932, 'recall': 0.5294117647058824, 'f1': 0.5675675675675675, 'number': 119} | {'precision': 0.8938294010889292, 'recall': 0.914577530176416, 'f1': 0.9040844424047729, 'number': 1077} | 0.8712 | 0.8872 | 0.8792 | 0.8014 |
| 0.0004 | 126.32 | 2400 | 1.7029 | {'precision': 0.8619489559164734, 'recall': 0.9094247246022031, 'f1': 0.8850506253722453, 'number': 817} | {'precision': 0.648936170212766, 'recall': 0.5126050420168067, 'f1': 0.5727699530516431, 'number': 119} | {'precision': 0.8972046889089269, 'recall': 0.9238625812441968, 'f1': 0.9103385178408051, 'number': 1077} | 0.8712 | 0.8937 | 0.8823 | 0.8004 |
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
