bert-finetuned-uia
This model is a fine-tuned version of bert-base-cased on the None dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
Trained on 100,000 questions from the Natural Questions dataset where the short answer is present.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
Training results
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.2