This is a finetuned version of RuRoBERTa-large for the task of linguistic acceptability classification on the RuCoLA benchmark.
The hyperparameters used for finetuning are as follows:
- 5 training epochs (with early stopping based on validation MCC)
- Peak learning rate: 1e-5, linear warmup for 10% of total training time
- Weight decay: 1e-4
- Batch size: 32
- Random seed: 5
- Optimizer: torch.optim.AdamW