ZINC-t5

This model is a fine-tuned version of google/t5-v1_1-base on the sagawa/ZINC-canonicalized dataset. It achieves the following results on the evaluation set:

Model description

We trained t5 on SMILES from ZINC using the task of masked-language modeling (MLM). Compared to ZINC-t5, ZINC-t5-v2 uses a character-level tokenizer, and it was also trained on ZINC.

Intended uses & limitations

This model can be used for the prediction of molecules' properties, reactions, or interactions with proteins by changing the way of finetuning. As an example, We finetuned this model to predict products. The model is here, and you can use the demo here. Using its encoder, we trained a regression model to predict a reaction yield. You can use this demo here.

Training and evaluation data

We downloaded ZINC data and canonicalized them using RDKit. Then, we dropped duplicates. The total number of data is 22992522, and they were randomly split into train:validation=10:1.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

Training results

Training Loss Step Accuracy Validation Loss
0.2090 100000 0.9264 0.1860
0.1628 200000 0.9349 0.1613
0.1632 300000 0.9395 0.1467
0.1451 400000 0.9435 0.1345
0.1311 500000 0.9465 0.1261