#cross_encoder-msmarco-distilbert-word2vec256k-MLM_400k
This CrossEncoder was trained with MarginMSE loss from the vocab-transformers/msmarco-distilbert-word2vec256k-MLM_400k checkpoint. Word embedding matrix has been frozen during training.
You can load the model with sentence-transformers:
from sentence_transformers import CrossEncoder
from torch import nn
model = CrossEncoder(model_name, default_activation_function=nn.Identity())
Performance on TREC Deep Learning (nDCG@10):
- TREC-DL 19: 72.62
- TREC-DL 20: 73.22