cross_encoder-msmarco-distilbert-word2vec256k

#cross_encoder-msmarco-word2vec256k

This CrossEncoder was trained with MarginMSE loss from the nicoladecao/msmarco-word2vec256000-distilbert-base-uncased checkpoint. Word embedding matrix has been frozen during training.

You can load the model with sentence-transformers:

from sentence_transformers import CrossEncoder
from torch import nn
model = CrossEncoder(model_name, default_activation_function=nn.Identity())

Performance on TREC Deep Learning (nDCG@10):

• TREC-DL 19: 72.49
• TREC-DL 20: 72.71