ColBERT NQ Checkpoint

This trained model is based on the ColBERT model, trained on the Natural Questions dataset.

Model Details

Model is based on ColBERT, which in turn is based around a BERT encoder. The model is trained for text retrieval using a contrastive loss; given a query there's a relevant and non relevant passages.

The corpus is based on Wikipeida.

Uses

Model can be used by the ColBERT codebase to initiate a retriever; one needs to build a vector index and then queries can be ran.

Evaluation

Evaluation results on NQ dev:

<table> <colgroup> <col class="org-right"> <col class="org-right"> <col class="org-right"> </colgroup> <thead> <tr> <th scope="col" class="org-right">NQ</th> <th scope="col" class="org-right">Recall</th> <th scope="col" class="org-right">MRR</th> </tr> </thead>

<tbody> <tr> <td class="org-right">10</td> <td class="org-right">71.1</td> <td class="org-right">52.0</td> </tr>

<tr> <td class="org-right">20</td> <td class="org-right">76.3</td> <td class="org-right">52.3</td> </tr>

<tr> <td class="org-right">50</td> <td class="org-right">80.4</td> <td class="org-right">52.5</td> </tr>

<tr> <td class="org-right">100</td> <td class="org-right">82.7</td> <td class="org-right">52.5</td> </tr> </tbody> </table>