hf-causal (pretrained=ccore/opt-125-smart-test), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| openbookqa | 0 | acc | 0.1560 | ± | 0.0162 |
| acc_norm | 0.3420 | ± | 0.0212 | ||
| piqa | 0 | acc | 0.6197 | ± | 0.0113 |
| acc_norm | 0.6023 | ± | 0.0114 | ||
| prompt format: |
[INSTRUCTION] what's the capital of Brasil? ?
[RESPONSE] The capital of Brazil is Brasilia
datasets: OpenOrca, Wizard dataset, custom papers data .
