wandb run: https://wandb.ai/eleutherai/pythia-rlhf/runs/e0drjcsz?workspace=user-yongzx

Model Evals:

Task Version Filter Metric Value Stderr
arc_challenge Yaml none acc 0.1877 ± 0.0114
none acc_norm 0.2372 ± 0.0124
arc_easy Yaml none acc 0.4390 ± 0.0102
none acc_norm 0.4082 ± 0.0101
logiqa Yaml none acc 0.1889 ± 0.0154
none acc_norm 0.2473 ± 0.0169
piqa Yaml none acc 0.6213 ± 0.0113
none acc_norm 0.6279 ± 0.0113
sciq Yaml none acc 0.7230 ± 0.0142
none acc_norm 0.6840 ± 0.0147
winogrande Yaml none acc 0.5162 ± 0.0140
wsc Yaml none acc 0.3654 ± 0.0474
lambada_openai Yaml none perplexity 58.9478 ± 2.7662
none acc 0.2602 ± 0.0061