This model is a merge of 80% starchatplus_beta and 20% wizardcoder.

It is intended as a research tool into merging and routing of experts.

"multiple-py": { "pass@1": 0.36645962732919257 }

hf-causal (pretrained=Multi-Domain-Expert-Layers/scorpius_16b,dtype=bfloat16), limit: 0.1, provide_description: False, num_fewshot: 0, batch_size: None

Task Version Metric Value Stderr
arc_challenge 0 acc 0.4103 ± 0.0457
acc_norm 0.4103 ± 0.0457
arc_easy 0 acc 0.7350 ± 0.0410
acc_norm 0.6923 ± 0.0429
hellaswag 0 acc 0.5812 ± 0.0458
acc_norm 0.7778 ± 0.0386