Emerhyst-20B EXL2-5bpw

Model creator: Undi95 Original model: Emerhyst-20B

Description

• 5bpw per weight
• This is a test for the exllamav2 model

I converted the model using the convert.py script from the exllamav2 repo: https://github.com/turboderp/exllamav2

Its documentation: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md

I used the WikiText-2-v1 dataset for calibration: https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet