Euryale-1.3-L2-70B-2.18bpw-h6-exl2

作者 waldie · 下载 0 · 点赞 1 · 更新 2023-11-04

quant of Sao10K's Euryale-1.3-L2-70B

fits into 24gb at 8k context lol

python3 convert.py \
    -i /input/Sao10K_Euryale-1.3-L2-70B/ \
    -c /input/wikitext/0000.parquet \
    -o /output/temp/ \
    -cf /output/2.18bpw/ \
    -b 2.18 \
    -hb 6