pygmalion-13b-ggml

Model description

Warning: THIS model is NOT suitable for use by minors. The model will output X-rated content.

Quantized from the decoded pygmalion-13b xor format. https://huggingface.co/PygmalionAI/pygmalion-13b

Quantization Information

GGML quantized with: https://github.com/ggerganov/llama.cpp

Note: These are the newest/bumped quantized version as of May 20th 2023.

The old quants can be found here: https://huggingface.co/notstoic/pygmalion-13b-ggml/tree/456c8100c2041fa975defe846220e3a23a614a6d

python3 convert.py models/pygmalion-13b/

./quantize ./models/pygmalion-13b/ggml-model-f16.bin ./models/pygmalion-13b-ggml-q4_0.bin q4_0
./quantize ./models/pygmalion-13b/ggml-model-f16.bin ./models/pygmalion-13b-ggml-q4_1.bin q4_1
./quantize ./models/pygmalion-13b/ggml-model-f16.bin ./models/pygmalion-13b-ggml-q5_0.bin q5_0
./quantize ./models/pygmalion-13b/ggml-model-f16.bin ./models/pygmalion-13b-ggml-q5_1.bin q5_1
./quantize ./models/pygmalion-13b/ggml-model-f16.bin ./models/pygmalion-13b-ggml-q8_0.bin q8_0