GPT4-x-AlpacaDente2-30b-4bit

This is a 4bit quant of https://huggingface.co/Aeala/GPT4-x-AlpacaDente2-30b

My secret sauce:

• Using comit 3c16fd9 of 0cc4m's GPTQ fork
• Using PTB as the calibration dataset
• Act-order, True-sequential, percdamp 0.1 (the default percdamp is 0.01)
• No groupsize
• Will run with CUDA, does not need triton.
• Quant completed on a 'Premium GPU' and 'High Memory' Google Colab.

Benchmark results