Wizard-Vicuna-13B-Uncensored-SpQR
Overview
This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF
Quantization Specifications
- Quantization: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
- Outliers: Threshold set at 0.2.
- Permutation Order:
act_order. - Dampening: Set at 1e0.
- Sampling: 128 samples.
- Logging: Via Weights & Biases.
Evaluation Metrics
The following perplexity scores were obtained on various datasets:
| Dataset | Perplexity |
|---|---|
| c4 | 7.354 |
| wikitext2 | 5.685 |
| ptb | 20.822 |