4.65 bit quantization of Uni-TianYan (70B) (https://huggingface.co/uni-tianyan/Uni-TianYan), using exllamav2.

The model is on sharded files, to prevent massive system RAM usage when compiling large output files. It loads as it should for inference.

Similar to 32g on size but better ppl.