Update End token generation problem. The LLM here keeps generating output for long. Please wait for TheBloke's Quantization.

This repository hosts the Q4_K_M and Q5_K_M variants of the model named uukuguy/speechless-mistral-dolphin-orca-platypus-samantha-7b. These versions are compatible with the llama.cpp framework.

I specifically chose these two quantization methods based on my prior experience, as they have shown to retain a substantial portion of the original quality across various use cases.