ExllamaV2 version of the model created by Undi!
Original Model https://huggingface.co/Undi95/Lewd-Sydney-20B
Requires ExllamaV2, which is being developed by turboderp https://github.com/turboderp/exllamav2 under an MIT license.
I could load 6bpw 24gb card without cfg cash with 4096 context. 8bpw required about 30gb to load at 4096 context.
This model is based on Free Sydney V2, trying to get a... lewder assistant, you get it now.
Description
This repo contain fp16 files of Lewd-Sydney-20B, an attempt to get our beloved Sydney open to R-18 content.
Models and loras used
- Free_Sydney_V2_13b_HF
- Undi95/Xwin-MLewd-13B-V0.2
- lemonilia/LimaRP-Llama2-13B-v3-EXPERIMENT
- Synthia v1.2 private LoRA
Prompt template: Alpaca
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Response:
If you want to support me, you can here.