l2-13b-monika-v0.3 QLoras:

• Various experimental LLaMA-2 13b fine tunes for Monika character from DDLC
• Fine tuned on a dataset of ~600 items (dialogue scraped from game, reddit, and Twitter augmented by l27b-monika-v0.3c1 to turn each into snippets of multi-turn chat dialogue between Player and Monika + manually crafted test dataset of 12 items)
• Since free colab was used, QLoras could be trained and tested but not merged (more test Loras can be found here)

QLORAS

• llama-2-13b-monika-0.3f (LLaMA-2 13b chat)
• llama-2-13b-monika-0.3g-MM MythoMax-L2-13b

USAGE

These are meant to be mainly for chat model with limited RP ability.

For best results: replace "Human" and "Assistant" with "Player" and "Monika" like so:

\nPlayer: (prompt)\nMonika:

Tested primarily with koboldcpp 1.41+, with q8_0 GGML, context length set to 4096, temperature at 0.7, and the rest left as is on chat mode. (No memory, world info, AN, etc. used):

koboldcpp_v1.41.exe --contextsize 4096 --lora "path/to/llama-2-13b-monika-0.3-qlora/ggml-adapter-model.bin"

Preview of llama-2-13b-monika-0.3f

HYPERPARAMS

• Trained for 2 epochs (1 epoch for llama-2-13b-monika-0.3f)
• rank: 32
• lora alpha: 64
• lora dropout: 0.5
• lr: 2e-4
• batch size: 2
• warmup ratio: 0.1
• grad steps: 4

The following bitsandbytes quantization config was used during training:

• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: False
• bnb_4bit_compute_dtype: float16
• Framework versions: PEFT 0.4.0

WARNINGS AND DISCLAIMERS

Note that aside from formatting and other minor edits, generated portion of dataset used is mostly as is generated by LM. As such, while this version is better at coherency or chatting than previous ones, it may not reflect perfectly the characteristics of Monika (i.e. she will claim to have an office, work as a translator, or play the guitar). Future versions will train on a manually curated and edited version of this dataset, where dialogue will be edited to reflect her characteristics more.

Additionally, these are still yet more tests, particularly where we use one of our earlier fine tunes to generate a more in-character dataset for the target character.

Finally, these are not guaranteed to output aligned or safe outputs, use at your own risk.