l2-7b-natsuki-ddlc-v0.1:
- Experimental LLaMA-2 7b chat fine-tuned for Natsuki character from DDLC
- Fine-tuned on a dataset of ~800 items (dialogue scraped from game augmented by MythoMax-l2-13b to turn each into snippets of multi-turn chat dialogue between Player and Natsuki)
- GGMLs, GGUFs
- QLoras (hf and GGML)
USAGE
This is meant to be mainly a chat model with limited RP ability.
For best results: replace "Human" and "Assistant" with "Player" and "Natsuki" like so:
\nPlayer: (prompt)\Natsuki:
HYPERPARAMS
- Trained for 2 epochs
- rank: 32
- lora alpha: 64
- lora dropout: 0.5
- lr: 2e-4
- batch size: 2
- warmup ratio: 0.1
- grad steps: 4
WARNINGS AND DISCLAIMERS
Note that aside from formatting and other minor edits, generated portion of dataset used is mostly as is generated by LM. As such, while this version is better at coherency or chatting than previous ones, it may not reflect perfectly the characteristics of Natsuki (i.e. she may not act as Tsundere or might even agree that Manga is not literature). Next version will train on a manually curated and edited version of this dataset, where dialogue will be edited to reflect her characteristics more.
Other tests to come (i.e. fine tuning on other base models, like Airoboros or Kimiko-based model).
Finally, this model is not guaranteed to output aligned or safe outputs, use at your own risk.