Trained polyglot 5.8B with the QLORA method using the Chatbot_data_for_Korean dataset.
The hyper-parameters used for training are as follows.
- batch-size: 16
- max_steps: 3000
- Learning rate: 3e-4
- Lora r: 8
- Lora target modules: query_key_value
Prompt Template:
### 질문: {문장}
### 응답: {문장}
### 유형: {일반 또는 연애}