Model Card for Model ID
This is a chat bot for Dbahn
Model Details
Model Description
- Developed by: OTSI https://otsi-global.com/
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: llama2-7b
- Language(s) (NLP): German
- License: Meta
- Finetuned from model [optional]: flozi00/Llama-2-7b-german-assistant-v3
Model Sources [optional]
Training Details:
num_train_epochs = 1 max_steps = -1 bf16 = False fp16 = True per_device_train_batch_size = 4 per_device_eval_batch_size = 4 gradient_accumulation_steps = 1 max_grad_norm = 0.3 optim = "paged_adamw_32bit" learning_rate = 1e-5 lr_scheduler_type = "constant" warmup_ratio = 0.03 weight_decay = 0.001 group_by_length = True gradient_checkpointing = True save_steps = 50 logging_steps = 10