KoLLaVA : Korean Large Language and Vision Assistant (feat. LLaVA)

This model is a large multimodal model (LMM) that combines the LLM(LLaMA-2-7b-ko) with visual encoder of CLIP(ViT-14), trained on Korean visual-instruction dataset using QLoRA.

Detail codes are available at KoLLaVA github repository

• Training hyperparameters
• learning rate : 2e-4
• train_batch_size: 16
• distributed_type: multi-GPU (RTX3090 24G)
• num_devices: 4
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• total_eval_batch_size: 4
• lr_scheduler_type: cosine
• num_epochs: 1
• lora_enable: True
• bits: 4

Model License: cc-by-nc-4.0