Model Card: Llama-7b with LoRA Fine-tuning on QACR data

Model Overview

• Model Name: Llama-7b
• Model Architecture: Transformer-based Language Model
• Fine-tuning Method: LoRA
• Training Datasets:
  • Educational Question Generation Dataset (described in the dataset chart)
  • Alpaca GPT-4 french dataset (chat instruction task)
  • Dolly_fr dataset (chat instruction task)

Model Details

• Base Model: decapoda-research/llama-7b-hf
• Fine-tuning Approach: LoRA fine-tuning method, which combines pre-training on a large corpus with additional task-specific fine-tuning.
• Training Objective: The model is trained to generate relevant and useful questions based on educational texts and to handle chat instruction tasks from the Alpaca GPT-4 and Dolly datasets.
• Training Procedure: The base Llama-7b model is first pretrained on a large corpus to learn general language patterns and representations. It is then fine-tuned using a combination of the aforementioned datasets to specialize in educational question generation and chat instruction tasks.

Intended Use

• Primary Task: Question generation for educational purposes and chat instruction tasks.
• Potential Use Cases:
  • Automated question generation for educational platforms and tutoring systems.
  • Chat-based instruction and assistance in various domains.