PEFT Model Fine-tuned on UAE QA Pairs
This repository contains a fine-tuned model based on the PEFT framework for question answering tasks. The model has been trained on a dataset of question and answer pairs related to the UAE.
Installation
Before using the model, make sure to install the necessary packages:
pip install transformers
pip install torch torchvision
pip install peft
Usage
The model can be used for generating responses to prompts. Here is an example:
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
peft_model_id = "insomeniaT/falcon-7b-uae-qapairs-67"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, trust_remote_code=True)
model = PeftModel.from_pretrained(model, peft_model_id)
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
tokenizer.pad_token = tokenizer.eos_token
text = "### Human: What is the minimum requirement for the UAE's GCC residency?? ### Assistant: "
device = "cuda:0"
inputs = tokenizer(text, return_tensors="pt")
inputs.to(device)
model.to(device)
outputs = model.generate(input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"], max_new_tokens=300, pad_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)