GermanGPT_dolly_lora_1b5

This is a LoRA for databricks dolly dataset, curated and translated to German by argilla applied to the model of malteos: bloom-1b5-clp-german.

It was trained on the dolly dataset for 3.43 epochs with a final training loss of 2.33.

Here is an example on how to use it.

Setup:

from transformers import AutoModelForCausalLM,AutoTokenizer,
from peft import PeftModel, PeftConfig
peft_model_id = "aari1995/GermanGPT_dolly_lora_1b5"
peft_config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(peft_config.base_model_name_or_path,device_map="auto",load_in_8bit=True,return_dict=True)
model = PeftModel.from_pretrained(model, peft_model_id)
tokenizer = AutoTokenizer.from_pretrained("malteos/bloom-6b4-clp-german-oasst-v0.1")

Usage with example format:

def format_text(text):
    return f"<|prompter|>{text}<|endoftext|><|assistant|>"
    encoded_input = tokenizer(text, return_tensors='pt')
    out = tokenizer.batch_decode(model.generate(**encoded_input,max_length=100,repetition_penalty=1.1))
    return out

Example:

text = "Was soll ich heute kochen um mich gesund zu ernähren?"
batch = tokenizer(format_text(text), return_tensors='pt')

with torch.cuda.amp.autocast():
  output_tokens = model.generate(**batch, max_new_tokens=45)

print('\n\n', tokenizer.decode(output_tokens[0]))
#Generated Text: Es gibt viele gesunde Lebensmittel, die du essen kannst. Du kannst zum Beispiel Obst und Gemüse essen, das reich an Vitaminen und Mineralstoffen ist. Du kannst auch Vollkornprodukte essen, die reich an Ballaststoffen sind. Du kannst