J.O.S.I.E.: Just an Outstandingly Smart Intelligent Entity

This mdoel is for personal use, ONLY!!!

Fine-tuned from the Llama V2 Chat Model, JOSIE is specialized in function calling tasks. This project utilized a personalized dataset from glaiveai/glaive-function-calling to enhance the model's capability in understanding and executing function calls. A new model is in the pipeline, further fine-tuned to specialize in function calls for web searching, smart home control, file writing on the local machine, weather condition retrieval and summarization, unread email retrieval and summarization, email deletion, and interacting with MetaGPT. To get closer to my version of Tony Starks, J.A.R.V.I.S.

Model

• Base model: meta-llama/Llama-2-7b-chat-hf
• Fine-tuned model: JOSIE_7B_FC_v30

Fine-tuning

The model was trained with the following parameters:

• Lora attention dimension: 64
• LoRA scaling alpha: 16
• Dropout probability for LoRA layers: 0.1
• Bitsandbytes 4-bit precision: Enabled
• fp16: Enabled
• bf16: Disabled
• Learning rate: 2e-4 with the cosine scheduler type
• Optimizer: paged_adamw_32bit
• Warm-up steps: 100
• Training epochs: 200
• GPU: A100

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Isaak-Carter/JOSIE_FC_7B_v30"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Your code for interaction with JOSIE

license: apache-2.0 pipeline_tag: conversational