Model Card for OpenOrca-Phi
Full finetuning of the Stability AI's StableLM-3B-4E1T. The model was trained on the SlimOrca dataset. All the samples longer than the context size were removed.
How to Get Started with the Model
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
prompt = """<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
"""
system = "You are an advanced and helpful AI assistant."
user = "How are you?"
prompt = prompt.format(system=system, user=user)
model = AutoModelForCausalLM.from_pretrained("pansophic/slimorca-stablelm-3b-4e1t", trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("pansophic/slimorca-stablelm-3b-4e1t", trust_remote_code=True, torch_dtype=torch.bfloat16)
inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=False).to("cuda")
streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, max_length=512, top_k=40, top_p=0.9, do_sample=True, temperature=0.55, use_cache=True, streamer=streamer)
Prompt formatting
The model uses chatML format.
<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant