Model Card for OpenOrca-Phi

Full finetuning of the Stability AI's StableLM-3B-4E1T. The model was trained on the SlimOrca dataset. All the samples longer than the context size were removed.

<div style="display: flex; justify-content: center;"> <img src="https://cdn-uploads.huggingface.co/production/uploads/6501bfe0493fd9c8c2e32402/i2fc9OApv6_BgRCKFKg4T.png" alt="orcaslim-stablelm-3b" width="50%" style="display: block; margin: 0 auto;"> </div>

How to Get Started with the Model

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

prompt = """<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
"""

system = "You are an advanced and helpful AI assistant."
user = "How are you?"

prompt = prompt.format(system=system, user=user)

model = AutoModelForCausalLM.from_pretrained("pansophic/slimorca-stablelm-3b-4e1t", trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("pansophic/slimorca-stablelm-3b-4e1t", trust_remote_code=True, torch_dtype=torch.bfloat16)

inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=False).to("cuda")

streamer = TextStreamer(tokenizer)

_ = model.generate(**inputs, max_length=512, top_k=40, top_p=0.9, do_sample=True, temperature=0.55, use_cache=True, streamer=streamer)

Prompt formatting

The model uses chatML format.

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant