OpenSourceTransformers-OST Project

OST-OpenSourceTransformers Github

Hello community

this model is only 1B but you can call it somehow an SOTA

this model can also run on 4 GB GPU RAM and know dialogs as well

Train Parametes

Usage Code



from transformers import AutoTokenizer, AutoModelForCausalLM
from IPython.display import clear_output
import textwrap


tokenizer = AutoTokenizer.from_pretrained("erfanzar/PGT-1B-2EP")

model = AutoModelForCausalLM.from_pretrained("erfanzar/PGT-1B-2EP",device_map='auto',load_in_8bit=True)


verify_text = lambda txt : '\n'.join([textwrap.fill(txt, width=110) for txt in txt.split('\n')])


def ppp(text:str):
  """
  pre processing prompt
  """
  return f"<|prompter|> {text} <|endoftext|><|assistant|>"

def generate(text,max_new_tokens:int=1024,use_ppp:bool=False,b_pair=False):
  text = ppp(text) if use_ppp else text
  
  for i in range(max_new_tokens):
    enc = tokenizer(text,return_tensors='pt',add_special_tokens=False)
    text_r = text
    enc = model.generate(enc.input_ids,max_new_tokens=1,pad_token_id=0)
    text = tokenizer.decode(enc[0],skip_special_tokens=False)
    text = text[:-4]+tokenizer.eos_token if text[-4:] == '\n\n\n\n' else text
  
    if text.endswith(tokenizer.eos_token) or text.endswith('\n\n\n\n\n'):
      yield text[len(text_r):] if b_pair else text
      break
    else:
      yield text[len(text_r):] if b_pair else text


for v in generate('what is a gpu',512,True):
  clear_output(wait=True)
  print(verify_text(v),end='')

Pythia-1B

Model Details

Pretrained Model

NOTE

The Pythia Suite is NOT intended for deployment. It is not in itself a product and cannot be used for human-facing interactions. For example, the model may generate harmful or offensive text...

and also remember that this model is not good enough for Persian, French, and Dutch at least for this version