To run this model, please clone: https://github.com/ggerganov/llama.cpp and install using Make (see instruction in Github)

Then, download the model and place it in the models folder and run:

Linux: ./main -m models/baize_lora_13b_q4.bin -n -1 -t 4 --ignore-eos --temp 0.4 --interactive-first --repeat_penalty 1.2 --instruct --color

Windows: main.exe -m models/baize_lora_13b_q4.bin -n -1 -t 4 --ignore-eos --temp 0.4 --interactive-first --repeat_penalty 1.2 --instruct --color

It should look something like this: