JPNsensei-V2

Model Application

This is a QA model specifically tailored for answering questions about learning Japanese in English.

Model Description

It is built upon the japanese-stablelm-instruct-gamma-7b base model. To optimize the quality of responses, please adhere to the specified prompt format.

Uses

Ensure you are using Transformers 4.34.0 or newer.

model = AutoModelForCausalLM.from_pretrained(
    "kanxxyc/JPNsensei-V2", low_cpu_mem_usage=True,
    return_dict=True,torch_dtype=torch.bfloat16,
    device_map= {"": 0}
)
tokenizer = AutoTokenizer.from_pretrained("kanxxyc/JPNsensei-V2")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

text_generation_pipeline = transformers.pipeline(
    model=model,
    tokenizer=tokenizer,
    task="text-generation",
    pad_token_id=tokenizer.eos_token_id, 
    temperature=0.2,
    do_sample=True,
    repetition_penalty=1.1,
    max_new_tokens=1024,

)

mistral_llm = HuggingFacePipeline(pipeline=text_generation_pipeline)
prompt_template = """
### Instruction: Given a title and a question, your task is to generate an appropriate answer based on the context provided, using simple English to explain any Japanese language-related queries.

### title:
{title}

### question:
{question}

### answer:
"""
prompt = PromptTemplate(
    input_variables=["title", "question"],
    template=prompt_template,
)


llm_chain = LLMChain(llm=mistral_llm, prompt=prompt)

print("Chatbot: Hello! How can I assist you today?")
while True:
    title_input = input("Chatbot: What's the topic or context of your question? ")
    
    if title_input.lower() in ["exit", "quit", "bye"]:
        print("Chatbot: Goodbye!")
        break

    question_input = input("Chatbot: Please specify your question: ")

    if question_input.lower() in ["exit", "quit", "bye"]:
        print("Chatbot: Goodbye!")
        break

    response_text = llm_chain({
        "title": title_input,
        "question": question_input
    })
    print("Chatbot:", response_text['text'].replace("### answer:", "").strip())

Training Data

Data from Stack Exchange Japanese site (https://japanese.stackexchange.com/)

Evaluation

To be updated.