Manticore-13b-Landmark

Key Features

• Landmark Attention
• Large Context Size (~18k)

Composition

Manticore-13b-Landmark is a blend of:

• Manticore-13B
• Manticore-13B-Landmark-QLoRA

Using Oobabooga

• Trust Remote Code - (Enabled)
• Add the bos_token to the beginning of prompts - (Disabled)
• Truncate the prompt up to this length - (Increased)

Landmark Training Code

See GitHub for the training code.