Manticore-13b-Landmark-GPTQ

Manticore-13b-Landmark-GPTQ

Key Features

• Landmark Attention
• Large Context Size (~18k)
• 4-Bit Quantization

Composition

Manticore-13b-Landmark-GPTQ is a 4-bit quantization of:

• Manticore-13B-Landmark

Using Oobabooga

• Trust Remote Code - (Enabled)
• Add the bos_token to the beginning of prompts - (Disabled)
• Truncate the prompt up to this length - (Increased)

Landmark Training Code

See GitHub for the training code.