This is model is a simple extension to mosaic/mpt-30b. The alibi positional embedding is manually interpolated by 2x hence giving 2x the original context length.