This is model is a simple extension to mosaic/mpt-30b. The alibi positional embedding is manually interpolated by 4x hence giving 4x the original context length.