OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

Please see this link for detailed information.