Xiaomi MiMo: A Trillion Parameters, With Four Percent Active

Xiaomi has unveiled the MiMo family of language models, spanning a wide range of configurations from compact 7B models to a massive Mixture of Experts (MoE) solution featuring one trillion parameters. A key highlight of the announcement is the high computational efficiency: despite the massive total parameter count, only about four percent are active during text generation. This approach significantly accelerates model performance and reduces hardware requirements without compromising output quality. Experts note that the use of sparse architectures is becoming a new industry standard, allowing companies to scale AI capabilities to trillion-parameter levels while maintaining acceptable inference speeds. This development underscores Xiaomi's commitment to strengthening its position in generative AI by offering flexible tools for diverse applications.