Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

A new project titled Strata has emerged, enabling users to run the massive 125B parameter Qwen 3.8 Flash Next model on consumer-grade hardware, specifically the NVIDIA RTX 4090. The project claims to achieve inference speeds of 100 tokens per second, a significant milestone for local large language model deployment. By leveraging advanced optimization techniques, Strata bridges the gap between high-end enterprise infrastructure and personal computing setups. This development allows researchers and developers to experiment with state-of-the-art AI models without the need for expensive, multi-GPU server clusters. The repository, hosted on GitHub, provides the necessary tools and documentation for users to implement this setup locally. As local LLM performance continues to improve, this breakthrough highlights the rapid pace of optimization in the AI field, making powerful generative tools increasingly accessible to individual enthusiasts and independent developers working with limited hardware resources.