Local AI Developer Pushes Qwen 3.6 to Million-Token Milestone

An independent developer, operating under the username Jorlen, has achieved a significant milestone in local large language model (LLM) testing by utilizing over one million tokens across three separate sessions with the Qwen 3.6 35B model. The experiment, which leveraged the new Multi-token Prediction (MTP) architecture, reportedly delivered a 1.5x increase in tokens-per-second performance compared to previous benchmarks. This test underscores the accelerating pace of innovation in locally-hosted AI, moving complex model interaction further into the realm of consumer-grade hardware.

Technical Setup and Performance Breakthrough

The developer's project involved building a step-by-step, iterative Pygame application—a small mystery dungeon-style game—to stress-test the model's capabilities. Initially setting a context window between 100,000 and 200,000 tokens, they successfully raised it to 300,000. The model used was the Qwen3.6-35B-A3B-UD-Q5_K_S variant with MTP, running via a prototype version of the llama.cpp server within a Docker container on Ubuntu 24.04. The system was powered by an ASUS Radeon R9700 AI Pro graphics card with 32GB of VRAM, of which approximately 28.3GB was utilized during the high-context tests.

According to the developer's report, the use of Multi-token Prediction models represents a "100% game changer for local LLMs," primarily due to the substantial speed gains. The test was not just about raw token processing but also explored the practical limits of context window size within a multi-file software development project. The developer noted that even with a 300k context, performance remained effective, suggesting the possibility of pushing to 400k contexts with the appropriate model configuration.

Navigating the Open-Source Software Ecosystem

The experiment's success was contingent on a specific software stack, highlighting both the flexibility and the fragility of cutting-edge, open-source AI tooling. The developer had to utilize a custom Docker image (`havenoammo/llama:vulkan-server`) to access the MTP prototype, as it was not yet available in standard distributions. This reflects a common theme in advanced computing where enthusiasts often operate on the bleeding edge, assembling their own solutions from community-driven projects.

This reliance on community infrastructure echoes broader discussions in the tech world about platform independence and developer experience. Analysis from industry commentators, such as those on Hacker News, often points to the challenges when corporate priorities shift, potentially leaving niche or advanced user needs unaddressed. In this case, the local LLM community itself provides the essential tools, from model quantization to specialized server software, enabling such groundbreaking tests to occur outside major corporate labs.

The developer later switched from the 35B Mixture-of-Experts (MoE) model to the Qwen 3.6 27B non-MoE version after encountering stability issues deep into a 200k-token context session. This adjustment underscores the ongoing experimentation required to balance model size, context length, and system stability when pushing hardware to its limits.

Hardware and Driver Considerations for AI Workloads

The test was conducted on Ubuntu using the Vulkan API, a choice that comes with its own set of considerations. Performance on Linux, particularly with AMD's Radeon hardware, can be influenced by ongoing driver developments. While not directly cited as an issue in this test, the broader ecosystem notes that performance for Radeon cards on Vulkan and OpenGL can fluctuate. According to community support forums like the Ubuntu Community Hub, users occasionally report performance regressions following system or driver updates, which requires vigilant community support and troubleshooting.

This environment means that developers and researchers working at the frontier of local AI must be as adept at system administration and driver management as they are at prompt engineering. The choice of the Radeon R9700 AI Pro card, a recent entrant designed for AI workloads, signifies a growing hardware market catering specifically to this democratization of high-performance machine learning.

The Future of Democratized AI Development

The developer concluded their report with profound appreciation for the local LLM community, marveling at the progress made in just one year. The ability to run a 35-billion-parameter model with a 300,000-token context on a single consumer-grade GPU was nearly unthinkable in the recent past. This experiment is a concrete data point in the trend of powerful AI capabilities migrating from exclusive cloud data centers to individual workstations and hobbyist setups.

This shift has profound implications for privacy, cost, customization, and the pace of innovation. Developers are no longer solely dependent on API calls to large corporations; they can iterate, test, and deploy complex AI interactions entirely offline. The successful million-token test with the Qwen 3.6 model is a testament to this new era, proving that ambitious AI projects can begin on a desktop. The frontier of local AI continues to expand, driven by community collaboration and rapid iterations on models like Qwen 3.6.