MSO Expert https://msoexpert.com/ Local AI inference, MSO infrastructure, telecom architecture, and enterprise technology from John Pezzulli. en-us Sat, 12 Sep 2026 15:17:45 GMT 4.8 Billion Prompt Tokens Later: Somebody Else Put My Custom SGLang Runtime to Work https://msoexpert.com/articles/pennyroyal-real-workload-field-report/ https://msoexpert.com/articles/pennyroyal-real-workload-field-report/ Sat, 12 Sep 2026 15:17:45 GMT Nineteen days after publishing Pennyroyal, another user reported 4.8 billion prompt tokens, 55 million generated tokens, and more than a week of real work on a 300 W Max-Q. Then a second report arrived from the 27B configuration. Messing With SGLang and Qwen3.8: 40 Percent More Decode on One RTX PRO 6000 https://msoexpert.com/articles/qwen38-dflash2-rtx-pro-6000/ https://msoexpert.com/articles/qwen38-dflash2-rtx-pro-6000/ Tue, 25 Aug 2026 02:39:43 GMT Several speculative paths moved Qwen3.8 from 74.06 to 108.75 tokens per second on one RTX PRO 6000. A 29 percent long-prefill regression then sent the runtime through HiCache and NIXL so it could stop rebuilding prefixes it already knew. I Lowered My RTX PRO 6000’s Power Ceiling by 150 Watts. Local AI Barely Noticed. https://msoexpert.com/articles/rtx-pro-6000-600w-local-ai-efficiency/ https://msoexpert.com/articles/rtx-pro-6000-600w-local-ai-efficiency/ Thu, 20 Aug 2026 02:00:24 GMT My RTX PRO 6000 can consume 600 watts, but local AI rarely asks for all of it. A tuned 450-watt profile preserved practical Qwen performance, while a completed 994,987-token DeepSeek V4 Flash request averaged just 202.78 watts. I Bought the Ferrari of GPUs to Stop Compromising. Then the Server Demanded a Blood Sacrifice. https://msoexpert.com/articles/rtx-pro-6000-server-demanded-blood-sacrifice/ https://msoexpert.com/articles/rtx-pro-6000-server-demanded-blood-sacrifice/ Tue, 18 Aug 2026 14:48:30 GMT A 96 GB RTX PRO 6000 led to transient-power reboots, a third power supply, dual Xeons, external GPUs, a Dremel, blood, and a 1,500-watt MiniMax M3 experiment that still came up six gigabytes short. The Most Interesting Physical AI Isn’t a Robot. It’s a Model Etched Into Silicon. https://msoexpert.com/articles/the-model-is-the-silicon/ https://msoexpert.com/articles/the-model-is-the-silicon/ Sat, 08 Aug 2026 16:33:03 GMT Taalas physically embodied Llama 3.1 8B in silicon, reached roughly 17,000 tokens per second for one user, and was acquired by AMD before I finished writing about it. AMD AI PRO R9700: The card that just works. My stack around it didn’t. https://msoexpert.com/articles/amd-ai-pro-r9700-rocm-vulkan/ https://msoexpert.com/articles/amd-ai-pro-r9700-rocm-vulkan/ Tue, 04 Aug 2026 22:11:07 GMT ROCm worked on the first try. Vulkan was substantially faster, and the hardest failures belonged to everything around the card. Running High-Quality DeepSeek V4 Flash on One 96 GB GPU https://msoexpert.com/articles/making-deepseek-v4-flash-fit-one-96gb-gpu/ https://msoexpert.com/articles/making-deepseek-v4-flash-fit-one-96gb-gpu/ Tue, 04 Aug 2026 00:47:58 GMT Five mapped expert layers made room for speculative decoding, a correction tier, and near-million-token context. From Architecting and Selling AI to Building a Fairly Insane Edge Node in My Living Room https://msoexpert.com/articles/living-room-ai-edge-node/ https://msoexpert.com/articles/living-room-ai-edge-node/ Mon, 03 Aug 2026 17:08:34 GMT The origin story, hardware escalation, runtime experiments, failures, and corrections behind a private local AI system.