<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Sglang on NextDoorHacker</title><link>https://nextdoorhacker.com/tags/sglang/</link><description>Recent content in Sglang on NextDoorHacker</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Fri, 20 Nov 2026 09:00:00 -0800</lastBuildDate><atom:link href="https://nextdoorhacker.com/tags/sglang/index.xml" rel="self" type="application/rss+xml"/><item><title>I entered a GPU kernel contest and didn't write a kernel</title><link>https://nextdoorhacker.com/2026/11/20/i-entered-a-gpu-kernel-contest-and-didnt-write-a-kernel/</link><pubDate>Fri, 20 Nov 2026 09:00:00 -0800</pubDate><guid>https://nextdoorhacker.com/2026/11/20/i-entered-a-gpu-kernel-contest-and-didnt-write-a-kernel/</guid><description>&lt;p>&lt;em>How I ran a GPU kernel competition as a team of AI agents from different labs, on a budget I enforced in code.&lt;/em>&lt;/p>
&lt;blockquote>
&lt;p>&lt;strong>Disclaimer:&lt;/strong> An early draft of this post was generated by Claude from my project notes and records. I edited it from there.&lt;/p>&lt;/blockquote>
&lt;p>For thirteen days, from Oct 23 to Nov 4, I entered a GPU kernel competition, and I didn&amp;rsquo;t write a single kernel. I started the evening it opened, with my first scored run at 10:43 pm Pacific. I&amp;rsquo;d written CUDA a long time ago, but modern kernels with tensor cores and NVFP4 were not my world.&lt;/p>
&lt;p>The point was to explore the optimization space for serving an LLM: where the time goes, and what actually moves it. Typing out the CUDA was just the mechanics. Competitions are one of the fastest ways I know to learn something new: the scoreboard tells you every day whether you understood it or only thought you did. I wanted to use this one to learn two things. How far can one person get by directing AI agents instead of writing the code? And how much of the work can I offload to open models running locally on my Mac Studio (an M5 Max with 128 GB of memory)?&lt;/p>
&lt;p>There was a third question underneath both: how far could I get with the life I actually have? I have a day job and kids. I didn&amp;rsquo;t take time off, and I couldn&amp;rsquo;t watch a GPU queue all evening like I would have in my 20s. So the setup was designed around my time rather than the contest&amp;rsquo;s.&lt;/p>
&lt;p>So I ran the contest as a small organization: several Claude sessions working in parallel, ChatGPT as an adversarial reviewer, Grok and Gemini as outside researchers, and two Qwen models running locally for the reading. I set the budgets, approved any scored run past the agents&amp;rsquo; five a day, and decided which ideas to chase and when to stop. The agents wrote the kernels, built the tools to measure them, merged each other&amp;rsquo;s work and kept the tickets.&lt;/p>
&lt;p>I finished around the top quarter on the main board, for about $500 of Modal GPU time plus about $6 of Runpod credits (model subscriptions extra). This post is about how the setup worked: which model did what, how I kept spending under control, what the process caught, and what it missed. The logs are in collapsible tables for anyone who wants to check my work.&lt;/p>
&lt;h2 id="the-contest-briefly">The contest, briefly&lt;/h2>
&lt;p>The &lt;a href="https://openkernelchallenge.dev">Open Kernel Challenge&lt;/a> asked entrants to make &lt;a href="https://github.com/sgl-project/sglang">SGLang&lt;/a>, a popular open-source LLM serving engine, serve one specific model faster on NVIDIA H200 GPUs. The model is Muse Spark Open: 816 billion parameters, stored in NVFP4.&lt;/p>
&lt;p>A few ideas make the rest of this post easier to follow:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Mixture of experts (MoE).&lt;/strong> Muse Spark doesn&amp;rsquo;t run all 816B parameters for every token. Each layer has 256 small &amp;ldquo;expert&amp;rdquo; networks, and a router picks 8 of them per token, so only about 42B parameters do work for any one token. That makes the model cheap to run but awkward to make fast: every token reads a different set of weights.&lt;/li>
&lt;li>&lt;strong>NVFP4.&lt;/strong> The weights are 4-bit floating-point numbers, with a shared scale for every block of 16. Four bits per weight means less memory to read, but the H200&amp;rsquo;s tensor cores can&amp;rsquo;t multiply FP4 directly, so every kernel unpacks the weights first. That unpacking turned out to be one of the main costs.&lt;/li>
&lt;li>&lt;strong>Decode and verify.&lt;/strong> When a model generates text, each step produces one token per user: a &lt;em>decode&lt;/em> step. SGLang&amp;rsquo;s default recipe also has a small draft model guess 8 tokens ahead, then checks all 8 in one step: a &lt;em>verify&lt;/em> step. The contest timed both, for 1, 16 and 32 users at once.&lt;/li>
&lt;li>&lt;strong>The judge.&lt;/strong> Each submission ran on one H200 against SGLang&amp;rsquo;s own kernels on the same shapes. Your score on a board is your speedup over SGLang, as a geometric mean over its loads, so each load counts equally. Runs are noisy: the judge&amp;rsquo;s own intervals on my runs ranged from about ±1.5% to ±3.4%. Each team gets ten scored runs per problem per day.&lt;/li>
&lt;/ul>
&lt;p>The kernel track ran from Oct 23 to Nov 4 and had five boards. The decode layer is one full layer of the model, and it set the overall ranking. Three other boards each time one piece of it (attention, the experts, the BF16 projections); the fifth times the whole layer on prefill.&lt;/p>
&lt;details>
&lt;summary>&lt;b>The five boards and what each one times&lt;/b>&lt;/summary>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Board&lt;/th>
&lt;th>What it times&lt;/th>
&lt;th>Scored shapes&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Decode layer&lt;/td>
&lt;td>One full layer: attention, the router, the MoE experts and the projections&lt;/td>
&lt;td>12: decode and verify × 1, 16, 32 users × sliding-window and full-attention layers&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Attention decode&lt;/td>
&lt;td>Just the attention, reading an ~80K-token KV cache&lt;/td>
&lt;td>the same 12&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>MoE w4a16&lt;/td>
&lt;td>Just the routed experts: 4-bit weights, 16-bit activations&lt;/td>
&lt;td>6: one per load&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>BF16 projections&lt;/td>
&lt;td>The five bf16 matrix multiplies around the experts&lt;/td>
&lt;td>30: 6 loads × 5 projections&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Prefill layer&lt;/td>
&lt;td>Processing a prompt: turn 1, turn 4, and a 16K-token first turn&lt;/td>
&lt;td>8&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;/details>
&lt;h2 id="the-setup">The setup&lt;/h2>
&lt;p>I ran the contest as one person with a small organization of agents. Three ideas held it together.&lt;/p>
&lt;p>&lt;strong>Lanes.&lt;/strong> Each line of work (the MoE kernel, the attention, the projections, tooling) was a &lt;em>lane&lt;/em>: an agent session in its own git worktree, with its own brief, branch and GPU budget. A coordinator session merged results and owned the queue of scored runs. Lanes never touched each other&amp;rsquo;s files, so three or four could run at once without stepping on each other.&lt;/p>
&lt;p>&lt;strong>One change at a time.&lt;/strong> Each candidate changed one thing against the one before it, so when an A/B moved I knew what to look at.&lt;/p>
&lt;p>&lt;strong>Linear as the source of truth.&lt;/strong> One issue per workstream, every result posted as a comment, and every scored run past the agents&amp;rsquo; daily five approved by me in chat and quoted in the ticket (&amp;ldquo;Send S1&amp;rdquo;, &amp;ldquo;You have my ok to send it&amp;rdquo;, &amp;ldquo;Send the winner&amp;rdquo;).&lt;/p>
&lt;h3 id="five-model-families-each-with-one-job">Five model families, each with one job&lt;/h3>
&lt;p>I didn&amp;rsquo;t use one model for everything. Each had a role that played to what it was good at, and the hand-offs between them were files in git, not pasted prompts.&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Model&lt;/th>
&lt;th>Role&lt;/th>
&lt;th>How it worked&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;strong>Claude&lt;/strong> (&lt;a href="https://claude.com/claude-code">Claude Code&lt;/a> sessions)&lt;/td>
&lt;td>The lanes and the coordinator: wrote kernels, tools, tests and merges; ran GPU jobs; kept Linear current&lt;/td>
&lt;td>One session per lane, each in its own worktree. Also used for deep-research passes, such as checking whether a finding was really new before calling it that.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>ChatGPT&lt;/strong> (&amp;ldquo;Astra&amp;rdquo;)&lt;/td>
&lt;td>Adversarial reviewer. Its brief was to find what&amp;rsquo;s wrong, not to agree.&lt;/td>
&lt;td>Worked its own queue in the repo: review requests went into a handoff folder, findings came back as files. Each finding was triaged as TRUE, FALSE or UNTESTED; runtime claims needed a reproduction, while wording and source issues were checked by reading.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Grok&lt;/strong>&lt;/td>
&lt;td>Outside scout and framing partner: what&amp;rsquo;s public, who&amp;rsquo;s doing what, how to tell the story&lt;/td>
&lt;td>Its notes went into their own file, kept apart from verified intel. Its claims were checked against sources before I used them; some were wrong.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Gemini&lt;/strong>&lt;/td>
&lt;td>Second research voice on open questions&lt;/td>
&lt;td>Separate prompt files and an intel file, checked the same way.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Qwen3.8 and Qwen3-Coder-30B&lt;/strong>, local (&lt;a href="https://github.com/ggml-org/llama.cpp">llama.cpp&lt;/a>)&lt;/td>
&lt;td>Reading: summarizing logs, sweeping source trees, first drafts of notes&lt;/td>
&lt;td>Ran on my Mac Studio at no marginal cost. Inputs were kept small enough for them (about 60 KB and 30 KB), and every output went through a review ledger before anyone used it.&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>Two lessons from mixing models:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Disagreement is the feature.&lt;/strong> The reviewer caught claims the lanes were sure of. One test was meant to prove a hang was fixed, so the old code had to hang and the new code had to pass; Astra noticed it also counted the old code failing to build as a reproduced hang, so a broken setup would have looked like proof. Another time my own board audit treated the prefill score as an average over four loads. Astra pointed out there are three, and the cube root of the three load speedups (1.195, 1.359 and 1.483) is exactly the judged 1.340. Neither came from a model checking its own work.&lt;/li>
&lt;li>&lt;strong>In my setup, local models were for reading, not editing.&lt;/strong> Qwen3-Coder-30B failed twice on a 12-line kernel edit, so I stopped giving it edits. That&amp;rsquo;s two tries with a 30B model on my Mac Studio, not a verdict on local models: a larger one might have done better, and I didn&amp;rsquo;t try one. As a reader of logs and source trees it was fine, with no extra provider charge.&lt;/li>
&lt;/ul>
&lt;h3 id="sessions-context-resets-and-roles">Sessions, context resets, and roles&lt;/h3>
&lt;p>The thing I understood least at the start was the context window. A Claude session can only hold so much conversation. When it fills up, the session summarizes itself and carries on from the summary, and details that didn&amp;rsquo;t make the summary are gone.&lt;/p>
&lt;p>At first I treated that as the end of a session. When one got long and started losing track, I opened a new one: the first session, then one I started by pasting in the project&amp;rsquo;s handoff file, then OKC2, OKC3, and so on. Each restart cost time while the new session re-learned where things stood.&lt;/p>
&lt;p>What made restarts cheap was moving the state out of the conversation and into files:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>A handoff file&lt;/strong> (CLAUDE.md, from day 1): the rules, the budgets, every standing decision with my exact words and the date.&lt;/li>
&lt;li>&lt;strong>Decision and learning logs&lt;/strong>, a run log, and Linear, so &amp;ldquo;what did I decide and why&amp;rdquo; had one answer outside anyone&amp;rsquo;s memory.&lt;/li>
&lt;li>&lt;strong>A handoff queue in the repo&lt;/strong> for work between models: requests in one folder, claimed work in another, results in a third.&lt;/li>
&lt;/ul>
&lt;p>Files carry mistakes too. Once, a handoff note carried a wrong belief about an accuracy margin from one session to the next, and it nearly made me drop a precision that was actually fine. A scripted re-measurement caught it for $0.84. Since then, any number that matters gets re-measured, not just copied forward.&lt;/p>
&lt;p>By the seventh session, OKC7, I&amp;rsquo;d stopped thinking of sessions as disposable and started thinking of them as &lt;strong>staff with roles&lt;/strong>. A session with a narrow job and its rules written down survives many context resets, because the summary only has to keep a little and the files hold the rest. By the serving track, the setup looked like this:&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Session&lt;/th>
&lt;th>Role&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>OKC7&lt;/td>
&lt;td>Serving lead: patches, one-GPU tests, scored runs (with my approval)&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>AWS session&lt;/td>
&lt;td>GPU infrastructure: spot capacity, boxes, weights, cost reports&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Research and writing&lt;/td>
&lt;td>Visualizations, the follow-up log, this post. Read-only on GPUs and the judge.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Astra (ChatGPT)&lt;/td>
&lt;td>Adversarial review, through the handoff queue&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Scheduled sessions&lt;/td>
&lt;td>One narrow job each: send approved runs right after the daily reset, post the rank to Linear twice a day, sweep public kernel work twice a day, scout other competitions&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>A context reset stopped being an emergency and became a shift change. And because I could open any of these sessions from my phone, approving a run or redirecting a lane didn&amp;rsquo;t need me at my desk.&lt;/p>
&lt;h3 id="built-around-a-day-job-and-kids">Built around a day job and kids&lt;/h3>
&lt;p>Every part of the setup had to work when I wasn&amp;rsquo;t there:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>The agents worked while I didn&amp;rsquo;t.&lt;/strong> Lanes ran in the background in their own worktrees. When I came back, the state of every lane was in Linear and the repo, not in my head.&lt;/li>
&lt;li>&lt;strong>My part was decisions, and decisions are small.&lt;/strong> A scored run past the agents&amp;rsquo; five needed one line from me (&amp;ldquo;Send the winner&amp;rdquo;). A budget change was one line. Most of my time on the contest was reading a summary and saying yes, no, or stop.&lt;/li>
&lt;li>&lt;strong>Little of importance depended on me watching.&lt;/strong> Scouts swept public code on a schedule. GPU work was packed into long batches that ran unattended. The Modal launcher refused new jobs once the day&amp;rsquo;s reported spend neared my cap, job timeouts bounded each run, and Modal&amp;rsquo;s billing-cycle limit was the backstop behind both: once it was hit, nothing new started until I raised it, so a forgotten job had little room to run up a bill while I was at work or putting kids to bed.&lt;/li>
&lt;li>&lt;strong>Stopping was a decision I made on purpose.&lt;/strong> On Nov 1 I stopped new large optimization work short of the top 100, rather than let it eat more evenings for a few tenths of a percent. The last few days were small ports and resubmits.&lt;/li>
&lt;/ul>
&lt;h2 id="cost-controls">Cost controls&lt;/h2>
&lt;p>The budget was the constraint I cared about most, so I enforced it in code instead of trusting anyone, me included, to remember it.&lt;/p>
&lt;ul>
&lt;li>&lt;strong>A daily cap inside the GPU launcher.&lt;/strong> Every Modal job went through one script that checked Modal&amp;rsquo;s reported spend before launching and refused new jobs once the day&amp;rsquo;s cap was near. It was a soft limit: jobs already running, Modal&amp;rsquo;s billing lag or a failed billing read could get past it, and Runpod went through a different tool. The cap started at $45/day and went to $100/day when the work justified it. I raised it twice for single days, to $125 and then $140 on Oct 31, with the reason written down; that day spent $131.&lt;/li>
&lt;li>&lt;strong>A billing-cycle ceiling on Modal&amp;rsquo;s side.&lt;/strong> Separately, Modal&amp;rsquo;s workspace limit stopped new work once a billing cycle&amp;rsquo;s spend reached it, though a job already running could finish. It applies per cycle, so the track&amp;rsquo;s $508 total spans two cycles: about $438 in October and $70 in November. It started at $50. When a job hit it, I told the agents &amp;ldquo;I cannot keep spending all this money&amp;rdquo;, raised it to $70, and stopped the background lanes for the rest of the cycle. It later went to $170, then $500.&lt;/li>
&lt;li>&lt;strong>Per-lane budgets.&lt;/strong> When lanes restarted, each got its own GPU cap: MoE $15, attention $10, projections $6.&lt;/li>
&lt;li>&lt;strong>CPU first.&lt;/strong> For Triton kernels, a GPU-less pre-flight on my Mac took about 5 seconds instead of ~50 on a GPU. Build errors, register problems and some logic bugs died there for free, before they cost any GPU time.&lt;/li>
&lt;li>&lt;strong>Packed jobs.&lt;/strong> Instead of one GPU container per experiment, one container ran many variants back to back: one setup, many measurements, plus a byte-identical control copy to catch drift. This came out of a day where only ~20% of the paid time was GPU-busy.&lt;/li>
&lt;li>&lt;strong>Kill criteria written before the run.&lt;/strong> Every experiment said in advance what result would end it.&lt;/li>
&lt;li>&lt;strong>Judged runs as a budget.&lt;/strong> The contest allowed ten scored runs per problem per day. The agents could send five on their own; the rest needed my approval in chat, so I&amp;rsquo;d always have chances left.&lt;/li>
&lt;/ul>
&lt;p>The whole kernel track cost &lt;strong>$508 of Modal GPU time&lt;/strong> over thirteen days, plus about $6 on Runpod, mostly from early credits (model subscriptions not included).&lt;/p>
&lt;details>
&lt;summary>&lt;b>Modal spend by day, kernel track&lt;/b>&lt;/summary>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Day (UTC)&lt;/th>
&lt;th>Spend&lt;/th>
&lt;th>What happened&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Oct 24&lt;/td>
&lt;td>$12.09&lt;/td>
&lt;td>day 1 (the evening of Oct 23, Pacific): framework settings and glue&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 25&lt;/td>
&lt;td>$53.55&lt;/td>
&lt;td>hit the $50 cycle limit; raised to $70, background lanes stopped&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 26&lt;/td>
&lt;td>$62.98&lt;/td>
&lt;td>limit raised to $170; lanes restarted with per-lane caps&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 27&lt;/td>
&lt;td>$15.02&lt;/td>
&lt;td>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 28&lt;/td>
&lt;td>$51.32&lt;/td>
&lt;td>cycle limit raised to $500 spend / $470 usage&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 29&lt;/td>
&lt;td>$70.56&lt;/td>
&lt;td>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 30&lt;/td>
&lt;td>$41.13&lt;/td>
&lt;td>the rule change that moved me ~180 places (&lt;a href="#wrong-turns">below&lt;/a>)&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Oct 31&lt;/td>
&lt;td>$131.17&lt;/td>
&lt;td>the one day over $100, by approval: caps of $125, then $140&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Nov 1&lt;/td>
&lt;td>$46.40&lt;/td>
&lt;td>stopped large optimization work short of the top 100&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Nov 2&lt;/td>
&lt;td>$20.00&lt;/td>
&lt;td>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Nov 3&lt;/td>
&lt;td>$3.19&lt;/td>
&lt;td>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Nov 4&lt;/td>
&lt;td>$0.82&lt;/td>
&lt;td>track closes&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Total&lt;/strong>&lt;/td>
&lt;td>&lt;strong>$508.23&lt;/strong>&lt;/td>
&lt;td>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;em>Source: Modal billing report, Oct 22 – Nov 5.&lt;/em>&lt;/p>
&lt;/details>
&lt;h2 id="finding-h200s-colab-modal-runpod">Finding H200s: Colab, Modal, Runpod&lt;/h2>
&lt;p>The judge ran on H200s, and I didn&amp;rsquo;t have one. Every number I cared about depended on renting the same GPU, in the same software, by the minute.&lt;/p>
&lt;details>
&lt;summary>&lt;b>Colab, Modal and Runpod: how I rented the judge's GPU&lt;/b>&lt;/summary>
&lt;p>&lt;strong>Colab was a dead end on day 1.&lt;/strong> Colab advertises H100s, and even an H100 would only have been a stand-in for the judge&amp;rsquo;s H200. But every runtime I got was an RTX 6000: a different architecture, with different memory and bandwidth, so its timings said nothing about how a kernel would do on the judge. My account also had no compute units, and &lt;a href="https://research.google.com/colaboratory/faq.html">Colab&amp;rsquo;s terms&lt;/a> don&amp;rsquo;t allow SSH or containers on free runtimes, so I couldn&amp;rsquo;t run the judge&amp;rsquo;s SGLang image either. That ruled it out for anything resembling the judge.&lt;/p>
&lt;p>&lt;strong>&lt;a href="https://modal.com">Modal&lt;/a> fit the way I wanted to work.&lt;/strong> I picked it for three reasons:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>H200s on demand, billed per second.&lt;/strong> A container starts in about five seconds and stops when the command does, so a two-minute experiment costs a couple of minutes of GPU, not an hour&amp;rsquo;s rental.&lt;/li>
&lt;li>&lt;strong>The judge&amp;rsquo;s own image.&lt;/strong> Modal can run any container image, so every experiment ran inside the exact SGLang image the contest judged in. SGLang&amp;rsquo;s baseline kernels ran beside mine on the same GPU, in the same run.&lt;/li>
&lt;li>&lt;strong>Scriptable from my Mac.&lt;/strong> One launcher script took a list of jobs, ran each in its own container on its own H200 in parallel, and copied logs and profiler traces back. The agents could drive it without me.&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>How it changed the iteration loop.&lt;/strong> Modal&amp;rsquo;s absolute times read about 7% faster than the judge&amp;rsquo;s, with a wide spread, so I never trusted a single number from it. What I trusted was an A/B &lt;em>inside one run&lt;/em>: my version and the baseline, timed side by side on the same GPU, with a control copy to catch drift. That gave a three-stage loop, each stage slower and scarcer than the last:&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Stage&lt;/th>
&lt;th>Time per check&lt;/th>
&lt;th>Cost&lt;/th>
&lt;th>What it catches&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>CPU pre-flight on my Mac (Triton kernels)&lt;/td>
&lt;td>~5 s&lt;/td>
&lt;td>free&lt;/td>
&lt;td>build errors, some logic errors, obviously bad register use&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A/B on a Modal H200, in the judge&amp;rsquo;s image&lt;/td>
&lt;td>a few minutes&lt;/td>
&lt;td>cents to under a dollar per batch&lt;/td>
&lt;td>whether it&amp;rsquo;s actually faster, and by how much&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>The judge&lt;/td>
&lt;td>10 per problem per day (5 without my approval), sometimes hours in the queue&lt;/td>
&lt;td>the scarcest thing I had&lt;/td>
&lt;td>the real score&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>With that loop, a lane went from an idea to a measured candidate in 25–60 minutes, and twelve judged runs in one afternoon all landed within 2% of their Modal-based predictions.&lt;/p>
&lt;p>&lt;strong>Where Modal got in the way.&lt;/strong> I kept a running list:&lt;/p>
&lt;ul>
&lt;li>Each container starts cold, so I shared a compile cache between containers to skip minutes of JIT builds. That cache then hid how long setup took: one entry passed every Modal check and failed the judge&amp;rsquo;s 300-second setup limit at 355 seconds, costing a scored run. After that, every candidate got a cold-start check before sending.&lt;/li>
&lt;li>Nsight Compute couldn&amp;rsquo;t lock GPU clocks inside Modal&amp;rsquo;s containers, so profiler timings drifted more than on a dedicated box.&lt;/li>
&lt;li>A hung job ran to the 30-minute timeout and lost everything it hadn&amp;rsquo;t printed.&lt;/li>
&lt;li>An account can run 10 GPUs at once, so the launcher handed out slots and queued the rest instead of letting parallel lanes collide.&lt;/li>
&lt;li>Spend shows up in the billing report with a lag, so per-batch costs were estimates until it caught up.&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>When Modal had no H200s.&lt;/strong> On day 2, Modal ran out of H200s for about 2.8 hours. At first it looked like a hung batch; it was waiting for a GPU that didn&amp;rsquo;t exist. Rather than lose the afternoon, a second session set up &lt;a href="https://www.runpod.io">Runpod&lt;/a> as a second backend, and the same A/B on both clouds agreed (1.036× vs 1.034×) before I trusted it. From then on, Runpod was a standing second backend.&lt;/p>
&lt;/details>
&lt;h2 id="where-i-finished">Where I finished&lt;/h2>
&lt;p>&lt;strong>Final standings at the close&lt;/strong>&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Board&lt;/th>
&lt;th>My best&lt;/th>
&lt;th>Final rank&lt;/th>
&lt;th>#1 at the close&lt;/th>
&lt;th>Judge&amp;rsquo;s modeled limit&lt;/th>
&lt;th>Share of limit: me / #1&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Decode layer (sets the overall rank)&lt;/td>
&lt;td>2.243×&lt;/td>
&lt;td>#302 of 1,297&lt;/td>
&lt;td>2.614×&lt;/td>
&lt;td>3.88×&lt;/td>
&lt;td>58% / 67%&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>MoE w4a16&lt;/td>
&lt;td>2.403×&lt;/td>
&lt;td>#193 of 1,423&lt;/td>
&lt;td>2.844×&lt;/td>
&lt;td>4.35×&lt;/td>
&lt;td>55% / 65%&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Attention decode&lt;/td>
&lt;td>2.429×&lt;/td>
&lt;td>#460 of 1,380&lt;/td>
&lt;td>3.339×&lt;/td>
&lt;td>6.73×&lt;/td>
&lt;td>36% / 50%&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>BF16 projections&lt;/td>
&lt;td>1.397×&lt;/td>
&lt;td>#518 of 1,741&lt;/td>
&lt;td>1.728×&lt;/td>
&lt;td>2.02×&lt;/td>
&lt;td>69% / 86%&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Prefill layer&lt;/td>
&lt;td>1.355×&lt;/td>
&lt;td>#635 of 896&lt;/td>
&lt;td>2.147×&lt;/td>
&lt;td>3.06×&lt;/td>
&lt;td>44% / 70%&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>I finished the decode layer at #302. On the morning of Nov 4 I was still #251, about 2% under that day&amp;rsquo;s top-100 line of 2.288×, and my best rank was #123 on Oct 31. My first scored entry, on day 1, was 1.051×.&lt;/p>
&lt;h3 id="how-far-up-the-hill-is-that">How far up the hill is that?&lt;/h3>
&lt;p>A speedup by itself doesn&amp;rsquo;t tell you much. 2.24× could be nearly perfect or barely started, depending on what the hardware allows. So I worked out the ceiling.&lt;/p>
&lt;p>The judge publishes a &lt;em>floor&lt;/em> for every shape it times: the time to move that shape&amp;rsquo;s data at the H200&amp;rsquo;s full memory bandwidth (4.8 TB/s), or to do its math at full tensor-core speed (989 TFLOP/s), whichever is longer. By the judge&amp;rsquo;s model, no kernel can beat its floor.&lt;/p>
&lt;p>One example is the decode layer&amp;rsquo;s single-user, sliding-window shape, where SGLang takes about 173 µs. Just reading that step&amp;rsquo;s weights and KV cache once, at full memory speed, takes 40 µs. So nothing can beat SGLang on that shape by more than about 4.3×. Put every shape on its floor and you get each board&amp;rsquo;s limit under the judge&amp;rsquo;s model, which I&amp;rsquo;ll call the &lt;strong>hardware limit&lt;/strong>: 3.88× for the decode layer.&lt;/p>
&lt;p>Nobody got close. The winners finished at 50–86% of their board&amp;rsquo;s limit, and I finished at 36–69%. As a ratio, the gap between me and #1 was widest on prefill (44% vs 70%, so I reached about 63% of #1&amp;rsquo;s share) and next widest on attention decode (36% vs 50%, about 72%). Attention was also the board where I reached the smallest share of the limit. Attention, it turned out, was the lever I missed.&lt;/p>
&lt;p>&lt;img alt="Figure 1. Share of each board&amp;rsquo;s judge&amp;rsquo;s modeled limit, me vs #1" loading="lazy" src="https://nextdoorhacker.com/assets/images/okc/hardware-limit.svg">&lt;/p>
&lt;h2 id="hill-climbing-literally">Hill climbing, literally&lt;/h2>
&lt;p>People describe optimization as hill climbing all the time. In this contest it isn&amp;rsquo;t a metaphor, and drawing it that way explains most of what happened.&lt;/p>
&lt;p>&lt;strong>The map is real.&lt;/strong> Four boards score decode and verify loads: the decode layer, attention decode, MoE and BF16 projections. Each of their scores is a geometric mean, so it splits &lt;em>exactly&lt;/em> into two halves: the geomean of the decode shapes (call it D) and of the verify shapes (V). The prefill board scores first and later turns instead, so it doesn&amp;rsquo;t split this way. The score is √(D × V). So every candidate kernel is a point on a two-dimensional map, its height is its judged score, and the lines of equal height are curves you can draw. Here is my decode-layer climb on that map, from the first scored run to the last.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;board=muse_decode_layer&amp;amp;mode=ours" title="Decode-layer climb" loading="lazy"
style="width:100%;height:560px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 2. My 43 scored decode-layer runs on the board&amp;#39;s exact score map, from 1.05× to 2.24×. The white curves are lines of equal score; the black dashed curve is the judge&amp;#39;s modeled limit. &lt;a href="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;board=muse_decode_layer&amp;amp;mode=ours">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;p>&lt;strong>A step is one change.&lt;/strong> Each move on the map was meant to be one change, A/B&amp;rsquo;d against the version before it, though a few late steps bundled links I had already tested one at a time, and the judge&amp;rsquo;s noise moved heights too. If it went up, I kept it; if not, I stepped back. That&amp;rsquo;s the &amp;ldquo;one change at a time&amp;rdquo; rule, and it&amp;rsquo;s why the path is a staircase of small steps. Most of the height came from a handful of big ones.&lt;/p>
&lt;p>&lt;strong>The fog is the judge&amp;rsquo;s noise.&lt;/strong> Judged runs varied by about ±1.5% to ±3.4%, and late in the contest most single steps were smaller than that. In fog, you can&amp;rsquo;t tell up from flat by looking. That&amp;rsquo;s what the control copy in every benchmark batch was for, and why every prediction was written down before the judge saw the run (the Modal section above has how well they held up).&lt;/p>
&lt;p>&lt;strong>Feel the slope before you step.&lt;/strong> A judged run is expensive (ten per problem per day, and the agents could only send five of them), so every step was tested on the way: the CPU pre-flight first, then a one-GPU A/B with a control, and only then the judge. Plenty of bad steps were caught at the first stage, for free.&lt;/p>
&lt;p>&lt;strong>The ground doesn&amp;rsquo;t rise evenly.&lt;/strong> The map shows one number per run, but each run is twelve shapes, and they didn&amp;rsquo;t move together. Press Play to watch every shape&amp;rsquo;s bar across all 43 runs.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=terrain&amp;amp;board=muse_decode_layer&amp;amp;run=first" title="Decode-layer shapes, run by run" loading="lazy"
style="width:100%;height:600px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 3. One bar per scored shape: decode and verify loads left to right, sliding and full-attention layers front to back. The grid at 1.0× is SGLang; the small diamonds mark the first run. Press Play, or switch the height to each shape&amp;#39;s share of the judge&amp;#39;s modeled limit. &lt;a href="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=terrain&amp;amp;board=muse_decode_layer&amp;amp;run=first">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;p>&lt;strong>The biggest steps were jumps, not climbs.&lt;/strong> Two kinds of moves gave most of the height. One was moving to someone else&amp;rsquo;s position: porting a piece of public code. The other was taking down a fence I had built myself. My bit-identical rule kept me off a faster public base from Oct 28 to Oct 30; the day I dropped it, one run moved me from 1.896× to 2.163×.&lt;/p>
&lt;p>&lt;strong>I climbed the slope I knew, not the steep one.&lt;/strong> In the last days the agents kept working on the MoE kernel, which was already within 1.1–1.8× of its floor: a slope that had nearly flattened out. The attention had far more room, and I didn&amp;rsquo;t see it until after the close.&lt;/p>
&lt;p>&lt;strong>There&amp;rsquo;s no summit, just a ceiling.&lt;/strong> The score only rises with speed, so the surface is a ramp. Its top is the hardware limit, where every shape runs at its floor. Nobody reached it. The ceiling that mattered was the field&amp;rsquo;s, and it moved every day.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;board=muse_decode_layer&amp;amp;mode=field" title="Decode-layer climb with the field" loading="lazy"
style="width:100%;height:560px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 4. The same map with where the contest finished: #1 (solid), the top-10 and top-100 cutoffs, and the judge&amp;#39;s modeled limit. &lt;a href="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;board=muse_decode_layer&amp;amp;mode=field">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;p>All four boards that score decode and verify loads share the same surface, so their paths can be drawn together. Drag to rotate.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;mode=ours&amp;amp;cam=over" title="Four boards, one surface" loading="lazy"
style="width:100%;height:600px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 5. Decode layer, attention decode, MoE and BF16 projections on one √(D × V) surface, each with its own judge&amp;#39;s modeled limit. Use the buttons to focus on one board or switch to the field. &lt;a href="https://nextdoorhacker.com/okc/kernel-climb.html?embed&amp;amp;view=climb&amp;amp;mode=ours&amp;amp;cam=over">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;p>&lt;img alt="Figure 6. Every board, flat: my path, #1, the top-100 cutoff and the judge&amp;rsquo;s modeled limit" loading="lazy" src="https://nextdoorhacker.com/assets/images/okc/kernel-climb-field.svg">&lt;/p>
&lt;details>
&lt;summary>&lt;b>Run ledger: passed scored runs with saved results, by board (dates are log times)&lt;/b>&lt;/summary>
&lt;p>&lt;strong>Decode layer&lt;/strong> (43 runs)&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Date (UTC)&lt;/th>
&lt;th>Step&lt;/th>
&lt;th style="text-align: right">Score&lt;/th>
&lt;th style="text-align: right">Change&lt;/th>
&lt;th style="text-align: right">Decode half (D)&lt;/th>
&lt;th style="text-align: right">Verify half (V)&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>2026-10-24 05:43&lt;/td>
&lt;td>L1/M4&lt;/td>
&lt;td style="text-align: right">1.051×&lt;/td>
&lt;td style="text-align: right">&lt;/td>
&lt;td style="text-align: right">0.999&lt;/td>
&lt;td style="text-align: right">1.106&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-24 08:13&lt;/td>
&lt;td>v2&lt;/td>
&lt;td style="text-align: right">1.115×&lt;/td>
&lt;td style="text-align: right">+0.064&lt;/td>
&lt;td style="text-align: right">1.089&lt;/td>
&lt;td style="text-align: right">1.143&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-24 14:15&lt;/td>
&lt;td>v3a&lt;/td>
&lt;td style="text-align: right">1.124×&lt;/td>
&lt;td style="text-align: right">+0.008&lt;/td>
&lt;td style="text-align: right">1.115&lt;/td>
&lt;td style="text-align: right">1.132&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-24 23:13&lt;/td>
&lt;td>v6&lt;/td>
&lt;td style="text-align: right">1.188×&lt;/td>
&lt;td style="text-align: right">+0.064&lt;/td>
&lt;td style="text-align: right">1.188&lt;/td>
&lt;td style="text-align: right">1.188&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 03:41&lt;/td>
&lt;td>v7b&lt;/td>
&lt;td style="text-align: right">1.225×&lt;/td>
&lt;td style="text-align: right">+0.037&lt;/td>
&lt;td style="text-align: right">1.263&lt;/td>
&lt;td style="text-align: right">1.188&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 04:11&lt;/td>
&lt;td>v8&lt;/td>
&lt;td style="text-align: right">1.263×&lt;/td>
&lt;td style="text-align: right">+0.038&lt;/td>
&lt;td style="text-align: right">1.262&lt;/td>
&lt;td style="text-align: right">1.264&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 13:15&lt;/td>
&lt;td>v9&lt;/td>
&lt;td style="text-align: right">1.266×&lt;/td>
&lt;td style="text-align: right">+0.003&lt;/td>
&lt;td style="text-align: right">1.306&lt;/td>
&lt;td style="text-align: right">1.227&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 17:51&lt;/td>
&lt;td>v12&lt;/td>
&lt;td style="text-align: right">1.287×&lt;/td>
&lt;td style="text-align: right">+0.021&lt;/td>
&lt;td style="text-align: right">1.312&lt;/td>
&lt;td style="text-align: right">1.262&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 21:05&lt;/td>
&lt;td>v15&lt;/td>
&lt;td style="text-align: right">1.367×&lt;/td>
&lt;td style="text-align: right">+0.081&lt;/td>
&lt;td style="text-align: right">1.316&lt;/td>
&lt;td style="text-align: right">1.421&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 21:25&lt;/td>
&lt;td>c1&lt;/td>
&lt;td style="text-align: right">1.402×&lt;/td>
&lt;td style="text-align: right">+0.035&lt;/td>
&lt;td style="text-align: right">1.352&lt;/td>
&lt;td style="text-align: right">1.454&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:15&lt;/td>
&lt;td>c2&lt;/td>
&lt;td style="text-align: right">1.399×&lt;/td>
&lt;td style="text-align: right">-0.003&lt;/td>
&lt;td style="text-align: right">1.351&lt;/td>
&lt;td style="text-align: right">1.449&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:18&lt;/td>
&lt;td>c3&lt;/td>
&lt;td style="text-align: right">1.433×&lt;/td>
&lt;td style="text-align: right">+0.034&lt;/td>
&lt;td style="text-align: right">1.394&lt;/td>
&lt;td style="text-align: right">1.472&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:22&lt;/td>
&lt;td>c4&lt;/td>
&lt;td style="text-align: right">1.449×&lt;/td>
&lt;td style="text-align: right">+0.016&lt;/td>
&lt;td style="text-align: right">1.413&lt;/td>
&lt;td style="text-align: right">1.485&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:27&lt;/td>
&lt;td>c5&lt;/td>
&lt;td style="text-align: right">1.434×&lt;/td>
&lt;td style="text-align: right">-0.015&lt;/td>
&lt;td style="text-align: right">1.401&lt;/td>
&lt;td style="text-align: right">1.468&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 01:44&lt;/td>
&lt;td>c6&lt;/td>
&lt;td style="text-align: right">1.526×&lt;/td>
&lt;td style="text-align: right">+0.092&lt;/td>
&lt;td style="text-align: right">1.470&lt;/td>
&lt;td style="text-align: right">1.583&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 01:56&lt;/td>
&lt;td>c7&lt;/td>
&lt;td style="text-align: right">1.548×&lt;/td>
&lt;td style="text-align: right">+0.022&lt;/td>
&lt;td style="text-align: right">1.498&lt;/td>
&lt;td style="text-align: right">1.599&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:09&lt;/td>
&lt;td>c8_pdl&lt;/td>
&lt;td style="text-align: right">1.560×&lt;/td>
&lt;td style="text-align: right">+0.012&lt;/td>
&lt;td style="text-align: right">1.494&lt;/td>
&lt;td style="text-align: right">1.629&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:19&lt;/td>
&lt;td>c9v&lt;/td>
&lt;td style="text-align: right">1.571×&lt;/td>
&lt;td style="text-align: right">+0.011&lt;/td>
&lt;td style="text-align: right">1.524&lt;/td>
&lt;td style="text-align: right">1.620&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:29&lt;/td>
&lt;td>c10r&lt;/td>
&lt;td style="text-align: right">1.564×&lt;/td>
&lt;td style="text-align: right">-0.007&lt;/td>
&lt;td style="text-align: right">1.531&lt;/td>
&lt;td style="text-align: right">1.598&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:37&lt;/td>
&lt;td>c11c&lt;/td>
&lt;td style="text-align: right">1.583×&lt;/td>
&lt;td style="text-align: right">+0.019&lt;/td>
&lt;td style="text-align: right">1.531&lt;/td>
&lt;td style="text-align: right">1.637&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:47&lt;/td>
&lt;td>c12t&lt;/td>
&lt;td style="text-align: right">1.581×&lt;/td>
&lt;td style="text-align: right">-0.002&lt;/td>
&lt;td style="text-align: right">1.543&lt;/td>
&lt;td style="text-align: right">1.620&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 13:59&lt;/td>
&lt;td>c13t&lt;/td>
&lt;td style="text-align: right">1.594×&lt;/td>
&lt;td style="text-align: right">+0.013&lt;/td>
&lt;td style="text-align: right">1.545&lt;/td>
&lt;td style="text-align: right">1.645&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 22:15&lt;/td>
&lt;td>c14z&lt;/td>
&lt;td style="text-align: right">1.607×&lt;/td>
&lt;td style="text-align: right">+0.013&lt;/td>
&lt;td style="text-align: right">1.578&lt;/td>
&lt;td style="text-align: right">1.637&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 00:51&lt;/td>
&lt;td>c15a&lt;/td>
&lt;td style="text-align: right">1.644×&lt;/td>
&lt;td style="text-align: right">+0.037&lt;/td>
&lt;td style="text-align: right">1.630&lt;/td>
&lt;td style="text-align: right">1.659&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 01:02&lt;/td>
&lt;td>c15b&lt;/td>
&lt;td style="text-align: right">1.671×&lt;/td>
&lt;td style="text-align: right">+0.026&lt;/td>
&lt;td style="text-align: right">1.665&lt;/td>
&lt;td style="text-align: right">1.676&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 23:51&lt;/td>
&lt;td>c16a&lt;/td>
&lt;td style="text-align: right">1.663×&lt;/td>
&lt;td style="text-align: right">-0.008&lt;/td>
&lt;td style="text-align: right">1.658&lt;/td>
&lt;td style="text-align: right">1.669&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 00:31&lt;/td>
&lt;td>c16b&lt;/td>
&lt;td style="text-align: right">1.690×&lt;/td>
&lt;td style="text-align: right">+0.026&lt;/td>
&lt;td style="text-align: right">1.693&lt;/td>
&lt;td style="text-align: right">1.686&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 13:13&lt;/td>
&lt;td>c17b&lt;/td>
&lt;td style="text-align: right">1.710×&lt;/td>
&lt;td style="text-align: right">+0.020&lt;/td>
&lt;td style="text-align: right">1.730&lt;/td>
&lt;td style="text-align: right">1.690&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 23:26&lt;/td>
&lt;td>c19&lt;/td>
&lt;td style="text-align: right">1.868×&lt;/td>
&lt;td style="text-align: right">+0.158&lt;/td>
&lt;td style="text-align: right">1.826&lt;/td>
&lt;td style="text-align: right">1.911&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-29 00:04&lt;/td>
&lt;td>c21&lt;/td>
&lt;td style="text-align: right">1.877×&lt;/td>
&lt;td style="text-align: right">+0.009&lt;/td>
&lt;td style="text-align: right">1.833&lt;/td>
&lt;td style="text-align: right">1.923&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-29 02:53&lt;/td>
&lt;td>c24&lt;/td>
&lt;td style="text-align: right">1.864×&lt;/td>
&lt;td style="text-align: right">-0.013&lt;/td>
&lt;td style="text-align: right">1.811&lt;/td>
&lt;td style="text-align: right">1.919&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-29 03:26&lt;/td>
&lt;td>c25&lt;/td>
&lt;td style="text-align: right">1.887×&lt;/td>
&lt;td style="text-align: right">+0.023&lt;/td>
&lt;td style="text-align: right">1.847&lt;/td>
&lt;td style="text-align: right">1.927&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-29 06:40&lt;/td>
&lt;td>c28&lt;/td>
&lt;td style="text-align: right">1.896×&lt;/td>
&lt;td style="text-align: right">+0.009&lt;/td>
&lt;td style="text-align: right">1.859&lt;/td>
&lt;td style="text-align: right">1.934&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-30 19:58&lt;/td>
&lt;td>c30&lt;/td>
&lt;td style="text-align: right">1.891×&lt;/td>
&lt;td style="text-align: right">-0.005&lt;/td>
&lt;td style="text-align: right">1.870&lt;/td>
&lt;td style="text-align: right">1.912&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-30 19:55&lt;/td>
&lt;td>a1&lt;/td>
&lt;td style="text-align: right">2.163×&lt;/td>
&lt;td style="text-align: right">+0.272&lt;/td>
&lt;td style="text-align: right">2.111&lt;/td>
&lt;td style="text-align: right">2.216&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 02:12&lt;/td>
&lt;td>a3&lt;/td>
&lt;td style="text-align: right">2.206×&lt;/td>
&lt;td style="text-align: right">+0.043&lt;/td>
&lt;td style="text-align: right">2.145&lt;/td>
&lt;td style="text-align: right">2.269&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 10:4x&lt;/td>
&lt;td>a11&lt;/td>
&lt;td style="text-align: right">2.192×&lt;/td>
&lt;td style="text-align: right">-0.014&lt;/td>
&lt;td style="text-align: right">2.125&lt;/td>
&lt;td style="text-align: right">2.261&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 11:2x&lt;/td>
&lt;td>a18&lt;/td>
&lt;td style="text-align: right">2.187×&lt;/td>
&lt;td style="text-align: right">-0.004&lt;/td>
&lt;td style="text-align: right">2.137&lt;/td>
&lt;td style="text-align: right">2.239&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 19:38&lt;/td>
&lt;td>a19&lt;/td>
&lt;td style="text-align: right">2.215×&lt;/td>
&lt;td style="text-align: right">+0.028&lt;/td>
&lt;td style="text-align: right">2.129&lt;/td>
&lt;td style="text-align: right">2.304&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-11-01 15:17&lt;/td>
&lt;td>a21&lt;/td>
&lt;td style="text-align: right">2.225×&lt;/td>
&lt;td style="text-align: right">+0.010&lt;/td>
&lt;td style="text-align: right">2.135&lt;/td>
&lt;td style="text-align: right">2.319&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-11-01 16:15&lt;/td>
&lt;td>a22&lt;/td>
&lt;td style="text-align: right">2.232×&lt;/td>
&lt;td style="text-align: right">+0.006&lt;/td>
&lt;td style="text-align: right">2.158&lt;/td>
&lt;td style="text-align: right">2.309&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-11-01 19:30&lt;/td>
&lt;td>a23w&lt;/td>
&lt;td style="text-align: right">2.196×&lt;/td>
&lt;td style="text-align: right">-0.036&lt;/td>
&lt;td style="text-align: right">2.124&lt;/td>
&lt;td style="text-align: right">2.270&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-11-02 03:34&lt;/td>
&lt;td>a22d resubmit&lt;/td>
&lt;td style="text-align: right">2.243×&lt;/td>
&lt;td style="text-align: right">+0.047&lt;/td>
&lt;td style="text-align: right">2.174&lt;/td>
&lt;td style="text-align: right">2.314&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;strong>Attention decode&lt;/strong> (10 runs)&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Date (UTC)&lt;/th>
&lt;th>Step&lt;/th>
&lt;th style="text-align: right">Score&lt;/th>
&lt;th style="text-align: right">Change&lt;/th>
&lt;th style="text-align: right">Decode half (D)&lt;/th>
&lt;th style="text-align: right">Verify half (V)&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>2026-10-25 22:05&lt;/td>
&lt;td>v2&lt;/td>
&lt;td style="text-align: right">1.524×&lt;/td>
&lt;td style="text-align: right">&lt;/td>
&lt;td style="text-align: right">1.655&lt;/td>
&lt;td style="text-align: right">1.404&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:12&lt;/td>
&lt;td>v3&lt;/td>
&lt;td style="text-align: right">1.669×&lt;/td>
&lt;td style="text-align: right">+0.144&lt;/td>
&lt;td style="text-align: right">1.877&lt;/td>
&lt;td style="text-align: right">1.484&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:32&lt;/td>
&lt;td>v4&lt;/td>
&lt;td style="text-align: right">1.725×&lt;/td>
&lt;td style="text-align: right">+0.056&lt;/td>
&lt;td style="text-align: right">1.967&lt;/td>
&lt;td style="text-align: right">1.513&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:03&lt;/td>
&lt;td>v5s&lt;/td>
&lt;td style="text-align: right">1.800×&lt;/td>
&lt;td style="text-align: right">+0.075&lt;/td>
&lt;td style="text-align: right">2.051&lt;/td>
&lt;td style="text-align: right">1.580&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:17&lt;/td>
&lt;td>v6s&lt;/td>
&lt;td style="text-align: right">1.831×&lt;/td>
&lt;td style="text-align: right">+0.031&lt;/td>
&lt;td style="text-align: right">2.093&lt;/td>
&lt;td style="text-align: right">1.603&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 03:17&lt;/td>
&lt;td>v7c&lt;/td>
&lt;td style="text-align: right">1.853×&lt;/td>
&lt;td style="text-align: right">+0.021&lt;/td>
&lt;td style="text-align: right">2.104&lt;/td>
&lt;td style="text-align: right">1.631&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 03:17&lt;/td>
&lt;td>v7d_robust&lt;/td>
&lt;td style="text-align: right">1.885×&lt;/td>
&lt;td style="text-align: right">+0.032&lt;/td>
&lt;td style="text-align: right">2.148&lt;/td>
&lt;td style="text-align: right">1.653&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 03:17&lt;/td>
&lt;td>v7e_near&lt;/td>
&lt;td style="text-align: right">1.854×&lt;/td>
&lt;td style="text-align: right">-0.030&lt;/td>
&lt;td style="text-align: right">2.117&lt;/td>
&lt;td style="text-align: right">1.625&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 07:21&lt;/td>
&lt;td>v7h&lt;/td>
&lt;td style="text-align: right">1.893×&lt;/td>
&lt;td style="text-align: right">+0.038&lt;/td>
&lt;td style="text-align: right">2.164&lt;/td>
&lt;td style="text-align: right">1.655&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 04:54&lt;/td>
&lt;td>s1&lt;/td>
&lt;td style="text-align: right">2.429×&lt;/td>
&lt;td style="text-align: right">+0.536&lt;/td>
&lt;td style="text-align: right">2.814&lt;/td>
&lt;td style="text-align: right">2.096&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;strong>MoE w4a16&lt;/strong> (16 runs)&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Date (UTC)&lt;/th>
&lt;th>Step&lt;/th>
&lt;th style="text-align: right">Score&lt;/th>
&lt;th style="text-align: right">Change&lt;/th>
&lt;th style="text-align: right">Decode half (D)&lt;/th>
&lt;th style="text-align: right">Verify half (V)&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>2026-10-24 07:58&lt;/td>
&lt;td>M4&lt;/td>
&lt;td style="text-align: right">1.105×&lt;/td>
&lt;td style="text-align: right">&lt;/td>
&lt;td style="text-align: right">1.000&lt;/td>
&lt;td style="text-align: right">1.221&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-24 08:27&lt;/td>
&lt;td>v5&lt;/td>
&lt;td style="text-align: right">1.265×&lt;/td>
&lt;td style="text-align: right">+0.160&lt;/td>
&lt;td style="text-align: right">1.217&lt;/td>
&lt;td style="text-align: right">1.315&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 19:37&lt;/td>
&lt;td>v8&lt;/td>
&lt;td style="text-align: right">1.425×&lt;/td>
&lt;td style="text-align: right">+0.160&lt;/td>
&lt;td style="text-align: right">1.282&lt;/td>
&lt;td style="text-align: right">1.584&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 21:15&lt;/td>
&lt;td>v9&lt;/td>
&lt;td style="text-align: right">1.453×&lt;/td>
&lt;td style="text-align: right">+0.027&lt;/td>
&lt;td style="text-align: right">1.315&lt;/td>
&lt;td style="text-align: right">1.604&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 21:59&lt;/td>
&lt;td>v10&lt;/td>
&lt;td style="text-align: right">1.451×&lt;/td>
&lt;td style="text-align: right">-0.002&lt;/td>
&lt;td style="text-align: right">1.322&lt;/td>
&lt;td style="text-align: right">1.592&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-25 22:21&lt;/td>
&lt;td>v11&lt;/td>
&lt;td style="text-align: right">1.465×&lt;/td>
&lt;td style="text-align: right">+0.014&lt;/td>
&lt;td style="text-align: right">1.344&lt;/td>
&lt;td style="text-align: right">1.596&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 01:40&lt;/td>
&lt;td>v12&lt;/td>
&lt;td style="text-align: right">1.648×&lt;/td>
&lt;td style="text-align: right">+0.184&lt;/td>
&lt;td style="text-align: right">1.511&lt;/td>
&lt;td style="text-align: right">1.798&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 01:50&lt;/td>
&lt;td>v13&lt;/td>
&lt;td style="text-align: right">1.752×&lt;/td>
&lt;td style="text-align: right">+0.104&lt;/td>
&lt;td style="text-align: right">1.613&lt;/td>
&lt;td style="text-align: right">1.903&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:05&lt;/td>
&lt;td>v14&lt;/td>
&lt;td style="text-align: right">1.772×&lt;/td>
&lt;td style="text-align: right">+0.019&lt;/td>
&lt;td style="text-align: right">1.644&lt;/td>
&lt;td style="text-align: right">1.909&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:17&lt;/td>
&lt;td>v15b&lt;/td>
&lt;td style="text-align: right">1.795×&lt;/td>
&lt;td style="text-align: right">+0.024&lt;/td>
&lt;td style="text-align: right">1.667&lt;/td>
&lt;td style="text-align: right">1.933&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:22&lt;/td>
&lt;td>v16&lt;/td>
&lt;td style="text-align: right">1.779×&lt;/td>
&lt;td style="text-align: right">-0.016&lt;/td>
&lt;td style="text-align: right">1.658&lt;/td>
&lt;td style="text-align: right">1.909&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 02:30&lt;/td>
&lt;td>v17&lt;/td>
&lt;td style="text-align: right">1.772×&lt;/td>
&lt;td style="text-align: right">-0.007&lt;/td>
&lt;td style="text-align: right">1.647&lt;/td>
&lt;td style="text-align: right">1.908&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 20:33&lt;/td>
&lt;td>w4&lt;/td>
&lt;td style="text-align: right">2.140×&lt;/td>
&lt;td style="text-align: right">+0.368&lt;/td>
&lt;td style="text-align: right">1.990&lt;/td>
&lt;td style="text-align: right">2.302&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 07:21&lt;/td>
&lt;td>s3&lt;/td>
&lt;td style="text-align: right">2.370×&lt;/td>
&lt;td style="text-align: right">+0.230&lt;/td>
&lt;td style="text-align: right">2.177&lt;/td>
&lt;td style="text-align: right">2.580&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 09:05&lt;/td>
&lt;td>s3e&lt;/td>
&lt;td style="text-align: right">2.403×&lt;/td>
&lt;td style="text-align: right">+0.033&lt;/td>
&lt;td style="text-align: right">2.194&lt;/td>
&lt;td style="text-align: right">2.632&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-11-03 02:10&lt;/td>
&lt;td>s3f&lt;/td>
&lt;td style="text-align: right">2.369×&lt;/td>
&lt;td style="text-align: right">-0.034&lt;/td>
&lt;td style="text-align: right">2.163&lt;/td>
&lt;td style="text-align: right">2.595&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;strong>BF16 projections&lt;/strong> (10 runs)&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Date (UTC)&lt;/th>
&lt;th>Step&lt;/th>
&lt;th style="text-align: right">Score&lt;/th>
&lt;th style="text-align: right">Change&lt;/th>
&lt;th style="text-align: right">Decode half (D)&lt;/th>
&lt;th style="text-align: right">Verify half (V)&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>2026-10-26 00:40&lt;/td>
&lt;td>v4b&lt;/td>
&lt;td style="text-align: right">1.154×&lt;/td>
&lt;td style="text-align: right">&lt;/td>
&lt;td style="text-align: right">1.129&lt;/td>
&lt;td style="text-align: right">1.180&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 01:12&lt;/td>
&lt;td>v5a&lt;/td>
&lt;td style="text-align: right">1.183×&lt;/td>
&lt;td style="text-align: right">+0.029&lt;/td>
&lt;td style="text-align: right">1.160&lt;/td>
&lt;td style="text-align: right">1.207&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 23:26&lt;/td>
&lt;td>v7a&lt;/td>
&lt;td style="text-align: right">1.236×&lt;/td>
&lt;td style="text-align: right">+0.052&lt;/td>
&lt;td style="text-align: right">1.245&lt;/td>
&lt;td style="text-align: right">1.227&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 23:32&lt;/td>
&lt;td>v7c&lt;/td>
&lt;td style="text-align: right">1.250×&lt;/td>
&lt;td style="text-align: right">+0.014&lt;/td>
&lt;td style="text-align: right">1.253&lt;/td>
&lt;td style="text-align: right">1.246&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 02:30&lt;/td>
&lt;td>v7d_robust&lt;/td>
&lt;td style="text-align: right">1.249×&lt;/td>
&lt;td style="text-align: right">-0.000&lt;/td>
&lt;td style="text-align: right">1.253&lt;/td>
&lt;td style="text-align: right">1.246&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 23:32&lt;/td>
&lt;td>v8b&lt;/td>
&lt;td style="text-align: right">1.264×&lt;/td>
&lt;td style="text-align: right">+0.015&lt;/td>
&lt;td style="text-align: right">1.278&lt;/td>
&lt;td style="text-align: right">1.251&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 00:54&lt;/td>
&lt;td>v8c&lt;/td>
&lt;td style="text-align: right">1.296×&lt;/td>
&lt;td style="text-align: right">+0.032&lt;/td>
&lt;td style="text-align: right">1.336&lt;/td>
&lt;td style="text-align: right">1.256&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 12:36&lt;/td>
&lt;td>v10e&lt;/td>
&lt;td style="text-align: right">1.387×&lt;/td>
&lt;td style="text-align: right">+0.091&lt;/td>
&lt;td style="text-align: right">1.455&lt;/td>
&lt;td style="text-align: right">1.322&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-28 14:01&lt;/td>
&lt;td>v10h&lt;/td>
&lt;td style="text-align: right">1.392×&lt;/td>
&lt;td style="text-align: right">+0.005&lt;/td>
&lt;td style="text-align: right">1.431&lt;/td>
&lt;td style="text-align: right">1.355&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-31 09:4x&lt;/td>
&lt;td>s4&lt;/td>
&lt;td style="text-align: right">1.397×&lt;/td>
&lt;td style="text-align: right">+0.005&lt;/td>
&lt;td style="text-align: right">1.472&lt;/td>
&lt;td style="text-align: right">1.326&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;strong>Prefill layer&lt;/strong> (5 runs)&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Date (UTC)&lt;/th>
&lt;th>Step&lt;/th>
&lt;th style="text-align: right">Score&lt;/th>
&lt;th style="text-align: right">Change&lt;/th>
&lt;th style="text-align: right">Later turns&lt;/th>
&lt;th style="text-align: right">First turn&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>2026-10-26 03:29&lt;/td>
&lt;td>v1c&lt;/td>
&lt;td style="text-align: right">1.005×&lt;/td>
&lt;td style="text-align: right">&lt;/td>
&lt;td style="text-align: right">1.003&lt;/td>
&lt;td style="text-align: right">1.010&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 03:58&lt;/td>
&lt;td>v2&lt;/td>
&lt;td style="text-align: right">1.205×&lt;/td>
&lt;td style="text-align: right">+0.200&lt;/td>
&lt;td style="text-align: right">1.115&lt;/td>
&lt;td style="text-align: right">1.407&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 04:42&lt;/td>
&lt;td>v3&lt;/td>
&lt;td style="text-align: right">1.251×&lt;/td>
&lt;td style="text-align: right">+0.046&lt;/td>
&lt;td style="text-align: right">1.182&lt;/td>
&lt;td style="text-align: right">1.399&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-26 15:22&lt;/td>
&lt;td>v4&lt;/td>
&lt;td style="text-align: right">1.340×&lt;/td>
&lt;td style="text-align: right">+0.090&lt;/td>
&lt;td style="text-align: right">1.274&lt;/td>
&lt;td style="text-align: right">1.483&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2026-10-27 22:55&lt;/td>
&lt;td>v4c_hum256&lt;/td>
&lt;td style="text-align: right">1.355×&lt;/td>
&lt;td style="text-align: right">+0.015&lt;/td>
&lt;td style="text-align: right">1.294&lt;/td>
&lt;td style="text-align: right">1.485&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;em>Change is against the previous passed run on the same board. Steps are my internal version names; jumps came from the changes described below. Only runs with saved result files are listed, so my best MoE (2.403×) and BF16 (1.397×) runs are missing. Dates are when each result was logged, not when the run finished, which is why c30 shows Oct 30 even though it finished on Oct 29.&lt;/em>&lt;/p>
&lt;/details>
&lt;h2 id="what-the-process-caught">What the process caught&lt;/h2>
&lt;p>The agents made plenty of mistakes. The system around them caught most of them, and cheaply. The two habits that did the most work were the &lt;strong>control copy&lt;/strong> in every benchmark batch (it caught a +0.4–0.8% position bias that would have looked like a win) and &lt;strong>kill criteria&lt;/strong> written before each run.&lt;/p>
&lt;details>
&lt;summary>&lt;b>Mistakes and what caught them&lt;/b>&lt;/summary>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Mistake&lt;/th>
&lt;th>Caught by&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Local models said &amp;ldquo;done&amp;rdquo; with nothing written, or mocked missing inputs&lt;/td>
&lt;td>the review ledger: every local-model output is read before use&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A false &amp;ldquo;num_stages does nothing&amp;rdquo; finding, from GPU-less stats compiled without 16-byte alignment&lt;/td>
&lt;td>a second lane, cross-checking&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>The MoE kernel&amp;rsquo;s scales stored as exact bf16, against the contest&amp;rsquo;s precision rule&lt;/td>
&lt;td>another lane, then a rules lint on every candidate&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A &amp;ldquo;novel&amp;rdquo; upstream Triton fix that upstream had already rejected&lt;/td>
&lt;td>an independent model review&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A misread accuracy margin that nearly dropped FP8 queries&lt;/td>
&lt;td>an $0.84 multi-seed check: 104/104 pass&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A shell word-splitting bug that made a merge check compare nothing&lt;/td>
&lt;td>shell-free GPU jobs&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Position bias in packed A/B timing (+0.4–0.8% on an identical file)&lt;/td>
&lt;td>a byte-identical control copy in every batch&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A tool rewrite that crashed SGLang&amp;rsquo;s baseline and looked like my bug&lt;/td>
&lt;td>re-running the old harness first&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>A packed job silently skipping a build step after 235 s (~−2.7%)&lt;/td>
&lt;td>grepping logs for &lt;code>skipped-budget&lt;/code> after three batches were affected&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;/details>
&lt;h3 id="the-megakernel-i-didnt-build">The megakernel I didn&amp;rsquo;t build&lt;/h3>
&lt;p>A &lt;a href="https://hazyresearch.stanford.edu/blog/2025-05-27-no-bubbles">&lt;em>megakernel&lt;/em>&lt;/a> fuses a whole layer into one long-running kernel, so the GPU never idles between launches. Three separate performance models, built by the agents, priced it at up to 1.25× for the decode layer. It&amp;rsquo;s the kind of idea that eats a week.&lt;/p>
&lt;p>Seventeen minutes after the estimate, a $1.40 probe tested its central assumption: that the GPU sat idle between kernels, and one kernel could win that time back. Kernel boundaries weren&amp;rsquo;t costing anything I could remove: the four kernels run separately took 45.5 µs, and the same four inside one persistent kernel took 49.1 µs. Even with no synchronization at all, the persistent kernel took 52.7 µs. A second lane&amp;rsquo;s first fused piece came out bitwise correct, and slower. I closed the line about 5.5 hours after it opened, at $2.35 of GPU, with no judged run spent. All three were wrong in the same direction, and a cheap probe beat three careful estimates.&lt;/p>
&lt;details>
&lt;summary>&lt;b>The megakernel line, step by step&lt;/b>&lt;/summary>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Step&lt;/th>
&lt;th>Result&lt;/th>
&lt;th>Cost&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Three agent-built estimates of the gain&lt;/td>
&lt;td>up to 1.25× on the decode layer&lt;/td>
&lt;td>none (CPU)&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Probe: four kernels separately vs. inside one persistent kernel&lt;/td>
&lt;td>45.5 µs vs 49.1 µs (52.7 µs with no sync): no gap to remove&lt;/td>
&lt;td>$1.40&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Second lane: first fused piece&lt;/td>
&lt;td>bitwise correct, slower than the pieces it replaced&lt;/td>
&lt;td>~$0.95&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Decision&lt;/td>
&lt;td>closed against the pre-written kill criteria; no judged run used&lt;/td>
&lt;td>$2.35 total&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;/details>
&lt;p>The megakernel was one node on a bigger map. On Oct 25 the agents drew every lever they could see, with odds and a multiplier for each. Most of it shipped, for 0–6% a step. The two biggest jumps, +10.5% and +14%, came from public kernels the map didn&amp;rsquo;t have.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/plan-dag.html?embed" title="The plan, and what happened" loading="lazy"
style="width:100%;height:900px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 7. The Oct 25 experiment map: odds each lever works, the layer multiplier if it does, and days of work. Switch to What happened to colour each node by its fate and show the five wins that weren&amp;#39;t on the map. Click any node for details. &lt;a href="https://nextdoorhacker.com/okc/plan-dag.html?embed">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;h2 id="seeing-the-work">Seeing the work&lt;/h2>
&lt;p>With a dozen lanes and hundreds of GPU batches, I couldn&amp;rsquo;t read every log. I steered by pictures the agents generated from the repo&amp;rsquo;s own data, rebuilt as the data changed. Their numbers come from data files and scripts rather than being typed in, and tests fail when a picture and its source disagree.&lt;/p>
&lt;p>The best example: a &lt;strong>time lapse&lt;/strong> of every version of the decode layer, replayed the way the judge times it. On day 2 it changed two beliefs at once. The gaps between kernel launches were 0.1–0.6 µs, not ~25 µs, so &amp;ldquo;fewer kernels&amp;rdquo; wasn&amp;rsquo;t a lever by itself. And cuBLAS was running the 8-token router as an FP32 matrix multiply that took 34 µs alone. Fixing that took the verify-1 load from 1.27× to 1.48×. I found it by looking, not by planning.&lt;/p>
&lt;figure class="interactive" style="margin:1.5rem 0">
&lt;iframe src="https://nextdoorhacker.com/okc/timelapse.html?embed&amp;amp;frame=SGLang" title="Layer time lapse" loading="lazy"
style="width:100%;height:900px;border:1px solid var(--border,#ddd);border-radius:8px">&lt;/iframe>&lt;figcaption style="font-size:.85rem;color:var(--secondary,#666)">Figure 8. Every decode-layer version from SGLang to a3, replayed as a CUDA graph on one H200. The bars split each load&amp;#39;s time per layer into five kinds of work, on SGLang&amp;#39;s scale; the timeline below shows every kernel in one replay of the chosen shape. Press Play, or pick v8 and verify 1 to see the router fix. &lt;a href="https://nextdoorhacker.com/okc/timelapse.html?embed&amp;amp;frame=SGLang">Open full screen&lt;/a>&lt;/figcaption>
&lt;/figure>
&lt;details>
&lt;summary>&lt;b>All the instruments, and what each one changed&lt;/b>&lt;/summary>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Picture&lt;/th>
&lt;th>What it shows&lt;/th>
&lt;th>What it changed&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Layer time lapse&lt;/td>
&lt;td>Every decode-layer version replayed as a CUDA graph on all 12 shapes, with a kernel timeline per shape&lt;/td>
&lt;td>Launch gaps were tiny; the router was a 34 µs FP32 matmul. The fix lifted verify 1 from 1.27× to 1.48×.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Layer heatmap&lt;/td>
&lt;td>Versions × stages, coloured by bandwidth reached against 4.8 TB/s&lt;/td>
&lt;td>Separated &amp;ldquo;fast&amp;rdquo; from &amp;ldquo;at the floor&amp;rdquo;.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Board map&lt;/td>
&lt;td>How the five boards relate, and the megakernel&amp;rsquo;s predicted-vs-measured chart&lt;/td>
&lt;td>Showed changes flowing from component boards into the layer. In hindsight, it also held the attention signal I missed.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Daily dashboard&lt;/td>
&lt;td>Rank, scores with confidence intervals, the top-50/100 lines, spend, run counts&lt;/td>
&lt;td>Republished after each judged run and filled by script, so its numbers came from the data, not from memory.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Top-100 plan&lt;/td>
&lt;td>Each remaining lever&amp;rsquo;s estimated gain against a #100 line rising ~0.03×/day&lt;/td>
&lt;td>Most levers measured +0.3% or less, which is how I knew to stop on Nov 1.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Roadmap Monte Carlo&lt;/td>
&lt;td>The odds of each target if every lever landed with its estimated probability&lt;/td>
&lt;td>Said &amp;ldquo;~0% for top 100 on the known levers&amp;rdquo; on day 2. Right, until public code changed the base.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Explainer films&lt;/td>
&lt;td>One layer&amp;rsquo;s memory traffic on a drawn H200; one prompt followed to the bytes of a weight block&lt;/td>
&lt;td>Explaining, not steering. Every narrated number comes from a facts file under test.&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Kernel Climb (figures 2–5)&lt;/td>
&lt;td>Built after the close: each board&amp;rsquo;s exact score map with every run, and the hardware limit&lt;/td>
&lt;td>Showed the climb was many small steps plus a few big jumps, and how far below the hardware the field stopped.&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;/details>
&lt;h2 id="wrong-turns">Wrong turns&lt;/h2>
&lt;ol>
&lt;li>&lt;strong>A rule I imposed on myself kept me on a slower base.&lt;/strong> I required bit-identical replays, which the judge never asked for. It excluded a public base (&lt;a href="https://github.com/alygg77/okc-muse-spark-kernels">alygg77&amp;rsquo;s&lt;/a>, in &lt;a href="https://github.com/vjeux/okc-2026">vjeux&amp;rsquo;s&lt;/a> lineage, Apache-2.0) that I had already measured as faster on Oct 28. When I relaxed the rule on Oct 30, one run moved me from 1.896× to 2.163×, and from about #338 to #155. Agents follow the rules you give them very faithfully, including the ones you shouldn&amp;rsquo;t have given.&lt;/li>
&lt;li>&lt;strong>I optimized the part I understood best, not the part with the most room.&lt;/strong> In the last days, effort went to the MoE, already within 1.1–1.8× of its floor; the last attempts read 1.000×. Meanwhile the attention inside my layer sat at 2.43× on its own board, against leaders above 2.9×. A rough retro against &lt;a href="https://github.com/gauravjain14/muse-okc-submissions">gauravjain14&amp;rsquo;s code&lt;/a> (#5 overall, made public after the close), with judge and local timings mixed, put their attention at about +1.5–2% inside my layer: close to the top-100 line, though I never measured it in place. The signal was on a leaderboard I was already watching.&lt;/li>
&lt;li>&lt;strong>My benchmark didn&amp;rsquo;t reproduce the judge&amp;rsquo;s cold start.&lt;/strong> The judge frees and reallocates memory between repetitions, so its first repetition runs with cold TLBs as well as a flushed cache. Mine didn&amp;rsquo;t.&lt;/li>
&lt;li>&lt;strong>The megakernel&lt;/strong> took 5.5 hours of elapsed time, even though the probe that undercut it came 17 minutes in and the whole line cost $2.35 of GPU.&lt;/li>
&lt;li>&lt;strong>Colab as a free GPU&lt;/strong> was a dead end on day 1.&lt;/li>
&lt;/ol>
&lt;p>&lt;strong>The reuse decision.&lt;/strong> On Nov 1, another team&amp;rsquo;s public file was about 0.4% faster than my lineage. Submitting it as is would probably have bought rank sooner. I said no, and the agents ported its pieces one at a time instead: each piece got an A/B and a credit, four of the eight measured nothing and were dropped, and I ended about 0.4% behind what that file would have scored. It&amp;rsquo;s the clearest example I have of a judgment call the agents wouldn&amp;rsquo;t have made on their own.&lt;/p>
&lt;h2 id="notes-on-the-contest-itself">Notes on the contest itself&lt;/h2>
&lt;ul>
&lt;li>&lt;strong>Ten runs a day, and I only let the agents use five.&lt;/strong> The contest allowed ten scored runs per problem per day. I didn&amp;rsquo;t want to run out of chances to submit, so the agents could send five on their own and the rest needed my approval. The judge keeps each team&amp;rsquo;s best row, and its own intervals on my runs ranged from about ±1.5% to ±3.4%, bigger than most late improvements. So resubmitting rewards lucky draws. I didn&amp;rsquo;t fish for them: my one resubmission of the same kernels read 2.243× against the original&amp;rsquo;s 2.232×.&lt;/li>
&lt;li>&lt;strong>A correctness check can pass two different answers.&lt;/strong> Two of my own entries pass every check the contest runs, yet their CUDA-graph path and their eager path pick different top-8 experts at 128–256 tokens, because the graph path uses a different matrix-vector kernel for the router&amp;rsquo;s input than the eager path does. In synthetic 128- and 256-token cases, the largest relative L2 difference between the two paths&amp;rsquo; layer outputs was 13–17%. The contest allows routing to differ where scores nearly tie: &amp;ldquo;Each expert you pick must score (sigmoid + router_bias) within 0.05 of the reference&amp;rsquo;s 8th best,&amp;rdquo; plus a recall rule and count floors. Then, crucially, &amp;ldquo;the MoE output is then checked with the experts you picked.&amp;rdquo; That 0.05 is about ten times the median gap between the 8th and 9th expert scores in those same synthetic cases (5.3e-3), so there it admits many different expert sets. A trained router may have wider margins; I didn&amp;rsquo;t measure one. And once a different set of experts is allowed, the output check compares against those same experts, so the change never counts as error. The mechanism isn&amp;rsquo;t new (&lt;a href="https://arxiv.org/abs/2202.08906">ST-MoE&lt;/a>; &lt;a href="https://github.com/vllm-project/vllm/issues/61053">vLLM #61053&lt;/a>; graph-vs-eager divergence in &lt;a href="https://github.com/vllm-project/vllm/issues/55238">vLLM #55238&lt;/a>). What I&amp;rsquo;d argue is narrower: a tolerance-based check can&amp;rsquo;t see top-k flips, and comparing an entry&amp;rsquo;s graph and eager paths against each other is a cheap check that can. I haven&amp;rsquo;t measured how often this happens in practice: that needs per-token rates, a trained model&amp;rsquo;s router margins, and a control that pins the routes to see how much of the drift is routing.&lt;/li>
&lt;li>&lt;strong>Credit where it&amp;rsquo;s due.&lt;/strong> Most of my biggest single jumps came from reading other people&amp;rsquo;s public code: &lt;a href="https://github.com/alygg77/okc-muse-spark-kernels">alygg77&lt;/a> and &lt;a href="https://github.com/vjeux/okc-2026">vjeux&lt;/a> (Apache-2.0), &lt;a href="https://github.com/operatorstack/open-kernel-challenge-solutions">bigboateng&lt;/a> (MIT, ideas only), &lt;a href="https://github.com/keell0renz/muse-open-kernel-challenge">keell0renz&lt;/a>, &lt;a href="https://github.com/chenwainuo/okc">chenwainuo&lt;/a> and &lt;a href="https://github.com/ZhimaoL/sglang">ZhimaoL&lt;/a> (per-piece ports), and &lt;a href="https://github.com/sgl-project/sglang">SGLang&amp;rsquo;s&lt;/a> own kernels. &lt;a href="https://github.com/gauravjain14/muse-okc-submissions">gauravjain14&amp;rsquo;s repo&lt;/a>, public after the close, is a model of how to keep records: profile → diagnosis → change → judged result, next to a dead-ends file with numbers.&lt;/li>
&lt;/ul>
&lt;h2 id="the-serving-track">The serving track&lt;/h2>
&lt;p>The second track opened on Nov 12 and worked completely differently: you submitted a patch to SGLang and the judge served the whole model on a full 8×H200 node. It closed on Nov 19 with my best at 2.315×, and I&amp;rsquo;ll write it up separately. The H200 shortage was much worse there: no cloud would give me an 8×H200 node on demand, a Modal request sat in a queue for 45 minutes without one, and getting a working box on AWS took most of the first day. The cost controls got stricter there, because an 8×H200 node costs about $30 an hour: one long packed run per day at most, and actual billing pulled every few hours instead of estimates.&lt;/p>
&lt;h2 id="what-i-learned">What I learned&lt;/h2>
&lt;p>What I&amp;rsquo;d say from here:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Agents changed what my time was worth.&lt;/strong> My hours went to budgets, rules, priorities and stopping, not code. That&amp;rsquo;s what made it possible with a day job and kids.&lt;/li>
&lt;li>&lt;strong>Different models catch different things.&lt;/strong> The reviewer from another lab was worth more than another lane. The local models I could run earned their place as readers; editing may need a bigger model than I had.&lt;/li>
&lt;li>&lt;strong>Cheap feedback beats good predictions.&lt;/strong> Three models priced the megakernel; a $1.40 probe undercut its central assumption within 17 minutes, and I closed the line for $2.35 in total. The CPU pre-flight and same-run A/Bs on Modal did more for the score than any estimate.&lt;/li>
&lt;li>&lt;strong>Put the budget in the launcher, and keep a hard backstop.&lt;/strong> The launcher stopped new jobs near the cap, the provider&amp;rsquo;s limit stopped everything, and every exception was written down with its reason.&lt;/li>
&lt;li>&lt;strong>Sessions are staff, not chats.&lt;/strong> Give each one a narrow role, put its rules and state in files, and context resets turn into shift changes.&lt;/li>
&lt;li>&lt;strong>Agents are faithful to the rules you give them, including bad ones.&lt;/strong> The most expensive mistake of the contest was a rule I invented that the judge never asked for.&lt;/li>
&lt;/ul>
&lt;h2 id="in-retrospect">In retrospect&lt;/h2>
&lt;p>What I&amp;rsquo;d do differently next time:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Compute the hardware limit on day 1, for every component.&lt;/strong> The share of each limit already reached tells you where the room is. It would have pointed at attention (36% of its limit, against 55% for the MoE) long before the close.&lt;/li>
&lt;li>&lt;strong>Audit my own rules against the judge&amp;rsquo;s in the first week.&lt;/strong> The bit-identical rule kept me off a faster public base from Oct 28 to Oct 30, and dropping it was worth about 180 places. A one-hour comparison of my rules with the contest&amp;rsquo;s would have caught it.&lt;/li>
&lt;li>&lt;strong>Test setup the way the judge does.&lt;/strong> Cold caches, cold memory, fresh compile. Two of my measurement gaps were about warm state the judge never had.&lt;/li>
&lt;li>&lt;strong>Pack GPU work from the start.&lt;/strong> Packed jobs with a control copy came after a day of paying for idle GPUs. They should have been the default.&lt;/li>
&lt;li>&lt;strong>Start with roles and written state, not one long chat.&lt;/strong> It took me until the seventh session to run long-lived sessions with clear jobs. It should have been the setup on day 1.&lt;/li>
&lt;li>&lt;strong>Have two GPU providers from day 1.&lt;/strong> Modal running out of H200s cost an afternoon. Runpod as a ready second backend made it a non-event afterwards.&lt;/li>
&lt;li>&lt;strong>Watch the boards I wasn&amp;rsquo;t working on.&lt;/strong> The signal I missed was on a leaderboard I already had open.&lt;/li>
&lt;/ul>
&lt;h2 id="references-and-credits">References and credits&lt;/h2>
&lt;p>&lt;strong>The contest&lt;/strong>&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://openkernelchallenge.dev">Open Kernel Challenge&lt;/a>: rules, boards and leaderboards.&lt;/li>
&lt;li>&lt;a href="https://github.com/sgl-project/sglang">SGLang&lt;/a>: the serving engine every score was measured against.&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>Public code I learned from or ported pieces of (credited in the code)&lt;/strong>&lt;/p>
&lt;ul>
&lt;li>alygg77, &lt;a href="https://github.com/alygg77/okc-muse-spark-kernels">okc-muse-spark-kernels&lt;/a> (Apache-2.0), in the lineage of vjeux, &lt;a href="https://github.com/vjeux/okc-2026">okc-2026&lt;/a>.&lt;/li>
&lt;li>keell0renz, &lt;a href="https://github.com/keell0renz/muse-open-kernel-challenge">muse-open-kernel-challenge&lt;/a>.&lt;/li>
&lt;li>chenwainuo, &lt;a href="https://github.com/chenwainuo/okc">okc&lt;/a>.&lt;/li>
&lt;li>ZhimaoL, &lt;a href="https://github.com/ZhimaoL/sglang">sglang fork&lt;/a>.&lt;/li>
&lt;li>&lt;a href="https://github.com/operatorstack/open-kernel-challenge-solutions">bigboateng&amp;rsquo;s README&lt;/a> (MIT; ideas only).&lt;/li>
&lt;li>gauravjain14, &lt;a href="https://github.com/gauravjain14/muse-okc-submissions">muse-okc-submissions&lt;/a> (Apache-2.0; public after the close, used only for the retrospective).&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>MoE routing and correctness checks&lt;/strong>&lt;/p>
&lt;ul>
&lt;li>Zoph et al., &lt;a href="https://arxiv.org/abs/2202.08906">ST-MoE: Designing Stable and Transferable Sparse Expert Models&lt;/a>.&lt;/li>
&lt;li>vLLM issue &lt;a href="https://github.com/vllm-project/vllm/issues/61053">#61053&lt;/a>: expert-routing flips from per-rank kernel autotuning.&lt;/li>
&lt;li>vLLM issue &lt;a href="https://github.com/vllm-project/vllm/issues/55238">#55238&lt;/a>: CUDA-graph vs eager divergence on an MoE model.&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/abs/2607.28097">arXiv 2607.28097&lt;/a> (Zhu et al.) and &lt;a href="https://arxiv.org/abs/2510.11370">R3, arXiv 2510.11370&lt;/a>: routing differences across reduction orders and engines.&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>Tools and infrastructure&lt;/strong>&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://claude.com/claude-code">Claude Code&lt;/a>, &lt;a href="https://modal.com">Modal&lt;/a>, &lt;a href="https://www.runpod.io">Runpod&lt;/a>, &lt;a href="https://github.com/ggml-org/llama.cpp">llama.cpp&lt;/a>, &lt;a href="https://linear.app">Linear&lt;/a>.&lt;/li>
&lt;li>&lt;a href="https://research.google.com/colaboratory/faq.html">Colab FAQ and terms&lt;/a>.&lt;/li>
&lt;/ul></description></item></channel></rss>