october 2026 · 4 min

A 200 Is Not a Pass

Cloudflare's postmortem on the GLM-5.3 exposure in Workers AI is about a KV cache, but the bug is older than LLMs: an error path that returns success. A prompt that should have been rejected was turned into a valid-looking request, the model answered it from another user's leftover memory, and the client got a 200. Every check Cloudflare ran was looking at the status code, so every check passed.

What Happened

GLM-5.3 advertised a context window of 1,048,576 tokens. The KV cache pool on Cloudflare's prefill GPUs "only had room for 681,280 tokens." Between August 27 and September 9, prompts that landed in that gap went through an error path in SGLang's prefill/decode split.

The prefill side saw the prompt didn't fit and marked it rejected. But instead of retiring it, it "replaced [it] with a one-token placeholder." Decode had already reserved room for the full prompt, received one token of KV, and attended over the rest of its slots anyway -- "stale cache pages that still held data from earlier requests." In a shared pool, earlier requests are other customers. Then, as SGLang issue #30233 describes it, a later transfer-success check overwrote the abort status, so the request finished normally. In Cloudflare's words: "The client got HTTP 200 instead of HTTP 400."

155 responses were affected, all to prompts over 681,280 tokens. A researcher reported it through HackerOne on September 8; Cloudflare took the model offline within 40 minutes of declaring an incident on the 9th. Cloudflare says it "found no evidence of deliberate exploitation."

Green, Because Green Was All It Checked

Cloudflare knew about the size mismatch before launch. It expected the engine to return an error for any prompt it couldn't process, its long-context tests came back successful, and it read that as no problem: "successful HTTP status codes and a streaming body with no error frames." Their own diagnosis is precise: "we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence."

That's the class. A status code tells you the machine didn't complain. It doesn't tell you the part is right. When an engine's failure mode is "degrade quietly and keep going," a test that only checks for errors is measuring the one thing the bug never produces. And for a model it's worse than for most systems, because a fluent answer to the wrong context looks exactly like a fluent answer to the right one.

The Control

Two checks, and neither needs the bug to be understood in advance.

First, make the known mismatch impossible to deploy. A config where the advertised window is larger than the memory that has to hold it should fail before it serves a request:

def check_capacity(context_length: int, kv_pool_tokens: int) -> None:
    if context_length > kv_pool_tokens:
        raise SystemExit(
            f"context_length {context_length} exceeds KV pool "
            f"{kv_pool_tokens} tokens; refusing to start"
        )

check_capacity(1_048_576, 681_280)
# exits 1: context_length 1048576 exceeds KV pool 681280 tokens; refusing to start

This is close to what Cloudflare added: a check comparing KV pool size to context length before any disaggregated deployment change. It's also the mismatch they already knew about, written down as a gate instead of an assumption.

Second, test the output, not the envelope. Plant a marker the model can only know from this prompt -- a random string near the start of a prompt at the edge of the window -- and assert the answer contains it. Run it at the boundary on both sides: just under the advertised limit must answer from its own context, just over must come back as a 4xx. A canary like that doesn't need a coherence judge. If the answer doesn't know the marker, the context wasn't yours.

The Tradeoff

Output tests at a million tokens cost real GPU time, and they're slower and noisier than status checks, which is exactly why teams stop at the status code. A startup invariant also turns a deploy that "worked" into one that refuses to start until someone fixes the capacity. Cloudflare's fix was a config change that freed enough GPU memory for a prefill pool of about 2.04M tokens; not every deployment will have that memory to spare. Rejecting at admission means some customers' long prompts now fail loudly. That's the point.

I think this is where the job is going. As more of the system is generated and assembled rather than hand-written, most of the engineering becomes QA: deciding what "correct" means for the part and checking that, not whether the machine raised a flag. If your long-context tests are green, open one and read what the model said.

← all posts