Here’s a fun thought to start your day: for about two weeks, a model hosted on Cloudflare’s shiny AI platform was occasionally answering people with bits of other people’s prompts. Yes, really. The Cloudflare Workers AI leak was quietly fixed on September 9, and the company only told the world about it on October 26, in a long and, to its credit, brutally honest post-mortem.
The affected model is @cf/zai-org/glm-5.3, the big GLM 5.3 from Z.ai. Between August 27 (launch day) and September 9, 155 responses went out containing data from someone else’s request. Every single one of them was a reply to a prompt longer than 681,280 tokens. Hold onto that number, it’s the whole story.
What the Cloudflare Workers AI leak actually was
Modern LLM serving splits the work in two: “prefill” nodes chew through your prompt and build the KV cache (basically the model’s short-term memory of what you wrote), then “decode” nodes generate the answer from that cache. Cloudflare runs this with SGLang, an open source inference engine, and shuffles the cache between GPUs over RDMA.
Now the mistake. GLM 5.3 was advertised with a 1,048,576-token context window, but the KV cache pool on the prefill GPUs only had room for 681,280 tokens. Cloudflare knew that. Their long-context tests came back with HTTP 200 and decent latency, so nobody worried. As they put it themselves, they “validated response codes and performance but not output coherence.” Translation: the server said “fine” and nobody read the actual answers.
Meanwhile, SGLang had a bug in its disaggregated mode. When a prompt didn’t fit, the engine flagged it as rejected and swapped in a one-token placeholder. But the admission queue only checked whether a request was finished, not whether it was rejected. So the rejected request kept flowing: one tiny page of cache got written, the decode node assumed the whole prompt had been transferred, and happily attended over up to seven stale pages still holding earlier users’ data. The client got a 200 instead of the 400 it deserved, plus a dose of somebody else’s conversation.
Two bugs, each harmless alone, together a data leak. A classic.
Thirteen days, then a six-hour fix
The report came from Gadi Evron of Knostic through HackerOne on September 8 at 23:11 UTC. The next morning Cloudflare reproduced it with two accounts in 25 minutes, declared an incident at 13:22, set the model to private at 14:00, found the root cause at 14:35, and had the thing rebuilt and back online by 19:40. That’s a good afternoon’s work, and I’ll gladly say so.
The fix is almost embarrassingly simple: the pool now holds about 2.04 million tokens, roughly twice the context length. “A prompt that fits the context window now always fits the pool.” They also patched SGLang so rejected requests die at the admission queue before any cache transfer, pushed the fixes upstream (they landed in SGLang 0.5.22, out October 19), and are working on per-page integrity tags and even formal verification of the RDMA transfer protocol. Slightly annoying detail: a very similar bug had been filed against SGLang as issue #30233 back on July 6, more than a month before launch.
Cloudflare says it found “no evidence of deliberate exploitation” and that no other Workers AI model was affected, because every other deployment had a pool bigger than its context window. GLM 5.3 was simply the one that got the mismatched config.
Why you should care even if you don’t use Cloudflare
Because this is the second time this year I’m writing about AI infrastructure doing something nobody tested for (remember the AGENTS.md hijack story?). The whole industry is stacking open source engines, custom RDMA plumbing and billion-token context windows on top of each other at sprint speed, and the test suite checks whether the HTTP status is green.
If you run SGLang yourself, Cloudflare’s advice is worth taping to the wall: measure pool size against context length on every node after any change, upgrade to 0.5.22 or later, and turn on corruption detection. If you’re just a customer sending sensitive stuff to a hosted model, maybe read the post-mortems before you read the launch blogs. The honest ones tell you a lot about who you’re dealing with.
Context windows got a million tokens long. The cache didn’t get the memo.
Source: Cloudflare Blog — Cross-request KV cache exposure in the GLM 5.3 Workers AI model
Image: Taylor Vick / Unsplash
Discover more from TechPassion
Subscribe to get the latest posts sent to your email.
