The Hard Part of Shipping WebAssembly Is Everything Around It

Anubis took a year to ship a single WebAssembly file. That sounds absurd until you remember the file sits between every visitor and the website they want to reach.

On the open web, making code run fast is relatively easy. Making it work safely for almost everyone is the real engineering problem.

Anubis Charges Bots a Cover Fee

Anubis protects websites from aggressive crawlers by giving each visitor a small computational puzzle. A browser that solves the puzzle gets access to the site.

The technique is called proof of work. It shares an idea with Bitcoin mining and Hashcash, the anti-spam system proposed in the 1990s: make each individual action cheap, but make repetition expensive.

Suppose a challenge requires one second of computation per page. A human opening an article barely notices. A crawler attempting to collect 100,000 pages accumulates roughly 27.8 hours of compute time.

That is a simplified example. Actual costs depend on the device, challenge difficulty, concurrency and implementation. But the principle is what matters.

Anubis does not need to identify every bot with perfect accuracy. It only needs to make industrial-scale scraping less economical. Think of it less as a security guard checking IDs and more as a toll booth charging by the trip.

WebAssembly Solves the Obvious Problem

WebAssembly, usually shortened to WASM, is a compact bytecode format that runs inside modern browsers. It offers more predictable performance than JavaScript for computation-heavy workloads and lets developers bring code written in languages such as Rust or C++ to the web.

That makes it an obvious fit for proof of work. The browser can process repeated calculations efficiently, reducing the delay for legitimate visitors.

But a .wasm file does not execute itself.

JavaScript still has to download the module, initialize it, start the challenge and return the result to the server. The computation may need to run in a Web Worker so the page does not freeze while the puzzle is being solved.

The server must also deliver the correct MIME type for efficient loading through WebAssembly.instantiateStreaming(). A site’s Content Security Policy can block WASM execution. Proxies and corporate security products may interfere with the file. If streaming compilation fails, the application needs another loading path.

One file can therefore create half a dozen failure modes. The code is compact. The blast radius is not.

Faster Is Not Automatically Better

WebAssembly introduces an awkward trade-off: speeding up the challenge helps visitors, but it can also help crawlers.

Anubis is not trying to maximize benchmark performance. It is looking for a narrow sweet spot where the work feels trivial to a person and expensive to anyone repeating it at scale.

Tune the challenge for a high-end desktop, and an older iPhone or budget Android handset may struggle. Mobile users can pay through battery drain, heat and an interface that appears frozen. Visitors on accessibility tools or locked-down enterprise devices may have an even rougher time.

Tune it for the slowest supported device, and a crawler running data-center CPUs or parallel workers may barely notice the cost.

That turns a seemingly simple performance upgrade into a policy problem. Challenge difficulty, device capabilities, timeouts, retry limits and failure handling all become part of the design.

The hashing loop may be the easiest code in the system. Deciding who should wait, for how long and under what conditions is much harder.

The Worst Failure Is an Endless Waiting Room

Bot defenses often fail quietly. Everything looks fine to the site operator while a subset of real users gets trapped outside.

Imagine a CDN serving an older JavaScript client after the server-side verifier has been updated. The browser solves its puzzle correctly, but the server expects a different response format. The visitor sees only a challenge screen that never completes.

Older Safari releases can behave differently from current Chromium browsers. Privacy-focused browsers may restrict required features. Script blockers can interrupt initialization. Schools and companies may filter unfamiliar WASM files at the network boundary.

For users, none of this looks like a compatibility bug. It looks like the website is broken.

Preventing that outcome requires version negotiation, careful cache invalidation and staged rollouts. It also calls for failure telemetry, a JavaScript fallback and a rollback path that can restore access immediately.

That is the likely explanation for the one-year timeline. Building the WASM module was only part of the job. The harder task was ensuring that a failed module would not turn Anubis into a lock on the website it was meant to protect.

AI Defense Is Becoming an Economics Problem

Proof of work cannot stop every AI crawler. A well-funded operator can absorb the expense, distribute challenges across machines or pay a third party to solve them.

Still, Anubis reflects an important shift in web defense. Site owners are moving beyond the binary question of whether a request comes from a human or a bot. They are asking whether the request still makes economic sense when repeated 100,000 times.

Spending a year on one WebAssembly file is not evidence of slow development. It is evidence that raising the cost of scraping without taxing legitimate users is an operational balancing act—and the toll booth is only useful if it does not close the road.