How Cloudflare Saved 100 Terabytes of Memory Optimizing DNS Cache

Most memory-optimization stories end with a tradeoff: you shrink the footprint and something else gets slower. Cloudflare's doesn't. On August 27, 2026, engineer Sebastiaan Neuteboom published "How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache" on the Cloudflare engineering blog, walking through five successive changes to how DNS cache entries are represented in memory. The result: a 56% smaller per-entry footprint, ~100TB of memory freed across the fleet, cache inserts 43% faster, and lookups 19% faster. It hit #1 on Hacker News with 507 points and 139 comments by the next day.

There's no AI angle in the original post — it's pure Rust systems engineering, the kind of write-up that used to be the default on engineering blogs before every launch post needed a model attached to it. We're covering it anyway because the craft is the point: this is what disciplined performance work on a memory-resident data structure actually looks like, and the same discipline applies directly to anyone building high-throughput vector caches, embedding stores, or agent session state at scale.

TL;DR

Why a DNS cache entry costs more memory than you'd think

A DNS cache entry looks simple on paper: a domain name, a record type, some TTL bookkeeping, and one or more answer records. But Rust's default, ergonomic representations for "a list of things" and "a string" carry hidden overhead that's invisible until you're storing 250 billion of them.

Cloudflare's team profiled the actual in-memory layout of a cache entry and found five separate places where the natural, idiomatic Rust representation was paying for flexibility the cache never uses. Each fix targets one of those places. Here they are in the order Cloudflare applied them.

1. The cost of capacity: Vec and String vs boxed slices

Vec<T> and String in Rust are three-word structures: a pointer to the heap allocation, a length, and a capacity. Capacity exists so the collection can grow — push a new element and, if there's spare capacity, no reallocation is needed. That's exactly the right design for data that mutates over time.

A DNS cache entry is not that. It's built once at insert time from a parsed DNS response and never grows again. The capacity field — 8 bytes of pure bookkeeping, one-third of a Vec's 24-byte header — has no job to do. Worse, growable collections routinely over-allocate (doubling capacity on growth is a common strategy), so the backing heap allocation itself can be bigger than the data it holds.

The fix: swap Vec<T> for Box<[T]>, and String for Box<str>. Both are fixed-size, own their heap allocation, and drop capacity entirely — a Box<[T]> is just a pointer and a length, 16 bytes instead of 24. Because the entry is write-once, nothing is lost; there's no future .push() call to support.

Result: 64 bytes saved per entry directly from struct layout, plus the elimination of over-allocated heap padding — over 15 terabytes total across 250B+ live entries. This is the single most instructive line in the whole post: for a lot of production Rust code, "we use Vec because that's the default" is doing real, measurable damage in write-once hot paths.

2. Fewer lists, fewer pointers

A DNS message has three record sections — answer, authority, and additional. The naive representation is three separate Box<[T]> fields, one per section. But each Box<[T]> costs 16 bytes (pointer + length) even when empty, and two of the three sections are frequently unused.

Cloudflare's fix: store one combined list across all three sections, and mark where each section starts with a u16 offset. DNS record counts per section always fit comfortably in an unsigned 16-bit integer, so a 2-byte offset is enough — no precision lost. That replaces two full 8-byte-pointer-plus-8-byte-length pairs with two 2-byte integers.

Result: 28 bytes saved per entry. They applied the same instinct elsewhere in the struct, packing several boolean flags into a single bitflag field instead of one byte per boolean. The general lesson generalizes past DNS: Rust's struct layout and alignment rules mean shrinking or removing one small field can shrink the whole struct by more than that field's own size, because padding gets reclaimed too. It pays to actually look at std::mem::size_of on your hot-path structs, not assume the compiler is already doing the tight packing for you.

3. Dropping the owner

Every DNS record has an "owner" — the domain name it answers for. Query example.com for an A record, and the answer's owner is example.com. In the overwhelming majority of cache entries, the record's owner is identical to the domain that was queried — storing it again per record is redundant.

The exception is CNAME redirection: query www.example.com, get back a CNAME pointing to example.com, followed by an A record whose owner is example.com, not the original query name. So the owner genuinely does need to be stored — just not on the common path.

Cloudflare's fix: change the owner field to Option<Box<Name>>. None means "identical to the query key" and gets reconstructed at read time from the key that's already there. Some(name) stores the differing name explicitly, only when a CNAME actually redirected. Most records now need zero heap allocation for the owner field at all.

This is a clean example of a broader pattern worth stealing: when a field is usually derivable from context and only occasionally diverges, model it as Option rather than always materializing it. It's the same instinct behind delta encoding or copy-on-write — pay only for the exception, not the common case.

4. Enum sizing: why your biggest variant taxes every other variant

This is where Rust's enum layout rules bite people who haven't hit them before. A Rust enum is always sized to fit its largest variant, plus a discriminant tag. Cloudflare's RecordData enum — the type holding a parsed DNS record's actual data — had a NAPTR variant as its largest at 136 bytes (144 bytes once you add the tag and alignment padding).

But NAPTR records are rare. Over 80% of Cloudflare's DNS traffic is A and AAAA records, which need only 4–16 bytes of payload. Every single one of those common, small records was paying the full 144-byte tax because the enum has to be big enough to hold the rarest, largest case even when it's storing the smallest one.

The fix: box the large, rare variants. Instead of Naptr(Naptr) embedding the full struct inline, it becomes Naptr(Box<Naptr>) — the enum only needs to hold an 8-byte pointer for that variant, while the actual 136-byte payload lives in its own heap allocation, sized to what it needs. Txt, and the other larger record types, got the same treatment. The enum's overall size collapses to roughly "8-byte pointer plus whatever the small inline variants need," instead of "the size of the single largest variant."

This isn't free, and Cloudflare is upfront about the tradeoffs:

• Allocator overhead — jemalloc, like most allocators, rounds allocations up to fixed "size class" bins. A boxed variant wastes a few bytes to the nearest bin size rather than being packed exactly.
• Worse memory locality — the enum no longer holds its data inline; reading a boxed variant means following a pointer to a separate, potentially distant heap region, which risks a CPU cache miss that inline data wouldn't.

Boxing large enum variants is a genuinely useful technique — Rust's own clippy lint (large_enum_variant) nudges you toward it — but it's not a free win. Step 5 shows how Cloudflare largely sidestepped the locality cost it introduces.

5. Storing records in wire format — the biggest win

The first four changes optimize the parsed, structured representation of a DNS record. The fifth change asks a more radical question: does the cache need a parsed representation at all?

For most record types, the answer turned out to be no. Cloudflare's final change stores each DNS record as its raw wire-format bytes — the same bytes that arrived over the network — packed into a single Box<[u8]>, with each record prefixed by a 2-byte length so records can still be walked one at a time.

This eliminates two costs at once: the enum-tag overhead from step 4's RecordData type, and the boxed-heap-allocation costs that boxing introduced. Records are now packed contiguously in one buffer instead of scattered across individually boxed heap allocations — which also reclaims most of the memory-locality cost that step 4's boxing gave up, since reading a cache entry now mostly means scanning one contiguous byte buffer rather than chasing pointers.

The tradeoff: records can no longer be randomly indexed by position, only iterated sequentially. Cloudflare judged this an acceptable cost because record counts per cache entry are small (1–4 in their benchmark traffic mix) — sequential scan over a handful of records is effectively free.

The other payoff shows up when building a response. Most record types — A, AAAA, TXT, and the DNSSEC record types — can now be copied directly from the cached wire-format buffer straight into the outgoing DNS message, with no re-serialization at all. Only name-bearing record types (CNAME, NS, MX, SOA) still need to be parsed, because DNS name compression requires understanding where a name's bytes are to potentially point back into an earlier occurrence in the message.

Result: this step alone delivered -5% lookup latency and +13% insert throughput, the latter driven by a reusable scratch buffer that avoids repeated allocation during insert. It's the single biggest contributor to the overall numbers, and it's the cleanest illustration of a rule worth generalizing: a richly parsed in-memory representation is a means, not an end. If your hot path mostly copies data back out unchanged, storing it pre-serialized can beat storing it parsed, even though "parse once, use a nice typed struct" is the reflexive default in most codebases.

The results, measured two ways

Cloudflare benchmarked with synthetic cache fills matching production traffic distribution — 56% A records, 25% AAAA, 19% TXT (using a 64–224 byte range as a stand-in for the rest of the record-type mix), 1–4 records per entry — and separately measured real production resident-memory percentiles across the actual rollout.

Cloudflare is careful to flag the gap between the two measurement methods: production resident-memory numbers are always somewhat lower than what pure per-entry synthetic math would predict, because production memory includes more than just the cache (connection buffers, application overhead, allocator fragmentation elsewhere in the process). The synthetic per-entry numbers isolate the win the code changes actually produced; the production percentiles show what that translated to on real hardware, under real traffic, across a rollout that ran from May 18 to July 6, 2026.

Neither number is "the real one" in isolation — reporting both, and explaining why they diverge, is itself a small piece of good engineering communication that's easy to skip and shouldn't be.

The jemalloc pushback, and why it's fair color

Cloudflare uses jemalloc as the allocator underneath this cache, a defensible default choice for a multithreaded, allocation-heavy workload — jemalloc was built specifically to reduce lock contention and fragmentation under concurrent allocation pressure, which is exactly this workload's shape.

Some Hacker News commenters pushed back that jemalloc isn't necessarily the fastest multithreaded allocator available in 2026 — mimalloc and snmalloc both have benchmarks showing edge cases where they beat it, particularly on allocation/deallocation throughput under specific size-class distributions. That's fair color to include, not a rebuttal of the post: the five structural changes Cloudflare made reduce the number and size of allocations regardless of which allocator sits underneath them. Swapping the allocator is an orthogonal, separately measurable optimization on top of this work, not a substitute for it — you get more out of allocating less than you get out of allocating slightly faster.

Why this matters even if you never touch DNS

If you're building anything that is write-heavy at insert time, read-hot at lookup time, and lives entirely in memory at scale — a vector index, an embedding cache, agent session state, a feature store — every one of these five lessons transfers directly:

1. Audit your collection types. Vec and String are the ergonomic default, not the free one. If your data is write-once, a fixed-size Box<[T]>/Box<str> (or the equivalent in your language) removes real overhead.
2. Know your enum/union sizing rules. A large, rare variant taxes every small, common one. Box the big ones out.
3. Model "usually derivable, sometimes not" fields as optional, not always-materialized — pay for the exception, not the common case.
4. Boxing trades locality for size — it's a real cost, not a free lunch. Weigh cache-miss risk against the bytes saved for your actual access pattern.
5. Ask whether you need a parsed representation at all. If the hot path mostly re-emits data unchanged, storing it in wire/serialized form and parsing lazily can beat "parse once, use a typed struct" — especially for high-throughput caches like a Rust-native vector search index or an in-process database like Turso.

None of this required new hardware, a different language, or a research breakthrough — it required someone actually measuring std::mem::size_of on a hot-path struct and refusing to accept "that's just what the default type costs." That's the entire post, and it's a good reminder that the most durable performance wins in production systems are often unglamorous data-structure audits, not algorithmic cleverness.

FAQ

What did Cloudflare actually optimize in its DNS cache? How individual cache entries are represented in memory on Big Pineapple, the platform behind 1.1.1.1 and its other DNS services. Five Rust data-structure changes cut the average entry from 953 bytes to 420 bytes without changing what's cached or how queries are answered.

What were the five changes, in order? Vec/String → Box<[T]>/Box<str>; three record-section lists → one list with u16 offsets; owner name → Option<Box<Name>>; boxing large RecordData enum variants; and, biggest of all, storing records as raw length-prefixed wire-format bytes instead of parsed structs.

How much memory and speed did this save? ~100TB fleet-wide (130 Gen 13 servers' worth of RAM), 43% faster cache inserts, 19% faster lookups. In production, p99 per-instance memory dropped from 9.3GB to 5.3GB during the May 18–July 6, 2026 rollout.

What is Big Pineapple? Cloudflare's internal name for the shared DNS resolution platform behind 1.1.1.1, Gateway DNS, DNS Firewall, and AS112 — holding over 250 billion cache entries fleet-wide at any given time.

Why does a Rust Vec have a hidden capacity field? Because Vec is designed to grow — capacity tracks reserved-but-unused backing memory so pushing new elements doesn't reallocate every time. A write-once cache entry never grows, so that field (and any over-allocated headroom) is pure waste; Box<[T]> drops it.

Is this relevant if I'm not building a DNS resolver? Yes — the same five lessons apply to any write-once, read-hot, memory-resident structure: vector index entries, embedding stores, agent session state, feature caches. The specifics are DNS; the discipline is general-purpose systems engineering.

Related reading

• Turso: SQLite Rewritten in Rust — another from-scratch Rust rewrite of a data-storage system, with its own memory and concurrency tradeoffs
• Google TurboVec: Compressing 10M Vectors from 31GB to 4GB — a Rust-powered vector index solving the same class of memory-density problem for embeddings
• What Are Embeddings? Vector Search Explained — background on the memory-resident vector stores this post's lessons apply to directly
• GigaToken: A Rust Tokenizer Claiming ~1000x Faster — another Rust performance deep-dive built on SIMD and aggressive caching
• Claude Code CLI Now Uses 2x Less CPU at p99 — a similar profiling-driven fix, this time to garbage-collector scheduling rather than data layout
• Cloudflare Kitesurf: Agent-First Browser in V8 Isolates — Cloudflare's other recent Rust/memory-efficiency engineering post, for browser automation instead of DNS

Official source: Cloudflare Blog — How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache by Sebastiaan Neuteboom, published August 27, 2026.

This post reflects Cloudflare's published figures and the production rollout window (May 18 – July 6, 2026) as of August 28, 2026. Benchmark numbers are Cloudflare's own synthetic and production measurements; verify against the original post for the latest revisions.