The Cross-Core Contract: Memory Ordering and Single-Writer State in Lock-Free Rust

Unknown — Mon, 06 Jul 2026 00:00:00 +0000

The Cross-Core Contract: Memory Ordering and Single-Writer State in Lock-Free Rust</h1>
Part 2 of Low-Level Systems Design in Rust - a series on writing high-throughput, low-latency systems code, using a single-producer / single-consumer (SPSC) ring buffer as the running example.
Part 1, Cache-Conscious Data Layout</a>, decided where the shared fields of a concurrent structure live in memory.
This one covers how two cores read and write those fields - correctly, and as cheaply as the hardware allows. As before, the running example is a single-producer / single-consumer (SPSC) ring buffer, but the ideas apply to any lock-free queue, counter, or reclamation scheme.

Recap, and the question this post answers</h2>
In the first post we set the foundation with a ring buffer so that the two cursors, `tail</code> (where the producer writes next) and head</code> (where the consumer reads next), sit on separate cache lines. This removes false sharing between unrelated hot fields: producer-owned updates no longer invalidate the consumer’s private cursor state, and consumer-owned updates no longer invalidate the producer’s private cursor state. The published cursors are still shared synchronization points; they are simply isolated from unrelated traffic.`
#[repr(C)] pub struct Ring<T, A: BufferAllocator = HeapAllocator> { // producer hot tail: CacheAligned<AtomicU64>, cached_head: CacheAligned<UnsafeCell<u64>>, // consumer hot head: CacheAligned<AtomicU64>, cached_tail: CacheAligned<UnsafeCell<u64>>, // cold // closed, metrics, config, buffer ... }</code></pre> With the above solution we solved "placement", a static problem. In this post we talk about the dynamic problem, which deals with the runtime. When the producer publishes a new tail</code>, what guarantees that the consumer sees the data the producer wrote before it? One solution could be to make the fields AtomicU64</code>. But it has a cost that we can try to avoid. Can we make it cheaper ? Note that two of the four hot fields above are AtomicU64</code> and two are a bare UnsafeCell<u64></code> - that asymmetry is the subject of Part 2. We explore 2 ideas to answer the question, and both of them distill down to the same underlying theme: Minimum-sufficient memory ordering - every atomic operation is a contract with the hardware, pick the cheapest contract that's still correct.</li> Single-writer invariants - when a field has exactly one writer, the cheapest correct contract is no atomic at all.</li> </ol> Part 1 - Memory ordering: the minimum-sufficient rule</h2> The most common mistake with atomics is beig ultra defensive (or pessimistic) and treating the ordering parameter as a safety dial - "when in doubt, turn it up to SeqCst</code>." This can give the correct emergent behavior of your code, but along with correctness, it imposes a "tax" as well, which you may not need to pay. Treat memory ordering as a contract with the hardware about which reorderings are forbidden. Stronger orderings emit more fences and slower instructions on weakly-ordered ISAs (ARM, RISC-V), and forbid more compiler reordering on all of them. One general rule of thumb that you can follow is to use the minimum ordering that establishes the happens-before relationship your algorithm actually requires. Pattern</th> Correct ordering</th></tr></thead> Statistical counters (no control flow depends on value)</td> Relaxed</code></td></tr> A same-thread atomic used like a regular variable</td> Relaxed</code></td></tr> Publish data to another thread</td> Release</code> on the store</td></tr> Read data published by another thread</td> Acquire</code> on the load</td></tr> Mutually exclusive global ordering across all threads</td> SeqCst</code> (rare)</td></tr> </tbody></table> Publication: the one pattern that carries the whole protocol</h3> The producer writes its data into the buffer, then publishes the new tail</code> with Release</code>. The consumer observes tail</code> with Acquire</code>: // Producer - after writing buffer slots: self.tail.store(new_tail, Ordering::Release); // publish // Consumer: let tail = self.tail.load(Ordering::Acquire); // observe</code></pre> The above pattern enforces the exact guarantee that everything the producer wrote before the Release</code> store becomes visible to the consumer after the consumer performs an Acquire</code> load that observes that published value. We need no global ordering, and hence no SeqCst</code>. Now you can draw the chain explicitly - the correctness of any Release/Acquire protocol is an audit you can do by pointing at four edges: producer.write(buffer[idx]) | sequenced-before v producer.tail.store(new_tail, Release) | synchronizes-with v consumer.tail.load(Acquire) - observes new_tail | sequenced-before v consumer.read(buffer[idx]) - sees the producer's write</code></pre> The Release</code> store is the publication edge: it's the single point that makes everything sequenced before it on the producer become visible to any thread that observes the new value via an Acquire</code> load. "Publication edge" is a semantic role, not a claim about the emitted instruction - on some targets it compiles to an ordinary store (more on that below). Statistics: Relaxed</code>, but what "relaxed" really buys us</h3> A metrics counter doesn't gate any control flow - nobody branches on the exact running total - so it preserves atomicity for the counter itself, but establishes no happens-before relationship for surrounding memory operations. self.messages_sent.fetch_add(n, Ordering::Relaxed);</code></pre> Relaxed</code> drops the inter-thread ordering constraints: neither the compiler nor the hardware has to establish a happens-before relationship with surrounding operations on other locations. What Relaxed</code> does NOT drop is the atomicity of the read-modify-write itself. On x86 that fetch_add</code> is still a lock xadd</code> (~10 ns); on AArch64 it's an ldxr</code>/stxr</code> retry loop. Relaxed</code> is not a free local-variable increment. If per-call atomic cost shows up on a genuinely hot path, the fix isn't a weaker ordering (there isn't one) - it's to stop touching the shared atomic every call: accumulate in a thread-local and flush periodically. The cost is target-dependent - and that's the whole point</h3> Here's the subtlety that makes minimum-sufficient ordering matter in practice rather than in theory: the same source compiles to very different costs depending on the CPU's memory model. On x86-64, plain acquire loads and release stores are usually nearly free. The x86-TSO model already provides the ordering those operations need, so Release</code> stores and Acquire</code> loads typically lower to ordinary mov</code> instructions - the ordering annotations exist mostly to constrain the compiler. Do not over-generalize this to "atomics are free on x86": atomic read-modify-writes (fetch_add</code>, compare_exchange</code>) still lower to lock</code>-prefixed instructions regardless of ordering</li> SeqCst</code> stores lower to xchg</code>, or mov</code> + mfence</code>; - target-specific lowering can change.</li> </ul> On AArch64, ARMv7, and RISC-V the cost is real and visible. These are weakly ordered: the CPU reorders loads and stores freely unless told not to. Release</code> typically selects stlr</code> (store-release) on AArch64 or dmb ish</code> + str</code> on ARMv7; Acquire</code> selects ldar</code> or ldr</code> + dmb ish</code>. Each fence costs tens of cycles. SeqCst</code> is worse - a full barrier on every operation. This is why the rule isn't a micro-optimization. Using Relaxed</code> where it suffices and Release/Acquire only at publication points keeps the fence count to the minimum the algorithm requires. That's what makes the difference between source that's fast on both an Intel server and a Graviton or Apple-silicon core, and source that's correct but stalls the pipeline on half your fleet. On a weakly ordered machine, minimum-sufficient ordering is the line between an optimally working protocol and a stalled one. "Once per lifetime" is not a reason to reach for SeqCst</code></h3> In many related scenarios I have seen developers reaching out to a tempting rationalization: "this atomic is only touched once, at startup, so SeqCst</code> is fine - who cares about the cost?" That reasons from the wrong premise. The cost isn't the point; correctness intent is (sometimes we call this "convenience over correctness"). Pick the ordering the algorithm actually needs. If registering a new participant just needs a unique slot index followed by a separate Release</code> publication, then Relaxed</code> or AcqRel</code> is the correct ordering, and SeqCst</code> is noise that misleads the next reader into thinking a global total order matters here. Reach for SeqCst</code> only when the protocol genuinely requires a single global total order across unrelated atomics - and when it does, document which total order and why. "Two threads must observe these two unrelated events in the same order" is a justification. "It's only called once" is not. Documenting the protocol clearly can be a good idea</h3> Release/Acquire discipline is fragile precisely because the ordering arguments are non-local - the correctness of a store here depends on a load over there. So write the protocol down once, next to the type definition, as a numbered sequence: Producer (write path): Consumer (read path): 1. Load tail Relaxed 1. Load head Relaxed 2. Read cached_head (UnsafeCell) 2. Read cached_tail (UnsafeCell) 3. If short: Load head Acquire 3. If short: Load tail Acquire 4. Write buffer slots 4. Read buffer slots 5. Store tail Release (publish) 5. Store head Release (publish)</code></pre> Now anyone editing step 4 can see that step 5 is the publication that makes their write visible, and that step 3 is the matching acquire on the other side. The two columns are mirror images - which is exactly what you'd expect of a symmetric producer/consumer protocol, and a good sign you got it right. Finally: pair every non-trivial ordering decision with a model-checked concurrency test (e.g. a Loom test that exhaustively explores interleavings). Ordering bugs don't reproduce under casual testing - they need a tool that enumerates the reorderings the hardware is allowed to perform. Part 2 - Single-writer invariants and UnsafeCell</code></h2> Part 1 was about choosing the cheapest possible atomic. Part 2 is about noticing when you don't need an atomic at all. Even Relaxed</code> atomics aren't free in the way people assume. Beyond the instruction cost, an Atomic</code> access is opaque to the optimizer: the compiler may not reuse a loaded value across an opaque function call, nor merge adjacent loads of the same atomic, nor hoist one out of a loop. Those are real optimizations you forfeit. When a memory location has exactly one writer, you can reclaim all of them. The pattern: if you can prove - and document - that a location is written by exactly one thread, store it in UnsafeCell<T></code> instead of Atomic<T></code>. Reads from the writer thread are plain reads; any reads from other threads need a separate synchronization channel (in the ring, that channel is the Release/Acquire protocol on the cursors from Part 1). Recall the two cursor caches from the struct at the top. Each core keeps a private snapshot of the other core's cursor, to avoid a cross-core read on the fast path. The producer's snapshot of head</code> has exactly one writer - the producer - so it isn't an atomic: cached_head: CacheAligned<UnsafeCell<u64>>,</code></pre>// SAFETY: cached_head is only ever written by the (single) producer thread. let cached_head = unsafe { self.cached_head.get() };</code></pre> The matching write is just as plain - a dereference and a store, no fences, no compare_exchange</code>, no fetch_</code>: // SAFETY: same single-writer guarantee - only the producer writes here. unsafe { self.cached_head.get() = head; }</code></pre> Why this is worth the unsafe</code>: the compiler can now treat cached_head</code> like an ordinary variable - keep it in a register across function calls, hoist it out of the reservation loop, fold it into the surrounding capacity arithmetic. An atomic would have barred every one of those. The discipline: enforce single-writer outside the type</h3> The soundness of all this rests on one claim - exactly one thread ever writes this field - and that claim cannot be enforced by the UnsafeCell</code> itself. It has to be guaranteed by the surrounding API. The cleanest way is to make the single-writer property structural rather than a rule humans must remember: Give out a non-Clone</code> producer handle. If the Producer</code> type doesn't implement Clone</code>, it can't be duplicated and handed to a second thread, so only one thread can ever hold the right to call reserve()</code> - and therefore only one thread can ever write cached_head</code>. </blockquote> The type system now carries the invariant. There's no comment to forget and no runtime check to pay for; constructing a second writer simply doesn't compile. You must write unsafe impl Sync</code> by hand - and justify it</h3> A type containing UnsafeCell<T></code> is deliberately not Sync</code>. The standard library makes UnsafeCell</code> opt out of Sync</code> because unsynchronized interior mutability is the entire reason UnsafeCell</code> exists. But sharing the ring across threads is the whole point of the structure, so you have to supply the impl yourself - and the value of doing it by hand is that it forces you to write down the proof: // SAFETY: // - `tail` and `cached_head` are written only by the producer thread. // - `head` and `cached_tail` are written only by the consumer thread. // - Cross-thread visibility of buffer slots is established only via the // Release store on `tail` (producer) and the Acquire load on `tail` // (consumer); the symmetric protocol applies to `head`. // - Buffer-slot ownership is transferred by the [head, tail) range // invariant: outside that range, slots are exclusively the producer's // to write; inside it, exclusively the consumer's to read. // - The backing storage is valid for cross-thread access and deallocation // under this producer/consumer protocol. // - `T` may move across threads only when `T: Send`. unsafe impl<T: Send, A: BufferAllocator + Sync> Sync for Ring<T, A> {} unsafe impl<T: Send, A: BufferAllocator + Send> Send for Ring<T, A> {}</code></pre> A few things about this block are important for reasoning: The T: Send</code> bound is mandatory. If T</code> can't cross threads, neither can a ring of T</code>. Dropping the bound would let you transport a !Send</code> type across a thread boundary inside the ring.</li> Forgetting the impls is a compile error. Forgetting the justification is a correctness bomb that compiles fine. Treat the SAFETY:</code> comment as part of the code, not commentary on it.</li> Push proof obligations down to where they're owned. The clause about backing-storage validity really belongs to the allocator's contract, not to the ring. Stating it as an allocator-trait guarantee lets this impl rely on it instead of re-deriving allocator semantics inline.</li> </ul> And the general discipline that ties Parts 1 and 2 together: every unsafe</code> block gets a // SAFETY:</code> comment naming the specific invariant it relies on. Better still, give those invariants stable names (e.g. an INV-…</code> identifier in a spec) and cite the name in the comment, so the chain from "this code is sound" to "because this property holds" is auditable end to end rather than folklore. Where single-writer state shows up beyond ring buffers</h3> per-thread accumulators in scatter-gather pipelines (write locally, reduce once);</li> per-thread caches of read-mostly global state, refreshed only on notification;</li> exclusive slot access between a reserve()</code> and its matching commit()</code>.</li> </ul> The two ideas are one idea</h2> Minimum-sufficient ordering and single-writer state are the same principle applied at two levels. Ordering asks: given that I need an atomic, what's the cheapest correct contract with the hardware? Single-writer asks the question one level up: do I need the atomic at all? In both cases the move is to pay for exactly the synchronization the algorithm requires and not one fence more. Loosely this can correspond to the more general principle of using the least powerful abstraction when it comes to software engineering. There's a third level to this same principle: sometimes you don't need the cross-core read either, because a private, conservative snapshot of the other core's cursor is good enough on the fast path. That snapshot is exactly the single-writer UnsafeCell</code> cache introduced here. In Part 3 we will turn it into a throughput win. Cache-Conscious Data Layout in Rust: Field Zoning, False Sharing, and the 128-Byte Rule Unknown — Mon, 29 Jun 2026 00:00:00 +0000 Cache-Conscious Data Layout: Field Zoning, False Sharing, and the 128-Byte Rule</h1> Part 1 of Low-Level Systems Design in Rust - a series on writing high-throughput, low-latency systems code, using a single-producer / single-consumer (SPSC) ring buffer as the running example. Part 0 - Architectural Decomposition</a> made the highest-leverage decision (remove contention structurally, so every writer gets its own ring) and holds the full series index and guiding principles. This post starts the micro-level work: given one such SPSC ring, how should it be laid out in memory? The patterns aren't specific to ring buffers - they apply to any structure more than one core touches. The thing nobody tells you about multi-threaded structs</h2> When you write a single-threaded data structure, the only layout question that matters is "does it fit in cache." When you write a multi-threaded one, a second, relevant and a bit subtle question appears: For each field, which core touches it, and how often? </blockquote> Getting this wrong will make you pay in a way that no profiler flags as a single hot line. Two cores writing to different fields that happen to share a 64-byte cache line will quietly serialize each other through the hardware coherence protocol - a pathology called false sharing. The code looks lock-free. The benchmark says otherwise. This post is about designing the layout deliberately. It comes in two parts: Field zoning - grouping fields by (write owner, frequency)</code> so that one core's hot writes don't evict another core's hot working set.</li> Alignment and padding - the mechanics that make zoning real: why #[repr(C)]</code> is load-bearing, why the magic number is often 128 and not 64, and why adding prefetch hints can make things worse.</li> </ol> The running example is a single-producer / single-consumer (SPSC) ring buffer. One core (the producer) appends, another core (the consumer) drains. They coordinate through two monotonic cursors - tail</code> (where the producer writes next) and head</code> (where the consumer reads next). Though the following discussion and the design principles are general enough, you can refer to this ring buffer</a> implementation as a reference point that honors these guiding principles. Part 1 - Field zoning: design based on "who touches what"</h2> The cache line - commonly 64 bytes on x86-64 and many AArch64 cores - is the unit of currency for inter-core communication. Coherence traffic is accounted per line, not per byte or per field. So the first design move for any shared structure is to sort its fields into zones by access pattern: Zone</th> Fields</th> Write owner</th> Cross-core access</th></tr></thead> Producer-hot</td> tail</code>, cached_head</code></td> producer</td> consumer samples tail</code></td></tr> Consumer-hot</td> head</code>, cached_tail</code></td> consumer</td> producer samples head</code></td></tr> Cold</td> closed</code>, config</code>, metrics</code></td> lifecycle/observability</td> rare, not hot-path polling</td></tr> </tbody></table> "Producer-hot" does not mean "producer-only." The producer owns writes to tail</code>, but the consumer still reads tail</code> when its cached view runs dry. The important distinction is write ownership: the producer is the core that keeps making the line exclusive, so the fields near that write must be chosen deliberately. First, the vocabulary, since the rest of the post leans on it. The hot path is the code that runs on (almost) every operation - here, the send and receive loops that each core executes millions of times a second. Hot fields are the ones those loops touch, like tail</code>, head</code>, and the two cursor caches. The cold path, by contrast, is everything that runs rarely - construction, shutdown, an occasional metrics read - and cold fields are the ones only the cold path touches, like config</code> and metrics</code>. "Hot" and "cold" are about frequency of access, and they're what the zones in the table above are sorted by. The rule is then simple to state: place each zone on its own set of cache lines. A hot field written by one core must not share a line with hot fields another core reads or writes frequently - a producer store to tail</code> should not invalidate the line holding the consumer's head</code> or cached_tail</code>, and vice versa. Cold fields, since nobody contends over them on the hot path, can stay packed together at the back. Here is a ring buffer that encodes those zones directly in its declaration. The type parameter A</code> is just a pluggable allocator for the backing buffer, ignore it if you like - what matters is the order and grouping of the fields: #[repr(C)] pub struct Ring<T, A: BufferAllocator = HeapAllocator> { // PRODUCER HOT tail: CacheAligned<AtomicU64>, cached_head: CacheAligned<UnsafeCell<u64>>, // CONSUMER HOT head: CacheAligned<AtomicU64>, cached_tail: CacheAligned<UnsafeCell<u64>>, // COLD closed: AtomicBool, metrics: Metrics, config: Config, buffer: UnsafeCell<A::Buffer<T>>, }</code></pre> A few things are worth reading carefully here, because each one is a deliberate choice rather than an accident of style. The two cached_</code> fields live in the hot zones, not the cold one. A producer that wants to know "is there space?" must compare tail</code> against head</code>. But head</code> is written by the other core, so reading it directly is a potential cross-core coherence miss - tens to hundreds of cycles. Instead, the producer keeps cached_head</code>: a single-writer, producer-owned snapshot of where the consumer was last time we looked. Because a stale view of head</code> can only ever under-report available space (never over-report it), the cache is always safe to trust on the fast path, and the expensive Acquire</code> read of the real head</code> fires only when the cache says we're out of room. That snapshot is touched on every send, so it belongs in the producer-hot zone - and it is deliberately a plain UnsafeCell<u64></code>, not an atomic, precisely because only one core ever writes it. The consumer has the mirror image: cached_tail</code> lets it avoid reading the producer-owned tail</code> until the local snapshot is exhausted. Cold fields share lines on purpose. closed</code>, metrics</code>, and config</code> are packed together with no padding between them. That's not laziness - it's the point. The goal of zoning is not to align everything; it's to align only the things that ping between cores. Padding a cold field just wastes a cache line you could have spent keeping hot data resident. If a lifecycle flag is polled on every send or receive, it is no longer cold; promote it into its own hot or lifecycle zone instead of hiding it behind the word "shutdown." Why #[repr(C)]</code> is doing real work</h3> Notice the #[repr(C)]</code> on the struct. It is not decorative, and removing it would silently undermine the entire layout strategy. By default, Rust uses repr(Rust)</code>, and the Rust Reference is explicit</a> that repr(Rust)</code> guarantees only field alignment, that fields don't overlap, and that the aggregate is sufficiently aligned. It explicitly does not guarantee that fields are laid out in declaration order - the compiler is free to reorder them (typically to minimize padding). For a normal struct that's a feature. For a struct whose correctness-of-performance depends on tail</code> and head</code> sitting in different zones, a reorder could collapse your carefully separated zones back onto shared lines. #[repr(C)]</code> pins fields in declaration order, so the zone layout you wrote is the zone layout you get. One subtlety that's easy to miss: repr(C)</code> on the outer struct does not recursively freeze the layout of nested repr(Rust)</code> fields. It fixes the order of Ring</code>'s own fields, but the internal ordering of, say, Metrics</code> or Config</code> is still repr(Rust)</code> unless those types carry their own repr(C)</code>. If a nested struct has its own hot/cold split that matters, give it repr(C)</code> too. Here it doesn't matter, because Metrics</code> and Config</code> are entirely cold. Part 2 - Alignment and false sharing: making zoning real</h2> Zoning is the intent. Alignment is the mechanism. Declaring that tail</code> and head</code> belong to different zones accomplishes nothing unless the bytes actually land on different cache lines. That's the job of CacheAligned<T></code>. The one-line helper that pays for itself</h3> /// Aligns a value to a 128-byte boundary so that cross-core fields don't /// share a cache line. The 128 (vs. 64) is a target policy: it also leaves /// room for adjacent-line/spatial-prefetch effects on CPUs where those matter. #[repr(C, align(128))] struct CacheAligned<T> { value: T, } impl<T> CacheAligned<T> { const fn new(value: T) -> Self { Self { value } } } impl<T> std::ops::Deref for CacheAligned<T> { type Target = T; fn deref(&self) -> &Self::Target { &self.value } }</code></pre> That's the whole thing - a #[repr(C, align(128))]</code> newtype with a Deref</code> so it's transparent at the call site (self.tail.load(...)</code> just works). No library dependency required. Because Ring</code> is repr(C)</code>, the compiler lays fields out in declaration order and inserts whatever padding is needed to satisfy each field's alignment. Because a Rust type's size is a multiple of its alignment, each small CacheAligned<T></code> also consumes a full 128-byte slot. No two hot fields can share a line, and - critically - neither can they share a line with the cold fields that follow. Why 128 bytes, when the cache line is 64?</h3> This is the part that trips people up. If the cache line is 64 bytes, why pad to 128? Because strict line-level separation can still be too optimistic. Some CPUs have adjacent-line or spatial prefetch behavior: when you touch one 64-byte line, the core may speculatively pull in its neighbor. If producer-hot data sits on the line immediately adjacent to consumer-hot data, the prefetcher can drag the two into each other's caches even though, strictly speaking, they never shared a line. The result is prefetcher-induced false sharing: the lines are technically separate, the contention is real, and line-granularity tools may not point directly at it. You see it as throughput that's lower than the algorithm predicts. For small cursor fields, aligning each hot field to 128 bytes makes it start on an even 64-byte-line boundary, so an adjacent-line prefetch usually pulls in padding rather than another core's hot data. Treat the exact width as a target-specific policy, not a universal constant. Crossbeam's current CachePadded</code></a> policy is: 128 bytes on x86-64, AArch64, and powerpc64,</li> 256 bytes on s390x,</li> 32 bytes on arm, mips, mips64, sparc, and hexagon,</li> 16 bytes on m68k,</li> 64 bytes on everything else.</li> </ul> Crossbeam also documents that these are reasonable guesses, not a guarantee that they match the physical cache line of the machine you're running on. A local CacheAligned<T></code> with a fixed align(128)</code> is appropriate when you specifically want to pin the policy (e.g. you've benchmarked your target and want it frozen). Either way: verify with benchmarks, don't trust any single number - including 128. Putting it together: the layout the CPU actually sees</h3> With the zones and the alignment combined, construction is unremarkable - the layout work has already been done by the types: Ring { tail: CacheAligned::new(AtomicU64::new(0)), cached_head: CacheAligned::new(UnsafeCell::new(0)), head: CacheAligned::new(AtomicU64::new(0)), cached_tail: CacheAligned::new(UnsafeCell::new(0)), closed: AtomicBool::new(false), // metrics, config, buffer ... }</code></pre> The producer core writes the tail</code> line and uses the cached_head</code> line. The consumer core writes the head</code> line and uses the cached_tail</code> line. The published cursors still create the unavoidable cross-core communication - the consumer must sometimes observe tail</code>, and the producer must sometimes observe head</code> - but those invalidations no longer drag unrelated hot fields with them. That is the payoff, and it costs roughly one 128-byte slot per hot field: a good trade for a structure hammered by two cores, and one you'd never make for a cold field. Two counterintuitive corollaries</h2> Once you're thinking at the level of cache lines and prefetchers, two pieces of "obvious" performance folklore turn out to be wrong. Don't sprinkle in software prefetch hints</h3> A common instinct, having learned that the prefetcher exists, is to "help" it: drop _mm_prefetch</code> (or Rust's core::intrinsics::prefetch_</code>) into the hot loop to pull the next slots in early. On a ring buffer - i.e. stride-1, linear access - this is almost always a loss. Modern L1/L2 hardware prefetchers detect linear strides within the first 2–3 accesses and stream cache lines in ahead of the CPU automatically. A manual software prefetch doesn't add to that; it competes with it - consuming instruction issue slots and memory bandwidth that the demand fetches actually need, while the hardware was already going to bring those lines in. It's a mistake I've watched ship more than once: a hot loop seeded with manual prefetch hints, A/B benchmarked, and found to hurt throughput - the hints come back out. The honest rule: software prefetches earn their keep only for non-linear access patterns - graph walks, hash-table probe sequences, pointer chasing - where the hardware prefetcher can't predict the next address. And even then, only add them if a profile-guided experiment proves they help on your workload. For anything stride-1, get out of the prefetcher's way. Pick a capacity that fits a cache level - deliberately</h3> The ring's buffer has its own cache story, separate from the cursors. Capacity doesn't guarantee residency in a particular cache level, but it does set the working-set budget you are asking the hierarchy to carry. Choose it consciously rather than typing a round number and letting the hardware discover the trade: Sized to stay near L1. For 8-byte elements, 4,096 slots is 32 KiB. That can be a reasonable starting point for a cache-resident ring on cores with a 32 KiB-or-larger L1D, but it leaves no room for other hot data on a 32 KiB L1. L1D sizes vary, and fitting capacity-wise does not guarantee zero L2/L3 misses - conflict misses, prefetch behavior, coherence traffic, and other hot data on the same core all still matter. Confirm with hardware counters, not arithmetic.</li> Sized to absorb bursts. Sizing at 256K slots (2 MiB for 8-byte elements) deliberately stops pretending the buffer is L1-resident and accepts lower-cache or LLC latency in exchange for soaking up much larger bursts without backpressure.</li> </ul> Both are legitimate. What isn't legitimate is letting "a nice round number like one million" sneak into your config and accidentally deciding the working set your hot path has to carry. Where this generalizes</h2> None of this is specific to ring buffers. The "zone by (write owner, frequency)</code>, then pad the cross-core fields" pattern shows up wherever a structure is shared across cores: the tail</code>/head</code> cursors of SPSC queues, and the contended cursors of MPSC queues (padding prevents false sharing there; it does not remove the shared-tail contention),</li> per-thread counters in sharded metrics (an AtomicU64</code> per core, summed on read),</li> per-core epoch counters in EBR / hazard-pointer memory reclamation,</li> per-CPU scheduler run queues,</li> resize-epoch counters in lock-free hash tables.</li> </ul> In every case the recipe is the same: For each field, name the write owner, the readers, and the frequency.</li> Group fields into hot zones (one per write owner, plus isolated zones for any unavoidable multi-writer atomics) and a cold zone.</li> Lock the order with #[repr(C)]</code>.</li> Pad the hot, cross-core fields to a target-appropriate cache boundary - 128 bytes on x86-64/AArch64/powerpc64 in Crossbeam's policy, or let CachePadded</code> pick.</li> Leave the cold fields packed, and don't add prefetch hints to linear loops.</li> Verify the win with hardware counters such as perf c2c</code> where available, and with a throughput benchmark - because the bugs this prevents are exactly the ones a profiler won't point at for you.</li> </ol> The structure looks lock-free either way. The difference between the two versions is whether the hardware agrees. Architectural Decomposition: Remove Contention by Design Unknown — Mon, 22 Jun 2026 00:00:00 +0000 Architectural Decomposition: Remove Contention by Design</h1> Across the Rust projects I've worked on, the ones that demanded high throughput and low latency were almost always built around a few architectural principles that remained invariant across all of them. Many of these principles are not limited to Rust, but apply equally, with implementation-specific variations, to lower-level languages like C++ or Zig. In this series of blog posts I plan to mine those design principles and prepare a collection of patterns that can be applied in similar contexts. I call the series Low-Level Systems Design in Rust - this is the opening installment of it. Most of the posts work at the micro level - cache lines, memory ordering, allocation - using a single-producer / single-consumer (SPSC) ring buffer as the running example. But before any of that pays off, there's one decision that dwarfs all the others, and it's structural. This post is about that decision. The patterns aren't specific to ring buffers: they apply to any system where many writers feed a shared reader. Three principles run underneath everything that follows, and the first one is this post: The biggest wins are architectural, not microscopic. Eliminating a shared write cursor is worth orders of magnitude more than picking the "right" Ordering</code>. Cache-line padding, atomic tuning, and inlining only matter after you've made the structural choice that removes contention in the first place.</li> Optimize what the hardware actually does, not what the source looks like. Modern CPUs speculate past branches, reorder loads, and bounce cache lines between cores under contention. Many also have adjacent-line or spatial prefetch behavior that can make a nominally "separate" cache line relevant to performance. Source-level cleverness that ignores those realities buys nothing.</li> Measure, don't guess. Every technique in this series earns its place against a benchmark - and benchmark the code you think you're benchmarking: Rust's optimizer is happy to delete work whose result is unused, so reach for std::hint::black_box</code> and read the generated assembly before trusting any number.</li> </ol> Here's a rough idea of what I plan to cover as part of this series: Architectural Decomposition (this post) - remove contention by design.</li> Cache-Conscious Data Layout - field zoning, false sharing, and cache-line and adjacent-line placement heuristics.</li> The Cross-Core Contract - memory ordering and single-writer state.</li> Amortizing Cross-Core Coordination - cursor caching and batch processing.</li> Zero-Copy on the Hot Path - reserve/commit and fast-path/slow-path splitting.</li> Compile-Time Leverage - specialization, branch & arithmetic hygiene, and inlining.</li> Backoff & Memory Provisioning - adaptive spinning, custom allocators, NUMA & huge pages.</li> Safety as Performance - moves, the borrow checker, debug_assert!</code>, and cold-path hygiene.</li> Trustworthy Performance - verification, idioms, anti-patterns, and knowing when to stop.</li> </ol> Start here; the rest builds on the foundation this post lays. The most expensive instruction is the one two cores fight over</h2> Here's the scenario that drives the whole design. You have N independent producers - threads, tasks, cores - that all need to hand work to a single consumer. Assume a bounded in-memory handoff path where per-producer FIFO is acceptable and the consumer can poll. The obvious implementation is one shared multi-producer queue: a single buffer with shared write-side state that every producer must update, often through an atomic read-modify-write such as a compare-and-swap or fetch_add</code>. That obvious implementation has a problem that gets worse as you add cores. Every producer is hammering the same tail</code> cache line, or whatever cache line holds the shared write-side state. On a CAS or other read-modify-write, the line has to be in the exclusive coherence state on the writing core - which means other cached copies of that line must be invalidated before the write can complete. Two producers that want to append at the same instant don't run in parallel; the hardware coherence protocol serializes them, ping-ponging ownership of that one line from core to core. Add a third producer and it's worse. The shared cursor is a contention point, and the cost of touching it scales with the number of cores touching it. This is the trap: the code looks lock-free - there's no mutex anywhere - but the shared atomic cursor reintroduces serialization at the hardware level. You've moved the lock from the OS into the cache-coherence fabric, where it's harder to see and you can't even name it. The decomposition: give every writer its own lane</h2> The fix is to refuse the premise. Don't make independent writers share a structure at all. Give each writer its own private single-writer structure, and let the consumer poll across them: Producer 0 ──► [ SPSC Ring 0 ] ──┐ Producer 1 ──► [ SPSC Ring 1 ] ──┼──► Consumer (sweeps the rings in turn) Producer 2 ──► [ SPSC Ring 2 ] ──┘ … one dedicated ring per producer …</code></pre> Now each producer writes only to its own ring. A producer's append updates a cursor no other producer ever touches. In Rust that cursor is still an atomic value, because the consumer reads it from another thread, but the hot update is a simple atomic store</code>, not an atomic read-modify-write. The result: no CAS, no spin loop, and crucially no producer-producer cache-line bouncing. Two producers appending at the same instant genuinely run in parallel, because they're writing to different cache lines on different rings. The defining property: N independent single-writer structures have zero coherence traffic on the hot cursor path between the writers. </blockquote> The contention didn't vanish into thin air - it was converted. Instead of N cores serializing on one hot cursor, you have one consumer paying an O(N) sweep to poll the rings in turn. That's a fundamentally better trade when producer throughput dominates: a single thread doing a little more sequential work, in exchange for eliminating the quadratic-feeling coherence storm among the writers. Each individual ring is then a clean single-producer / single-consumer problem - which is exactly the structure the rest of this series spends its time optimizing. This decomposition also changes the semantics. Each ring preserves FIFO order for one producer, but the system no longer has a natural global enqueue order across all producers. If the consumer needs a total order, you need sequence numbers, timestamps, or a merge step - and that reconciliation has its own cost. Be honest about the trade-offs</h2> Decomposition isn't free, and the costs are worth stating plainly: Memory grows with N. One buffer per producer instead of one shared buffer. For a few dozen producers this is negligible; for tens of thousands it's a real budget line.</li> Single-message latency can rise. The consumer has to poll - a message sitting in ring 3 isn't seen until the sweep reaches ring 3. For small, hot sets of rings that overhead is often tiny compared with the contention it eliminates, but the exact number is hardware- and workload-dependent. For very large N you may need a smarter consumer: a ready-set, per-core consumers, or an event mechanism, all of which I will return to in later posts.</li> Backpressure becomes per-lane. If one producer's ring is full, that producer has to block, drop, or spill somewhere else, while other producers may continue normally. That isolation is often useful, but it is a policy decision, not a free consequence.</li> Fairness is now explicit. A consumer that drains one busy ring for too long can starve quieter rings. The sweep policy - round-robin, bounded drain, priority, batching - becomes part of the design.</li> Global ordering is no longer free. Per-producer FIFO is cheap. Cross-producer total order requires extra metadata and merge work.</li> </ul> When to reach for it: any time the asymmetry between writers and readers is large - many writers, one (or few) readers - and the readers can tolerate an O(N) sweep. That's precisely the shape where a shared cursor hurts most and a per-writer structure helps most. You've seen this pattern before</h2> Architectural decomposition is one of the most widely reinvented ideas in systems software, because the writer/reader asymmetry is everywhere: Allocator sharding - jemalloc uses arenas and thread caches, while mimalloc shards free lists and separates thread-local from concurrent free paths, so allocation and free traffic doesn't collapse onto one global structure.</li> Per-worker run queues - schedulers like Tokio and Go keep local queues around workers, with global queues and work-stealing as reconciliation mechanisms instead of putting every operation through one central queue.</li> Sharded counters - one AtomicU64</code> per core, summed on read, so the hot increment path never contends. (The thread-local metrics aggregation in Part 8 is the same idea taken one step further.)</li> Per-writer log segments - write-ahead logs that give each writer its own segment and merge at commit time.</li> </ul> In every case the move is identical: take a shared mutable thing that N agents fight over, and replace it with N private things plus a cheap reconciliation step on the read side. Why this comes first</h2> It's tempting to open a performance series with the satisfying micro-techniques - cache-line padding, the Relaxed</code> versus Acquire</code> decision, the zero-copy reservation. But those are refinements within a structure, and refining a structure that's architecturally contended is polishing a part that shouldn't exist. The padding in Part 1 matters because there's no longer a shared cursor competing for the same cache lines; the minimum-sufficient ordering in Part 2 is tractable because each ring has exactly one writer and one reader. So this is the decision that makes the rest of the series worth doing. Get it right and every later technique compounds on a clean foundation. Get it wrong - keep the shared cursor - and no amount of cache-line tuning will save you from the coherence traffic you designed in. From here on, the running example is a single one of those SPSC rings, examined up close. Part 1 starts where it should: how to lay that ring out in memory so two cores never fight over a cache line. Scalar Replacement of Aggregates: How "Copy to Locals" Unlocks the Compiler Unknown — Mon, 15 Jun 2026 00:00:00 +0000 Scalar Replacement of Aggregates: How "Copy to Locals" Unlocks the Compiler</h1> Some performance fixes look almost suspiciously small in the diff. You change a couple of lines, the code still says the same thing to a human reader, and a hot loop suddenly has far less memory traffic. TigerBeetle's PR #3201</a> is one of those fixes. It changed the AEGIS-128L state update in src/stdx/aegis.zig</code> from "operate through a pointer to state.blocks</code>" to "copy the eight lanes into a local array, update the local array, then write it back at the boundary." The PR reported about a 2x improvement in the AEGIS microbenchmark and a 3-10% end-to-end throughput improvement in the measured TigerBeetle workloads. The compiler idea behind this is SROA, or Scalar Replacement of Aggregates. More precisely, this is a manual SROA-friendly code shape: the source code removes an aliasing and visibility obstacle so LLVM can keep the hot state in registers instead of repeatedly loading and storing it. This post explains what SROA is, why pointers and opaque memory effects get in the way, and how the "copy to locals, work, write back" shape helps. Then we will build small Zig and Rust reproductions and connect the result back to TigerBeetle's AEGIS code. 1. What SROA Actually Means</h2> An aggregate is a value made from smaller values: a struct, array, tuple, or record. Scalar replacement of aggregates is the compiler transformation that breaks an aggregate into independently optimized pieces. LLVM documents the sroa</code> pass as a transformation that breaks aggregate alloca</code>s, such as local structs or arrays, into member-level alloca</code>s and then, when possible, promotes those pieces to scalar SSA values. That last part is the money: once the fields are SSA values, the register allocator can keep them in registers, and later optimization passes can reason about each piece independently. That phrasing matters. SROA is not magic fairy dust for any struct-shaped value anywhere in the program. It is strongest when the aggregate is local, non-escaping, and boring. If the compiler sees "this local array is private to this function," it has a much easier proof than "this pointer points to caller-owned memory and maybe something opaque can observe it." A Tiny Before/After</h3> Consider a function that updates a pair in place: struct Pair { long a, b; }; long sum_in_place(struct Pair p) { p->a += 1; p->b += 2; return p->a + p->b; }</code></pre> The function must store the new a</code> and b</code> back through p</code>, because mutating the caller's object is part of its behavior. A compiler may still optimize the loads and arithmetic, but it cannot simply pretend the pointed-to object is a private local. Now compare a by-value shape: long sum_local(struct Pair in) { long a = in.a + 1; long b = in.b + 2; return a + b; }</code></pre> These functions are not semantically identical: the first updates the caller's pair, the second does not. That difference is the point. In the second version the fields are just private values. On x86-64 SysV, a two-long</code> struct is commonly passed in registers, and the body can collapse to something like: lea rax, [rdi + rsi + 3] ret</code></pre> No aggregate has to survive as memory in the optimized code. That is SROA's happy path: aggregate-shaped source, scalar-shaped machine code. 2. What Blocks It: Aliasing and Visibility</h2> The hard question for the optimizer is not "would registers be faster?" The answer is nearly always yes. The hard question is "is it legal to keep this value in registers?" When code repeatedly mutates state through a pointer, the compiler has to preserve the observable behavior of that memory. This becomes especially restrictive around operations that may read or write memory in ways the optimizer cannot see through: calls that are not inlined,</li> volatile or atomic operations,</li> inline assembly with a "memory"</code> clobber,</li> compiler or hardware fences,</li> raw pointers whose aliasing contract is weaker than a language-level unique reference.</li> </ul> Compilers can recover in many cases, especially after inlining or when they have strong noalias</code> information. But the robust, easy-to-prove pattern is: copy aggregate from memory -> mutate private locals -> write aggregate back in one boundary phase</code></pre> Once the address of the local copy does not escape, the optimizer can scalarize the local copy freely. The pointer-owned memory is touched at the boundary. The hot loop works on private values. That is the whole trick. You are not hand-allocating registers. You are making the compiler's proof small enough that the normal optimization pipeline can do its job. 3. Minimal Reproduction in Zig and Rust</h2> The examples below intentionally include a compiler-only memory barrier inside the loop. The barrier is not part of the TigerBeetle change; it is just a demonstration tool. Why add it? Without a barrier, a sufficiently smart optimizer may hoist, sink, or combine memory operations in the small example, making the difference harder to see. A compiler barrier says: memory operations may not freely move across this point. It does not stop pure register arithmetic. That makes the contrast crisp: pointer version: every round touches memory, so the barrier pins the loads and stores in the loop;</li> local-copy version: the loop body touches private scalar values, so there is no per-round memory traffic to pin.</li> </ul> In optimized output, especially for Rust's compiler_fence</code>, you may not see a barrier marker inside every local-copy iteration. Once the loop is register-only, the compiler-only barrier has no loop memory operations to constrain, so it may be moved relative to the register arithmetic. That is not a failure of the demo; the important thing to inspect is whether loads and stores remain in the loop body. Zig</h3> const State = extern struct { blocks: [8]u128, }; // Zig 0.15+ typed clobber syntax. This is a compiler barrier, not a CPU fence. inline fn compilerBarrier() void { asm volatile ("" ::: .{ .memory = true }); } fn roundBad(state: State) void { // BAD: mutate the aggregate through a pointer every iteration. var i: usize = 0; while (i < 1000) : (i += 1) { state.blocks[0] +%= state.blocks[1]; state.blocks[1] ^= state.blocks[2]; compilerBarrier(); } } fn roundGood(state: State) void { // GOOD: copy by value into a private local array. var blocks: [8]u128 = state.blocks; var i: usize = 0; while (i < 1000) : (i += 1) { blocks[0] +%= blocks[1]; blocks[1] ^= blocks[2]; compilerBarrier(); } state.blocks = blocks; } export fn entry_bad(s: State) void { roundBad(s); } export fn entry_good(s: State) void { roundGood(s); }</code></pre> The important line is: var blocks: [8]u128 = state.blocks;</code></pre> Read it literally: var</code> declares a mutable local.</li> [8]u128</code> is a fixed-size array value, not a slice and not a pointer.</li> = state.blocks</code> copies the array value out of the state.</li> </ul> Mutating blocks</code> does not mutate state.blocks</code> until the explicit assignment at the end. These shapes do not make a private copy: const slice: []u128 = state.blocks[0..state.blocks.len]; const p: [8]u128 = &state.blocks;</code></pre> Both are views of the original storage. They reintroduce the pointer-shaped memory access that we are trying to avoid. Build and inspect: zig build-obj sroa.zig -O ReleaseFast -fllvm -femit-asm=sroa.s # Compare entry_bad and entry_good in sroa.s. # entry_bad should load/store state in the loop. # entry_good should load before the loop and write back after the loop.</code></pre> For LLVM IR: zig build-obj sroa.zig -O ReleaseFast -fllvm -femit-llvm-ir=sroa.ll</code></pre>Rust</h3> use core::sync::atomic::{compiler_fence, Ordering}; #[repr(C)] pub struct State { blocks: [u128; 8], } /// # Safety /// /// `state` must be valid, properly aligned, and exclusively writable for the /// duration of this call. // SAFETY: the exported symbol name must be unique in the final linked program. #[unsafe(no_mangle)] pub unsafe extern "C" fn entry_bad(state: mut State) { // BAD: mutate through the raw pointer every iteration. for _ in 0..1000 { unsafe { (state).blocks[0] = (state).blocks[0].wrapping_add((state).blocks[1]); (state).blocks[1] ^= (state).blocks[2]; } compiler_fence(Ordering::SeqCst); } } /// # Safety /// /// `state` must be valid, properly aligned, and exclusively writable for the /// duration of this call. // SAFETY: the exported symbol name must be unique in the final linked program. #[unsafe(no_mangle)] pub unsafe extern "C" fn entry_good(state: mut State) { // GOOD: copy the array, work on the local copy, write back at the boundary. let mut blocks = unsafe { (state).blocks }; for _ in 0..1000 { blocks[0] = blocks[0].wrapping_add(blocks[1]); blocks[1] ^= blocks[2]; compiler_fence(Ordering::SeqCst); } unsafe { (state).blocks = blocks; } }</code></pre> A few Rust details are deliberate: #[unsafe(no_mangle)]</code> is the Rust 2024-compatible spelling of #[no_mangle]</code>; the safety obligation is that the exported symbol name does not collide in the final linked program.</li> The functions are unsafe extern "C"</code> because a raw pointer from outside Rust needs a safety contract.</li> The unsafe blocks are small, which keeps the actual raw-pointer operations visible.</li> [u128; 8]</code> is Copy</code>, so let mut blocks = (state).blocks</code> is a by-value copy.</li> compiler_fence</code> is compiler-only. It restricts memory reordering by the compiler but emits no hardware fence instruction of its own.</li> </ul> Build and inspect: rustc sroa.rs \ --crate-type=lib \ --edition=2024 \ -O \ -C target-cpu=native \ --emit=asm \ -o sroa.s</code></pre> On a current optimized build, the pattern to look for is not "zero memory instructions anywhere." The good version still has to load the initial state and store the final state. The key property is narrower and more important: the loop body should be register arithmetic, while the bad version should retain loads and stores in the loop. For LLVM IR: rustc sroa.rs \ --crate-type=lib \ --edition=2024 \ -O \ -C target-cpu=native \ --emit=llvm-ir \ -o sroa.ll</code></pre> In the good function, you may still see setup and write-back memory operations. Inspect the loop itself. The updated lanes should appear as SSA values carried through the loop, not as repeated loads from and stores to state</code>. 4. What a Barrier Proves in the Demo</h2> A round is one repeatable step of an algorithm. In this toy example, a round is just: blocks[0] += blocks[1] blocks[1] ^= blocks[2]</code></pre> In AEGIS-128L, the state update is built from AES round-function calls plus XORs with message blocks. The exact operation is different, but the code-generation problem is the same: several 128-bit lanes are updated repeatedly, and the fast path wants those lanes in registers. The toy round only touches three lanes so the assembly stays easy to read. The TigerBeetle update is denser: it rotates through all eight lanes and then mixes in the two message blocks. That makes the register-residency problem more important, not less. The demo barrier is a compiler barrier for memory operations: Rust's compiler_fence</code> emits no machine instruction, but it restricts compiler memory reordering.</li> Zig's empty asm volatile</code> with a .memory</code> clobber tells the optimizer that the assembly may touch arbitrary undeclared memory.</li> </ul> Neither one is a general inter-thread synchronization recipe. If you need synchronization, use the language's atomic operations and fences correctly. Here the barrier is only there to make the optimizer's memory behavior easy to see. The asymmetry is the useful part: Through pointer: load -> math -> store -> barrier, repeated each round Local copy: load once -> register math loop -> boundary write-back phase</code></pre> That is why the local-copy version can be much faster even though it appears to do an extra copy at the start. It pays a small boundary cost to avoid a large per-round cost. At the source level the boundary is one assignment, although the backend may lower that assignment to several scalar or vector stores. 5. Back to TigerBeetle: AEGIS-128L</h2> The actual TigerBeetle PR is tiny. In State128L.update</code>, the old code took a pointer to the state lanes: const blocks = &state.blocks;</code></pre> The new code copies the lanes into a local array: comptime assert(state.blocks.len == 8); var blocks: [8]AesBlock = state.blocks; const tmp = blocks[7]; inline for ([_]usize{ 7, 6, 5, 4, 3, 2, 1 }) |i| { blocks[i] = blocks[i - 1].encrypt(blocks[i]); } blocks[0] = tmp.encrypt(blocks[0]); blocks[0] = blocks[0].xorBlocks(d1); blocks[4] = blocks[4].xorBlocks(d2); state.blocks = blocks;</code></pre> The inline while</code> to inline for</code> cleanup is nice, but the important performance change is pointer-to-array -> array value. That gives LLVM a private aggregate to scalarize. The final assignment is one source-level write-back phase after the update; the backend may still emit several stores to materialize the array. Why does AEGIS respond so strongly? AEGIS-128L has a 1024-bit state made of eight 128-bit AES blocks, and the family is constructed from the AES encryption round function. On CPUs with AES instructions, the round operation wants to live in vector registers: x86 has AESENC</code> and AESENCLAST</code>, plus vector forms such as VAESENC</code>.</li> ARM has AES instructions such as AESE</code> and AESMC</code>.</li> </ul> Those instructions consume register operands and produce register results. If the surrounding code bounces every lane through memory around the AES instruction, throughput suffers. If the lanes stay in registers, the CPU can keep more AES operations in flight and avoid repeated load/store traffic. The PR body shows exactly that contrast. The old assembly repeatedly loaded two lanes, ran vaesenc</code>, and stored the result. The new assembly had several vaesenc</code> instructions running on registers before the final stores. The reported microbenchmark improved from aegis-old</code> wall time 0.10</code> to aegis-new</code> wall time 0.04</code>, with lower instruction count and higher IPC. The reported end-to-end benchmark moved the i8g</code> throughput metric from 359,253</code> to 387,293</code> and i4i</code> from 263,106</code> to 271,259</code>. That is a real systems-performance lesson: a local code-shape change in a crypto inner loop can move whole-application throughput when that loop is on the critical path. 6. How Far This Generalizes</h2> The pattern is broadly useful, but it is not a law of nature. Use it when the hot state is small enough and plain enough that copying it locally lets the optimizer remove repeated memory traffic. Good candidates: fixed-size arrays of integers or SIMD/vector blocks,</li> small POD-style structs,</li> cipher or hash states,</li> parser states,</li> tight loops where the same fields are updated repeatedly.</li> </ul> Important caveats: Register pressure still exists. If the local state is larger than the available register file, the allocator will spill some pieces. That can still be better than pointer traffic every round, but it is not free.</li> Copy cost can dominate. If you only touch one field once, copying the whole aggregate is probably the wrong move.</li> Observable memory is different. Do not apply this blindly to volatile memory, memory-mapped IO, atomics, or data that must be observed between steps.</li> Rust ownership matters. Copy</code> arrays are ideal. Moving non-Copy</code> values into locals may be possible, but types with Drop</code>, borrowing invariants, or interior aliasing often make the optimization less clean.</li> Measure the loop. The goal is not to see no memory instructions anywhere. The goal is to remove unnecessary memory traffic from the hot loop.</li> </ul> If the state is slightly too large, tile it: copy the hot subset into locals, work on that subset, write it back, then move to the next subset. Shorter live ranges often help the register allocator more than one giant local copy. 7. Takeaways</h2> SROA is the compiler's ability to split local aggregate storage into scalar SSA values.</li> Pointer-shaped code can hide whether memory is private, especially around opaque memory effects.</li> Copying a small aggregate into a local value gives the optimizer a private object it can scalarize.</li> In hot loops, the win is not the copy itself. The win is avoiding repeated load -> operate -> store</code> traffic.</li> In Rust, prefer safe &mut</code> APIs in ordinary code, and use raw pointers only at real FFI or low-level boundaries with explicit safety contracts.</li> In Zig, remember that [N]T</code> assignment copies the array value; slices and pointers keep referring to the original storage.</li> Always verify with optimized assembly or LLVM IR.</li> </ul> The deeper habit is simple: write the code so the optimizer can prove the thing you already know. "Copy to locals, work privately, write back at the boundary" is one of the most useful ways to make that proof obvious. References</h2> TigerBeetle PR #3201: Improve AEGIS codegen</a></li> TigerBeetle PR #3201 discussion: matklad on SROA</a></li> LLVM sroa</code>: Scalar Replacement of Aggregates</a></li> Rust compiler_fence</code></a></li> Rust 2024 unsafe attributes</a></li> Zig inline assembly and clobbers</a></li> The AEGIS Family of Authenticated Encryption Algorithms, IETF draft</a></li> NIST FIPS 197: Advanced Encryption Standard</a></li> Intel AES-NI overview</a></li> Compiler Explorer</a></li> </ul> Why x86 Zeroes a Register With `xor eax, eax` Unknown — Mon, 08 Jun 2026 00:00:00 +0000 Why x86 Zeroes a Register With xor eax, eax</code></h1> If you have ever disassembled a program - even a trivial one - you have almost certainly seen this line: xor eax, eax</code></pre> It looks like a riddle. The instruction says "XOR the eax</code> register with itself," but what it actually means is "set eax</code> to zero." In optimized code, compilers and assembly programmers commonly reach for it instead of the more obvious: mov eax, 0</code></pre> Both leave eax</code> holding zero. So why does the industry so often prefer the cryptic one? The short answer: it is smaller, it avoids a false dependency, and many CPUs have been taught to recognize it as a special "I want a zero" signal. The longer answer touches instruction encoding, out-of-order execution, register renaming, and one subtle correctness trap involving CPU flags. This post works through all of it - starting from the intuition and ending at the details an assembly author actually needs to get right. The Essence (for the impatient)</h2> xor eax, eax</code> and mov eax, 0</code> both set eax</code> to 0. Their effect on the register value is identical.</li> xor</code> wins on size (2 bytes vs. 5 for eax</code>) and dependency behavior (modern CPUs recognize it as a "zeroing idiom" that does not need the old register value).</li> They are not fully interchangeable: xor</code> modifies the CPU flags; mov</code> leaves them untouched. In flag-sensitive code, this difference matters.</li> </ul> If you only remember one thing: prefer xor reg, reg</code> to zero a general-purpose register, unless you need the surrounding flags preserved. Why XOR-with-itself Equals Zero</h2> XOR (exclusive OR) is a bitwise operation that compares two bits and returns 1</code> only when the bits differ: a</th> b</th> a XOR b</th></tr></thead> 0</td> 0</td> 0</td></tr> 0</td> 1</td> 1</td></tr> 1</td> 0</td> 1</td></tr> 1</td> 1</td> 0</td></tr> </tbody></table> Now apply that to a value XOR'd with itself. Every bit is being compared to a copy of itself, so every pair of bits is identical - and identical bits always produce 0</code>. Whatever was in eax</code>, eax XOR eax</code> is guaranteed to be all-zero bits. That is the trick: you do not need to know the old value to wipe it out. Reason 1: Smaller Code</h2> Machine instructions are bytes, and bytes have costs. Here is how the two encode for the eax</code> case: xor eax, eax → 31 C0 (2 bytes) mov eax, 0 → B8 00 00 00 00 (5 bytes)</code></pre> The usual shortest encoding of mov eax, 0</code> has to carry the literal value 0</code> as a full 32-bit immediate - four bytes of zeros riding along just to say "nothing." xor eax, eax</code> encodes the operation alone; the operands are registers, so there is no immediate to store. Small x86-64 note: writing a 32-bit register such as eax</code> also clears the upper half of the corresponding 64-bit register, rax</code>. So xor eax, eax</code> is also a common way to zero all of rax</code>; writing xor rax, rax</code> works too, but needs an extra REX prefix byte. Three extra bytes per zeroing sounds trivial. But zeroing registers is one of the most common things code does (loop counters, return values, clearing accumulators). Multiply three bytes across a large binary and you affect code density - how much real work fits in the instruction cache. Tighter code means fewer cache misses, which on a modern CPU is a real performance lever, not a micro-optimization fetish. Reason 2: The CPU Knows This Trick</h2> Here is where it gets interesting. On modern x86 processors, xor eax, eax</code> is not treated as a generic XOR that happens to produce zero. The hardware specifically recognizes the pattern xor reg, reg</code> as a zeroing idiom and gives it special treatment. To understand why that helps, you need two ideas from how modern CPUs actually run. Out-of-order execution and dependencies</h3> Modern CPUs do not execute instructions strictly one after another. They run many in flight at once, reordering them to keep the execution units busy. The constraint is data dependencies: if instruction B needs the result of instruction A, B has to wait for A. A normal XOR reads both its operands. So in principle xor eax, eax</code> reads the old eax</code> before overwriting it - which would force it to wait for whatever last wrote eax</code>. That is a false dependency: the result (zero) does not actually depend on the old value at all. CPUs are smart enough to know this. When they see xor reg, reg</code>, they recognize the result is unconditionally zero and break the dependency chain - the instruction does not wait on the previous eax</code> writer. This frees the out-of-order engine to schedule it immediately, which is exactly what you want in a tight, performance-critical loop. Register renaming - zeroing for "free"</h3> There is a second, deeper trick. Internally, the named registers you write (eax</code>, ebx</code>, …) are mapped onto a much larger pool of physical registers - a mechanism called register renaming. It is how the CPU runs many instructions concurrently without them stepping on each other's register names. On many microarchitectures, a recognized zeroing idiom can be handled entirely in or near this renaming stage: the CPU allocates a fresh physical register whose value is known to be zero. No integer ALU has to run, no arithmetic happens, and in the best case the instruction has zero execution latency. It is not literally free - it still takes bytes in the instruction stream and uses front-end/retirement resources - but it can disappear from the execution units. A mov eax, 0</code> is still cheap, but it is not the canonical dependency-breaking zero idiom and generally has to create an immediate value through the normal instruction path. So xor eax, eax</code> is not just smaller - the processor can sometimes make it vanish from the execution part of the pipeline. Reason 3: Convention and History</h2> The idiom is old. Early x86 processors already benefited from the smaller encoding, so assembly programmers and compiler authors adopted xor reg, reg</code> as the standard way to zero a register when flags did not need to be preserved. Hardware designers, in turn, optimized for the pattern they saw everywhere - a feedback loop that cemented it as the idiom. Today GCC, Clang, MSVC, and hand-written assembly all commonly use it in that role. The Catch: Flags Are Not the Same</h2> Now the nuance that trips people up - and the reason "functionally equivalent" is an overstatement. x86 has a set of status flags (EFLAGS in 32-bit mode, RFLAGS in 64-bit mode) that many instructions update as a side effect. They record things about a result: was it zero, was it negative, did it carry, and so on. Conditional jumps like jz</code> (jump if zero) read these flags. The crucial difference: mov eax, 0</code> touches no flags. A mov</code> only moves data. Every flag keeps whatever value it had before.</li> xor eax, eax</code> rewrites the flags based on its result (zero):</li> </ul> Flag</th> Meaning</th> After xor eax, eax</code></th></tr></thead> ZF (Zero)</td> result was zero</td> 1 (set)</td></tr> SF (Sign)</td> result was negative</td> 0</td></tr> PF (Parity)</td> low byte has even parity</td> 1 (zero has even parity)</td></tr> CF (Carry)</td> unsigned carry/borrow</td> 0</td></tr> OF (Overflow)</td> signed overflow</td> 0</td></tr> AF (Adjust)</td> auxiliary carry</td> undefined</td></tr> </tbody></table> So the two instructions leave the register identical but the flags in completely different states. When this bites</h3> Consider code that sets up a flag, then zeroes a register, then branches on that flag: ; ... something earlier sets ZF based on a comparison ... mov eax, 0 ; eax = 0, ZF is preserved from the comparison jz label ; branches on the EARLIER comparison's result</code></pre> Swap in xor</code> and the meaning changes: ; ... something earlier sets ZF based on a comparison ... xor eax, eax ; eax = 0, but ZF is now forced to 1 jz label ; ALWAYS branches - the comparison's result was clobbered</code></pre> The xor</code> version destroyed the flag the jz</code> depended on. This is a genuine correctness bug, not a style preference. Conversely, sometimes the flag side effect is useful - if you wanted ZF set anyway, xor</code> gives you the zero and the flag in one 2-byte instruction. So Which Should You Use?</h2> Reach for xor reg, reg</code> in the overwhelmingly common case: you want a zero, and you do not need any flags preserved across the zeroing. You get the smallest encoding and the dependency-breaking form compilers generally prefer. Reach for mov reg, 0</code> when surrounding code depends on the flags as they were before the zeroing - you are deliberately interleaving a register clear into a sequence that will branch on an earlier comparison. Here you pay extra bytes and a possible execution cost to keep the flags pristine. A practical heuristic: in straight-line code where the zeroing is followed by a fresh comparison anyway, the flag difference is invisible, so xor</code> is free and correct. The flags only matter when a later instruction reads a flag set by an earlier one, with the zeroing wedged in between. A More Accurate One-Liner</h2> The tempting summary - "xor eax, eax</code> and mov eax, 0</code> are equivalent" - is wrong in one important way. A precise version: mov eax, 0</code> and xor eax, eax</code> both set eax</code> to zero, but they are not fully equivalent: xor</code> sets ZF, clears CF and OF, sets SF and PF from the result, and leaves AF undefined, while mov</code> leaves all flags unchanged. In nearly all cases xor eax, eax</code> is preferred for its compactness (2 bytes vs. 5) and dependency-breaking zero-idiom recognition - unless preserving the existing flags is required. </blockquote> Takeaways</h2> Value: identical - both produce zero.</li> Size: xor</code> is 2 bytes, mov</code> is 5. Better code density, better instruction-cache behavior.</li> Speed: the CPU recognizes xor reg, reg</code> as a zeroing idiom - it breaks the false dependency and can resolve during register renaming on many cores, sometimes with zero execution latency.</li> Flags: the real asymmetry. xor</code> rewrites most relevant status flags and leaves AF undefined; mov</code> preserves them. This is architectural, not a microarchitecture-specific optimization, and it is the one place the choice can change program behavior.</li> Rule of thumb: zero with xor</code> by default; switch to mov</code> only when flags must survive the zeroing.</li> </ul> Inspired by x86 Internals for Fun & Profit • Matt Godbolt • GOTO 2014</a>. Set-Associative Caches: Trading Global Optimality for Predictable Speed Unknown — Mon, 01 Jun 2026 00:00:00 +0000 Set-Associative Caches: Trading Global Optimality for Predictable Speed</h1> Caches are everywhere in modern systems: CPU L1/L2 caches, database buffer caches, storage-engine object caches, and application-level LRU maps. They do not all make the same trade-offs. A set-associative cache sits at a particular point in the design space: it gives up global replacement freedom in exchange for bounded probing, compact layout, and predictable cache-line-friendly access. This post walks through what that means, why TigerBeetle uses a set-associative tier inside its CacheMap</code>, and how a focused optimization to its SIMD search path improved instruction-level parallelism. 1. What Is a Set-Associative Cache?</h2> A set-associative cache is a caching strategy borrowed from CPU architecture. It sits between two extremes: A fully associative cache lets an item live anywhere in the cache.</li> A direct-mapped cache maps each item to exactly one slot.</li> A set-associative cache divides the cache into sets, each containing a small fixed number of ways or slots. That fixed number is the cache's associativity.</li> </ul> The hash of a key determines which set the item belongs to. Within that set, the item may occupy any way. When the set is full, a local replacement policy chooses a way to evict. For example, a 4-way set-associative cache with 256 sets has 1024 total slots. Every key hashes to one of the 256 sets and can occupy any of that set's 4 slots. When the set overflows, one of those 4 entries is evicted. Benefits</h3> Predictable lookup cost. A lookup checks a small fixed number of slots. There are no long collision chains or unbounded probe tails.</li> Fewer conflict misses than direct mapping. Multiple keys that hash to the same set can still coexist.</li> Compact memory layout. Sets and their metadata can be packed tightly, which matters in low-level hot paths.</li> </ul> Trade-Off</h3> Unlike a fully associative cache, an item cannot live anywhere. If one set is hot and overflows, the cache evicts aggressively from that set even if colder entries exist elsewhere. 2. How It Differs from an Ordinary Software Cache</h2> A typical general-purpose software cache, such as a HashMap</code> plus a global LRU list, behaves differently: Items can be placed anywhere in the structure.</li> Lookups are usually average-case O(1)</code> through the hash table.</li> Eviction is global: the victim is chosen from the whole cache, not just one small set.</li> </ul> A set-associative cache, by contrast: Restricts a key's possible locations to one set.</li> Evicts locally from that set.</li> Usually has a fixed capacity and a small fixed associativity.</li> </ul> This gives up some hit-ratio flexibility. You may evict a warm item from an overfull set while colder items sit untouched elsewhere. The payoff is a simple, bounded, CPU-friendly memory access pattern. 3. Memory Layout: Hash Map vs. Set-Associative Cache</h2> A Pointer-Heavy Chained Hash Map</h3> One classic hash-table layout uses separate chaining: [Bucket 0] -> pointer -> [Node(key, value, next)] [Bucket 1] -> pointer -> [Node(key, value, next)] [Bucket 2] -> pointer -> [Node(key, value, next)] ...</code></pre> In that layout: Nodes are scattered across the heap.</li> Traversal requires repeated pointer dereferences.</li> Spatial locality is poor, because adjacent buckets do not imply adjacent keys or values.</li> </ul> An illustrative lookup looks like this: index = hash(key) % bucket_count node = buckets[index] while node != null: if node.key == key: return node.value node = node.next return null</code></pre> This is deliberately pseudocode. Labeling it as Rust would be misleading: idiomatic Rust code would call map.get(key)</code>, and Rust's std::collections::HashMap</code></a> is an open-addressed table rather than a chained node table. The sketch is here only to show the memory-access pattern of a pointer-heavy hash table. Modern hash maps are often much more cache-conscious than this. SwissTable-like metadata groups, Robin Hood probing, and contiguous slot arrays all improve locality. So the comparison is not "set-associative cache versus every hash map." It is "bounded set scan versus a layout that may chase pointers or probe an unbounded number of locations." Set-Associative Cache</h3> A 4-way set-associative cache with 256 sets can be laid out like this: Set 0: [Slot0][Slot1][Slot2][Slot3] Set 1: [Slot0][Slot1][Slot2][Slot3] Set 2: [Slot0][Slot1][Slot2][Slot3] ... Set 255: [Slot0][Slot1][Slot2][Slot3]</code></pre> The lookup shape is: set_index = hash(key) % set_count set = sets[set_index] for slot in set: if slot.is_present and slot.key == key: return slot.value return null</code></pre> Again, this is pseudocode. The important property is not the surface syntax but the access pattern: the loop touches one small contiguous region, usually one cache line or a small handful of cache lines. 4. Why Set-Associative Caches Are CPU-Cache Friendly</h2> Four properties combine to make this layout fast: Contiguous memory layout. Slots or metadata for a set live in a compact array. When the CPU loads a 64-byte cache line, it can bring in several pieces of relevant metadata at once. </li> Bounded probing. Probe length is a fixed constant: 4, 8, 16, or another small associativity. The pipeline does not have to chase an unpredictable chain. </li> Fewer pointer indirections. Sequential scans over compact arrays are friendlier to caches and prefetchers than scattered linked nodes. </li> Predictable branches. A short fixed loop with simple comparisons tends to be easier for the branch predictor than data-dependent pointer chasing. </li> </ol> Side by Side</h3> Aspect</th> Pointer-heavy chained hash map</th> Set-associative cache</th></tr></thead> Memory layout</td> Scattered, pointer-heavy</td> Contiguous, array-based</td></tr> Lookup locality</td> Often poor</td> Excellent within one set</td></tr> Cache-line efficiency</td> Low for chain traversal</td> Dense metadata and bounded scan</td></tr> Probe length</td> Variable</td> Fixed by associativity</td></tr> Eviction</td> External/global if used as a cache</td> Local to one set</td></tr> </tbody></table> Cache-Line Behavior</h3> Chained hash map: Set-associative cache: Load bucket pointer Load compact set metadata Jump to node Scan a bounded set of ways Maybe jump again Compare candidate slots</code></pre> The set-associative layout trades global freedom for a probing pattern the CPU can handle well. 5. If Set-Associative Caches Are So Good, Why Use Ordinary Caches?</h2> Set-associative caches win on raw lookup predictability and hot-path locality. General-purpose caches win on flexibility, richer policies, and often better hit ratios for messy application workloads. When Set-Associative Caches Shine</h3> Hot-path, low-latency systems: CPU caches, databases, storage engines, and similar systems code.</li> Workloads where bounded worst-case lookup work matters.</li> Fixed memory budgets.</li> Compact metadata and predictable memory access.</li> </ul> This is why CPU designers use set associativity for L1/L2 caches: hardware cannot tolerate pointer chasing or unbounded probe lengths in the common path. Their Weaknesses</h3> Local eviction can be suboptimal. A hot set may evict useful entries while cold sets remain underused. A global policy may achieve a better hit ratio at the same capacity. </li> Fixed associativity can thrash. If more than N</code> hot keys map to an N</code>-way set, the set can churn even when the total cache has spare-looking capacity elsewhere. </li> Replacement policies are usually simple. LRU-within-set, CLOCK-like policies, and random replacement are common. Feature-rich application caches may offer TTLs, weights, admission policies, async refresh, and other workload-specific behavior. </li> Capacity is usually fixed. General-purpose caches often grow, shrink, or resize under memory pressure. Set-associative caches are normally sized up front. </li> </ol> Why Ordinary Caches Still Dominate Application Code</h3> Application caches often care more about policy than nanoseconds. They need dynamic sizing, TTLs, weighted entries, observability, workload-specific admission rules, and global eviction behavior. A set-associative cache is a sharp tool for a hot path, not a universal replacement for a full cache library. Analogy</h3> A CPU L1 cache is fast, compact, and set-associative, but not globally fair.</li> An OS page cache uses broader replacement machinery because it is optimizing a larger and messier working set.</li> </ul> Application code usually wants the page-cache model. Systems code on a hot path often wants the L1-cache model. 6. Case Study: TigerBeetle</h2> TigerBeetle is a distributed financial accounting database where correctness, latency, and predictability matter. Its CacheMap</code> is a useful real-world example because it combines a fast set-associative tier with a bounded std.HashMapUnmanaged</code> stash, and because the set-associative tier has received careful low-level optimization. 6.1 CacheMap</code>: A Two-Tier In-Memory Cache</h3> CacheMap</code> lives in TigerBeetle's LSM lookup path. The cache_map.zig</code></a> source describes it as a hybrid between SetAssociativeCache</code> and a HashMap</code> stash: SetAssociativeCache</code> fast tier. A compact structure optimized for predictable lookup and insertion. Bounded associativity gives constant-time probing and local replacement. </li> HashMap</code> stash. A preallocated std.HashMapUnmanaged</code> used as an auxiliary tier. It catches values evicted from the set-associative tier and helps guarantee that prefetched values remain in memory during their respective commit. </li> </ol> This is reminiscent of the "stash" idea used with cuckoo hashing: a small auxiliary structure catches cases the primary structure cannot cheaply retain. The setting is different here, but the shape is similar. See the cuckoo-stash paper</a> for the underlying idea. Crucially, the stash is not a complete in-memory copy of the database. The lookup hierarchy in TigerBeetle's cache_map.zig</code> is: cache -> stash -> immutable table -> LSM</code></pre> A miss in CacheMap</code> is therefore not a correctness failure. CacheMap.get()</code> checks the set-associative cache and then the stash; the broader LSM lookup path can continue below that. Correctness belongs to the whole hierarchy, not to the stash alone. The stash's job is narrower: it gives hard in-memory availability for specific short-lived cases, such as prefetched values during commit, and catches values evicted from the fast tier. 6.2 Bounded by Capacity, Invalidated by Compaction</h3> A natural objection is: does the stash retain entries indefinitely? No. Two separate mechanisms matter: Bounded by construction. CacheMap.init()</code> calls ensureTotalCapacity()</code> with stash_value_count_max</code>, and later stash updates use assume-capacity operations. The code path is designed around fixed allocation rather than unbounded runtime growth. </li> Cleared by compaction. CacheMap.compact()</code> clears the stash while retaining its allocated capacity. The source comment is explicit that stash invalidation is handled by compact</code>. </li> Driven by the LSM's compaction cadence. TigerBeetle's LSM compaction runs incrementally in "beats" after commits. In Groove, the object cache is compacted on the last beat of the compaction bar, aligning cache invalidation with a deterministic storage-engine event. </li> </ul> The correctness invariant is intentionally narrow: call compact()</code> only when the stash entries no longer need to provide their in-memory guarantee. If the cache is undersized, the operational cost is more misses and IO, not unbounded stash growth. 6.3 TigerBeetle's SetAssociativeCache</code> Layout</h3> The current default layout in src/lsm/set_associative_cache.zig</code></a> is more specific than the generic examples above: 16 ways per set by default. The implementation currently allows 2, 4, or 16 ways for an efficient CLOCK-hand representation.</li> 8-bit tags by default, stored compactly for fast tag filtering. The code also supports 16-bit tags.</li> 2-bit per-way counts by default, used by a CLOCK Nth-Chance replacement policy.</li> Packed metadata. Tags, counts, and clock hands are laid out with cache-line-sized packing constraints. Values are stored separately and may be larger than a cache line.</li> </ul> A lookup proceeds as follows: Hash the key into a 64-bit entropy value.</li> Derive a short tag from that entropy.</li> Map the entropy to a set index.</li> SIMD-compare the set's tags against the probed tag.</li> Do full key comparisons only for ways whose tag matched and whose count is nonzero.</li> </ol> The tag compare is only a filter. An 8-bit tag can collide, so the final key_from_value()</code> comparison is mandatory. Two clarifications are worth keeping in mind: When the earlier sections talked about a "single cache line," that is most precise for compact set metadata. Full values may be too large to fit in one line.</li> TigerBeetle does not use plain LRU or random replacement here. It uses a CLOCK Nth-Chance</a> policy driven by packed per-way counts and a per-set clock hand.</li> </ul> 6.4 Performance Deep Dive: PR #3200</h3> PR #3200</a> restructured the SIMD search function to improve instruction-level parallelism and out-of-order execution. The benchmark deltas reported in the PR discussion are roughly +3.5%, +4.2%, +5.9%, and +5.2%, with improvements on both x86 and ARM machines and higher measured IPC. Change 1: SIMD Tag Compare to Integer Mask via @bitCast</code></h4> The set's tags are compared against the probed tag using a Zig vector comparison: const x: @Vector(layout.ways, Tag) = tags.; const y: @Vector(layout.ways, Tag) = @splat(tag); const result: @Vector(layout.ways, bool) = x == y; return @bitCast(result);</code></pre> The resulting boolean vector is bit-cast into an integer mask whose set bits represent candidate ways. For the default 16-way layout, that mask is a u16</code>. This is clean Zig and matches the current implementation. Change 2: Bit Iterator to Fixed-Trip-Count Masked Scan</h4> The old shape consumed the mask by repeatedly finding the next set bit with ctz</code>, clearing that bit, and looping. The new shape scans all ways and tests each bit: const ways: u16 = search_tags(set.tags, set.tag); if (ways == 0) return null; for (0..layout.ways) |way| { if ((ways >> @as(u4, @intCast(way)) & 1) == 1 and self.counts.get(set.offset + way) > 0) { if (key_from_value(&set.values[way]) == key) { return @intCast(way); } } } return null;</code></pre> The explicit @as(u4, @intCast(way))</code> is not decorative. In Zig, the right-hand side of a shift has a specific integer type. Since TigerBeetle's current layout caps the default search at 16 ways, u4</code> is enough to represent shift counts 0 through 15. The code comment says the fixed scan is intended to help out-of-order execution. The reason is dependency shape. The bit-iterator loop had a loop-carried dependency: each iteration needed the mutated mask before it could know the next candidate way. The fixed scan leaves each bit test independent, which gives the compiler and CPU more freedom to overlap count loads, value loads, and comparisons. Change 3: Modulo to fastrange</code>, and Simpler Tag Derivation</h4> The index and tag derivation also changed: const entropy = hash(key); const tag: Tag = @truncate(entropy); const index = fastrange(entropy, self.sets); const offset = index layout.ways;</code></pre> The important performance point is removing integer division from the hot path. TigerBeetle's stdx.fastrange()</code></a> implements Daniel Lemire's multiply-high range reduction</a>: multiply a u64</code> by the range as u128</code>, then take the high 64 bits. That is not the same function as %</code>, but it is appropriate for well-distributed hash-derived input when its bias properties are acceptable. Do not blindly replace %</code> with fastrange</code> for structured or biased integers. The TigerBeetle source includes a test named "fastrange not modulo" for exactly that reason. Why These Changes Help ILP and OoO</h4> A) Fixed trip count exposes independent work. The fixed loop tests (ways >> way) & 1</code> independently for each way. With no mask mutation between iterations, the compiler can unroll more easily, and the out-of-order engine can overlap more address generation, count loads, and candidate value checks. B) Removing ctz</code> plus bit-clearing shortens the serial chain. ctz</code> is fast, but the old loop used it as part of a stateful iterator. Each step depended on the previous mask update. The fixed scan turns that into many small, regular tests. C) Fixed control flow is friendlier to the front end. A fixed 0..ways</code> loop has a simple shape. Even when it does more tiny tests, the regularity can be better for branch prediction, scheduling, and unrolling. D) fastrange</code> removes a divider bubble. Integer division and modulo are high-latency operations on many CPUs. Replacing % self.sets</code> with multiply-high range reduction shortens the hash-to-index-to-load chain. E) More candidate checks can be in flight. The new code may perform more scalar bit tests than the old bit iterator, but modern superscalar CPUs often prefer more independent work over fewer serialized operations. That is the core trade-off the PR exploits. Why "More Work" Can Be Faster</h4> The fixed scan can test all 16 ways even if only one tag matched. On paper, that looks wasteful. On a modern CPU, however, sixteen independent tiny tests can be cheaper than a smaller number of dependent operations. The PR's benchmark results support that trade-off: throughput improved while IPC also improved. 6.5 Before vs. After: A Schematic Pipeline Timeline</h3> The timings below are illustrative. The exact cycle counts depend on the microarchitecture. The important shape is that a long serial operation disappears, and the candidate-scan loop loses a loop-carried dependency. Before</h4> Time -> 0 4 8 12 16 20 24 28 32 36 40 ------------------------------------------------------------------------ Index |------ integer DIV: entropy % sets ------| idx Tag shift/truncate ----------> tag Set addr AGU ------------> base Tags load L1/L2 -----> tags SIMD eq ---> mask Bit-iterator loop: Iter0: ctz(mask) -> way0; clearbit(mask) -> mask' load count[way0]; maybe load value[way0]; compare Iter1 waits for mask': ctz(mask') -> way1; clearbit(mask') -> mask'' load count[way1]; maybe load value[way1]; compare Iter2 waits for mask'' ...</code></pre> The divider is on the critical path before the set load. The bit iterator then limits overlap because each iteration depends on the previous mask update. After</h4> Time -> 0 4 8 12 16 20 24 28 32 36 40 ------------------------------------------------------------------------ Index fastrange multiply/shift -> idx Tag truncate(entropy) -------> tag Set addr AGU ----------> base Tags load L1/L2 --> tags SIMD eq -> mask via @bitCast Fixed loop, independent tests: Way0: (mask >> 0) & 1 load count[0] maybe load value[0] Way1: (mask >> 1) & 1 load count[1] maybe load value[1] Way2: (mask >> 2) & 1 load count[2] maybe load value[2] ... Way15: (mask >> 15) & 1 load count[15] maybe load value[15]</code></pre> The index is available sooner, and the candidate checks have less serial state. That gives the out-of-order engine more independent work to overlap. Critical-Path Summary</h4> Stage</th> Before</th> After</th> Why it matters</th></tr></thead> Index computation</td> %</code> integer division</td> fastrange</code> multiply-high reduction</td> Removes a long-latency divider from the path</td></tr> Candidate iteration</td> ctz</code> plus mask mutation</td> Fixed loop with independent bit test</td> Removes loop-carried dependency</td></tr> SIMD to scalar handoff</td> Pointer-cast-style handoff</td> @bitCast(result)</code></td> Clean conversion from vector result to mask</td></tr> Memory-level parallelism</td> Fewer candidates in flight</td> More candidates can overlap</td> Better latency hiding on cache misses</td></tr> </tbody></table> 7. Takeaways</h2> Fixed-count masked scans can beat variable-length bit iteration. If the old loop carries state from one iteration to the next, a fixed scan may expose more instruction-level parallelism even when it performs more tests. </li> Replacing %</code> on a hot path can matter. Multiply-based range reduction, such as fastrange</code>, can remove a high-latency divider when the input distribution makes it valid. </li> SIMD filters work well with scalar verification. Compare compact tags in parallel, turn the result into a mask, then run full key comparisons only for candidate ways. </li> A fast bounded tier can compose with a correctness-owning storage hierarchy. TigerBeetle's set-associative tier is a hot cache. The stash catches evictions and prefetched values. The LSM below remains the source of truth. </li> Invalidation is easier when tied to a deterministic system event. Clearing the stash during compaction keeps the memory story simple and bounded. </li> </ol> Set-associative caches are not a universal hammer. They trade global replacement freedom for bounded, predictable access. In a hot path like TigerBeetle's CacheMap</code>, that trade-off is exactly the point. Building a custom Bitset - A study in access-pattern-driven data structure choice Unknown — Mon, 25 May 2026 00:00:00 +0000 Building a custom Bitset - A study in access-pattern-driven data structure choice</h1> I was porting porcupine-rust</code> to Zig - the core of the implementation is a Bitset data structure, which is used in the implementation of a cache (see below for the details of why we need the cache). porcupine-zig</code></a> carries two bitset implementations and uses them in two different places: Site</th> Type</th> Mode</th> Purpose</th></tr></thead> src/bitset.zig</code> (custom)</td> Bitset</code></td> Release + Debug</td> Inner DFS in checker.zig</code> - cache-probed, cloned per miss, hashed every step</td></tr> assertPartitionIndependent</code> (std)</td> std.DynamicBitSet</code></td> Debug only</td> One-shot validation that a partitioning is disjoint, complete, and in-bounds</td></tr> </tbody></table> That is not duplication or inconsistency. It is the most basic discipline in performance engineering: pick the data structure that fits the access pattern of the call site, not the one that fits "bitset" as a category. The two sites have effectively zero overlap in what they ask of a bitset, and collapsing them onto a single type would either pessimise the hot path or over-fit the cold one. This document walks through both call sites, derives the requirements from the algorithm rather than from intuition, and shows why the resulting choices are strongly justified - not stylistic. A wrapper around std.DynamicBitSetUnmanaged</code> could in principle furnish part of the hot-path surface (its allocator-on-call-site shape, clone</code>, and eql</code> are already there), but it would still lack SBO and the virtual-mutation hash/equality protocol; closing that gap is most of the work, which is why the project carries a project-local type. For a complete explanation we need to start with a basic understanding of the linearizability-checker algorithm. 1. The hot path: linearizability DFS in checker.zig</code></h2> 1.1 What the algorithm actually does</h3> The Wing & Gong / Lowe linearizability check explored in checkSingle</code> is a depth-first search over the lattice of partial linearizations of an operation history. At each frame the algorithm: Picks an unlinearized call op_id</code> from a doubly-linked candidate list.</li> Steps the user-supplied model: step(state, input, output) -> ?State</code>.</li> If the step succeeds, asks the cache: have we already explored the (linearized ∪ {op_id}, new_state)</code> pair from any other ordering?</li> On cache miss: commit. Push the old state onto a backtrack stack, set the bit, store the new (bitset, state)</code> in the cache, descend.</li> On cache hit: prune. Backtrack and try the next candidate.</li> </ol> The cache is the entire point of the algorithm. Without it the search is super-exponential in history length; with it the redundant subtrees collapse and the search is tractable for histories of a few hundred operations. The cache key is the pair (set of linearized op ids, model state)</code>, with the bitset standing in for the set. This puts the bitset squarely in the DFS inner loop - touched multiple times per frame: Hashed every probe (linearized.hashWithBit(op_id)</code>).</li> Compared for equality every probe whose hash collides (bs.eqlWithBit(...)</code>, inside cacheContainsWithBit</code>).</li> Cloned every cache miss, before the bit is actually set on the clone (linearized.clone(arena_alloc)</code>).</li> Mutated on every step forward and every backtrack (linearized.set(op_id)</code>).</li> </ul> On workloads shaped like the Rust/Go etcd and KV fixtures, cache hits are expected to dominate mid-search (the upstream Go and Rust ports report high hit rates on these histories; a measured Zig hit-rate table belongs here once the corresponding fixtures land in porcupine-zig</code>). That distribution matters: any cost that lands on a probe is paid many times more often than any cost that lands on a commit. 1.2 Requirements derived from the access pattern</h3> From the algorithm we get a precise contract for the bitset: Cheap clone in the common size range. Histories the project actually targets are 50-256 ops; cloning must not allocate in this range, because misses still happen at every level of the search tree.</li> hash</code> that admits an O(1) update under single-bit mutation. Every probe asks "what would the hash be if I set bit p</code>?". If answering that requires materializing the mutated bitset, the probe pays the cost of the commit even when it's a hit.</li> eq</code> that can be evaluated against a virtually-mutated self</code>. Symmetric to (2): hash collisions need to be resolved without realising the mutation either.</li> Allocator passed at the call site, not stored. The DFS uses a per-worker arena (constructed inside checkSingle</code>) and tears the whole partition down with one arena.deinit()</code>. A bitset that hides an allocator pointer inside its struct fights that lifetime model.</li> Stable inline layout, predictable indexing. Hot inner loops want the chunk pointer in a register and bounds checks elided in release.</li> </ol> The standard library's std.DynamicBitSet</code> (and DynamicBitSetUnmanaged</code>) satisfy none of (1)-(3) and only partially (4)-(5). DynamicBitSetUnmanaged</code> already takes the allocator at the call site and exposes clone</code>/eql</code>, so the lifetime and basic-comparison story can be borrowed; but SBO and the virtual-mutation hash/equality protocol have no counterpart in the standard library, and adding them is most of the work of writing a project-local type. 1.3 How the custom Bitset</code> answers each requirement</h3> 1.3.1 Small-buffer optimization (SBO)</h4> Bitset</code> stores an inline_buf: [4]u64</code> field directly in the struct - 256 bits, aligned, contiguous, no indirection. The heap</code> slice is consulted only when the bit count exceeds 256, which on the project's workloads happens for exactly one fixture (and even there only for the un-partitioned variant). What this buys, concretely: clone</code> is a struct copy plus an optional allocator.dupe</code> that almost never runs. In the inline case it is one [4]u64</code> move + a couple of scalar fields. No allocator vtable dispatch, no free-list traversal, no fragmentation.</li> The active storage tends to live in 1-4 consecutive cache lines, often prefetched as part of the surrounding CacheEntry</code> row, rather than being a heap pointer that may sit on a cold line.</li> The arena allocator never sees most bitsets at all. On the partitions that fit inline the arena reset at end-of-partition is reclaiming state-clones and hash-table buckets, not bitset bodies.</li> </ul> The 256-bit ceiling is not arbitrary: 256 was chosen because (a) it covers the histories porcupine-zig</code> is benchmarked against (etcd ~170 ops → 3 chunks; KV per-partition ≤ 50 ops → 1 chunk, per the workload notes at the top of bitset.zig</code>), and (b) n / 64</code> is a single shift in code that indexes chunks, with inline_cap = 4</code> keeping the struct under one cache line on common architectures (Bitset</code> is 48 + 16 + 8 = 56</code> bytes, plus padding). std.DynamicBitSet</code> has no small-buffer optimization; for any nonzero runtime size its mask storage is heap-backed. The type is designed for arbitrary runtime sizes where the average case is large enough to not care about the allocation. For the DFS this is the wrong default: the average case is small enough that the allocation would dominate the operation cost. (One Zig-specific ownership caveat worth flagging while we're on the SBO discussion: Bitset</code> contains a slice field for the heap-spilled case, so a plain struct assignment of a heap-spilled bitset is a shallow copy that aliases the heap chunks. The source comments require callers that need a deep copy to go through clone(allocator)</code>. The DFS already does, but the discipline is part of what makes the explicit allocator-per-call shape work.) 1.3.2 Algebraically updateable hash</h4> The hash</code> formula is h = popcnt(bits) // initial value for each chunk c: h ^= c // fold chunks in</code></pre> This looks pedestrian, but the chunk fold has a property no off-the-shelf hash (FNV, xxHash, FxHash, SipHash) gives you: it is XOR-decomposable in the chunk basis. Setting a single bit p</code> mutates exactly one chunk word, from old_word</code> to new_word = old_word | (1 << minor)</code>, and the chunk fold's contribution shifts by exactly (old_word ^ new_word)</code>. The popcnt seed shifts independently - by popcnt XOR (popcnt + 1)</code>, which is 1</code> when popcnt is even and a longer carry chain when odd. hashWithBit</code> returns a closed-form lookup key that folds in the chunk delta and the even-popcnt shape of the popcnt delta: hashWithBit(p) = self.hash() ^ old_word ^ new_word ^ 1</code></pre> Two technical notes on what this actually is and is not: It is a self-consistent cache key, not the post-set hash()</code>. When popcnt is odd, the true popcnt delta is wider than 1</code> (e.g., 3 → 4</code> differs by 0b111</code>), so hashWithBit(p)</code> does not in general equal hash()</code> of the bitset after set(p)</code>. What it does guarantee is self-consistency: any two probes that would land on the same (linearized + bit, state)</code> cache entry have equal predecessor popcnts and equal post-set chunks, so they hash to the same value under this formula. Since insertion and lookup both go through hashWithBit</code>, the cache's hit rate depends only on self-consistency - and self-consistency holds for every popcnt</code> parity. (The function name and the unit test in bitset.zig</code> both suggest the stronger property; only the weaker one is load-bearing for the algorithm.)</li> It is O(chunks)</code>, not O(1)</code>, and currently makes two passes over the active chunks. self.hash()</code> first calls popcnt()</code>, which walks self.data()</code> to count set bits, and then loops over self.data()</code> again to fold the chunks. hashWithBit</code> adds the three-XOR adjustment on top. The structural saving in the hot path therefore comes from hashWithBit</code> not having to allocate, clone, or mutate, not from skipping the chunk scan. Fusing the popcount and the XOR fold into a single loop is a trivial micro-optimisation; a more substantive version that maintained popcnt</code> and the running chunk-XOR incrementally on set</code>/clear</code> - and used the proper popcnt XOR (popcnt + 1)</code> delta - would deliver true O(1)</code> hashing and post-set equivalence as a free side-effect. The current code chose simpler state.</li> </ul> The architectural point still stands: chunk-level XOR-decomposition is what makes any incremental scheme possible. Swapping in FxHash, xxHash, or SipHash would force every probe to materialize the post-set bitset and rehash from scratch - which is exactly what the Go original does. The hash, the SBO, and the deferred-clone protocol form one design; you can't extract any single piece and keep the others. The hash table the cache lives in (an std.HashMapUnmanaged</code> in the checker's CacheEntry</code> map) is keyed by a precomputed u64</code> produced by hashWithBit</code>, and the table uses the identity context U64IdentityCtx</code> to consume that u64</code> directly as the bucket hash. The library's default context would re-hash the u64</code>, layering an unnecessary mixing pass over a value that already came out of a hash-shaped formula. Note the precise claim: the stored key is the hashWithBit</code> value, not necessarily Bitset.hash()</code> of the post-set bitset (on odd predecessor popcounts those differ, per the parity caveat above). The identity context is correct because the cache key is precomputed, not because Bitset.hash()</code> is universally the table key. 1.3.3 Deferred-clone cache probing</h4> Equipped with hashWithBit</code>, the matching eqlWithBit</code> closes the loop. Equality between "self with bit p</code> set" and an existing cache entry is checked by walking the chunk arrays in lockstep and OR-ing the relevant word on the fly: const set_mask = @as(u64, 1) << ix.minor; for (a, b, 0..) |x, y, i| { const adj = if (i == ix.major) x | set_mask else x; if (adj != y) return false; }</code></pre> The hot-path call site is cacheContainsWithBit</code>: const h = linearized.hashWithBit(op_id); if (!cacheContainsWithBit(M, &cache, model, h, &linearized, op_id, &ns)) { // miss: now we pay for clone + set + cache insert var new_linearized = try linearized.clone(arena_alloc); new_linearized.set(op_id); ... }</code></pre> The structural consequence: on every probe that hits the cache, the algorithm never allocates a bitset, never mutates a bitset, and never copies its chunks into a temporary. It hashes the predecessor (two chunk passes today, as discussed above), adjusts with three XORs, walks one hash bucket's chain (eqlWithBit</code> synthesizing the post-set word inline), and prunes. What is deferred is the clone and the bit-set: for inline bitsets that's a [4]u64</code> struct copy plus a single OR; for heap-spilled bitsets it's an allocator.dupe</code> plus a memcpy plus an OR. The clone happens only on the cache-miss branch, where the bitset is about to be stored anyway and the allocation amortises against the work of the new DFS subtree. The size of the win on a probe-heavy workload is therefore proportional to the probe-to-commit ratio, which is the property the algorithm-level numbers from the Go and Rust ports describe. This optimisation is present in the Rust port (where it is named hash_with_bit</code>) and absent in the Go original, which clones-then-hashes on every probe. The performance gap on long histories is meaningful for that exact reason. std.DynamicBitSet</code> exposes no hash</code> method at all, let alone an algebraically-updateable one. Bolting it on would require deciding the hash formula, which immediately re-introduces the design choice that drives the custom type. 1.3.4 Allocator on the call site</h4> The bitset stores no allocator handle. Every allocating method takes one explicitly: pub fn init(allocator: std.mem.Allocator, n: usize) !Bitset pub fn deinit(self: Bitset, allocator: std.mem.Allocator) void pub fn clone(self: const Bitset, allocator: std.mem.Allocator) !Bitset</code></pre> This matches the DFS's lifetime model. checkSingle</code> constructs an std.heap.ArenaAllocator</code> per partition (per worker thread when partitions parallelise) and feeds every bitset, every state clone, every hash-map bucket through that arena. At the end of the partition the arena is reset in one operation; thousands of bitsets and tens of thousands of hash buckets are reclaimed together with no per-object teardown. std.DynamicBitSet</code> (the managed variant) embeds an Allocator</code> pointer in the struct. Storing thousands of these in the cache would cost one extra pointer per entry for a value that is constant across all entries. DynamicBitSetUnmanaged</code> solves that, but still leaves you without SBO, without hash</code>/hashWithBit</code>, and without eqlWithBit</code>. The unmanaged form is closer to the right shape, but at that point you would be writing the custom type around it. 1.3.5 Small, inlineable indexing primitives</h4> isSet</code>, set</code>, clear</code>, and index</code> are declared inline fn</code>, and the chunk-pointer selection is a single chunks > inline_cap</code> branch against a field that is constant for the lifetime of a given bitset. The index</code> helper splits pos</code> into (major, minor)</code> with pos / 64</code> and pos % 64</code>; since 64 is a power of two and pos</code> is a usize</code>, this is the canonical shape the compiler can lower to a shift and an AND</code>. In -Doptimize=ReleaseFast</code>, the debug.assert</code> bounds checks fall away and the inline-vs-heap branch is predictable enough to be cheap (and in many call sites, hoistable). The point is not a specific assembly sequence - that would require a disassembly listing this post does not include - but that the primitives are small and shaped to be optimised, which matters because they run once per node visited in the DFS. 1.4 Summary table for the hot path</h3> Requirement</th> Custom Bitset</code></th> std.DynamicBitSet</code></th></tr></thead> No allocation for histories ≤ 256 ops</td> Yes (SBO)</td> No</td></tr> hash()</code> defined</td> Yes, XOR-decomposable</td> No</td></tr> Probe a "virtually-mutated" key</td> Yes (hashWithBit</code>/eqlWithBit</code>)</td> Not expressible without re-implementing the type</td></tr> Allocator passed per call (arena-friendly)</td> Yes</td> Managed: no. Unmanaged: yes (but still missing the above).</td></tr> Wire-compatible hash with Go/Rust ports</td> hash()</code> formula matches; hashWithBit</code> is a self-consistent cache key, not always equal to the post-set hash()</code> (parity caveat)</td> N/A</td></tr> </tbody></table> 2. The cold path: assertPartitionIndependent</code></h2> 2.1 What it is for</h3> assertPartitionIndependent</code> validates a debug-mode invariant: when a model declares a partition</code> hook, the returned list of partitions must be a partition of [0, N)</code> in the mathematical sense. Specifically: Disjoint - no index appears in two partitions.</li> Complete - every index in [0, N)</code> appears at least once.</li> In-bounds - no index is ≥ N</code>.</li> </ol> A bug in a user-supplied partitionEvents</code> could otherwise cause the DFS to silently skip events (violating soundness) or read past the history (a panic, but not a useful error). 2.2 Access pattern</h3> The function runs at most once per checkOperations</code> / checkEvents</code> invocation that actually receives partitions from the user-supplied partition</code> / partitionEvents</code> hook, and it compiles to a no-op outside Debug mode. When it does run, it performs: One allocation of a bit array of length N</code>.</li> One pass over all partitions; for each index, one bounds check, one isSet</code> test, one set</code>, one increment.</li> One terminal equality check on the count.</li> One deinit.</li> </ul> That is it. No clone, no hash, no equality, no nested probes, no per-step mutation across DFS frames, no allocator threading. The function exists to fail loudly on invalid partitionings, not to be fast. 2.3 Requirements derived from the access pattern</h3> Reading the algorithm rather than the type: Allocate N</code> bits, set each, query each, count. Nothing else.</li> Lifetime is one stack frame. No need for arena routing or for the bitset to outlive the function.</li> Performance is irrelevant. This path is removed in release builds.</li> Code clarity is paramount. This is an assertion: future readers should be able to see at a glance that the three invariants are enforced.</li> </ol> 2.4 Why std.DynamicBitSet</code> is the right choice here</h3> It is in the standard library. No reader needs to chase project-local semantics to understand what set</code>/isSet</code>/initEmpty</code> mean. The standard library type is the lingua franca; using it is a hint that nothing special is going on.</li> It exposes standard, familiar operations and no cache-specific protocol. std.DynamicBitSet</code> actually has a larger surface than this site uses (range ops, set algebra, iterators, clone), but crucially it has no hashWithBit</code>, no virtual-mutation invariants, no "bit must be currently clear" precondition to violate, no SBO branch to reason about. Nothing about its API hints at the cache-probe protocol. The shared vocabulary is the win, not minimalism.</li> One allocation, one deinit, simple lifetime. ~N/8</code> bytes via the caller's allocator, released by a single defer bits.deinit()</code>. SBO would be unused: the function holds exactly one bitset, ever, so an inline buffer would not save a heap allocation per clone - there are no clones.</li> The three invariants compose elegantly with bit semantics. Disjoint = assert(!bits.isSet(idx))</code> before set</code>; in-bounds = assert(idx < expected_total)</code>; complete = count == expected_total</code>. The bitset is doing exactly the work a set membership test should do, with a tight, obvious mapping from invariant to line of code.</li> </ul> 2.5 Why the custom Bitset</code> would be wrong here</h3> This is the more interesting direction. The custom Bitset</code> would mechanically work - init</code>/set</code>/isSet</code>/deinit</code> are present and correct. But choosing it would be a category error: The custom type carries preconditions tuned to the cache protocol. hashWithBit</code> requires the probed bit to be currently clear; the debug.assert</code>s inside hashWithBit</code> and eqlWithBit</code> enforce that. None of this matters here, but importing the type into a debug-only path creates the implicit suggestion that the whole protocol is in scope. Future maintainers reading the assertion code would have to figure out why.</li> SBO is dead code on this path. The function holds exactly one bitset of size N</code>. The inline buffer either adds 32 bytes of stack that go unused (when N > 256</code>) or replaces a single ~N/8</code>-byte allocation that nobody cares about (when N ≤ 256</code>). The optimisation is invisible at the call site and pays nothing.</li> The custom hash is unused machinery. A bitset whose hash</code>/eql</code> facilities are never called is a worse fit than one that does not have them; the API surface is wider than the requirement, and unused API surface attracts misuse.</li> Wire compatibility with Go/Rust does not apply. This is a Zig-only invariant check; there is no peer port doing the same check whose hash output we need to match.</li> </ul> The custom type was designed for a specific, demanding access pattern. Using it where that pattern doesn't exist would be like reaching for a B-tree to store five elements: it works, but it advertises a complexity class the data does not have. 3. When to revisit this decision</h2> A choice this load-bearing should come with explicit triggers for re-evaluation. The custom Bitset</code> is justified given the current shape of the algorithm and workload; if either shifts, the cost/benefit calculation shifts with it. Concrete triggers: The DFS stops doing virtual-mutation probes. If cacheContainsWithBit</code> is removed - say, the cache key changes shape, or the algorithm switches to a different pruning strategy - then hashWithBit</code>/eqlWithBit</code> lose their callers, and with them the constraint that pins the hash to a specific XOR-decomposable formula. At that point std.DynamicBitSetUnmanaged</code> plus a thin hash()</code> helper covers the remaining requirements. The custom type is no longer load-bearing; deleting it would be a strict simplification.</li> Typical histories outgrow the inline buffer. SBO is calibrated for n_ops ≤ 256</code>. If the project's target workloads shift to consistently larger histories - e.g., new fixtures in the few-thousand-ops range - the inline buffer becomes dead weight rather than a fast path, and the case for SBO weakens. The right response there is probably to retune inline_cap</code> rather than abandon the custom type, but the question is worth asking.</li> The cache hit rate collapses. The deferred-clone optimisation pays off in proportion to the probe-to-commit ratio. If a future model shape produces hit rates closer to 50/50, the savings from hashWithBit</code> shrink (though SBO continues to pay on the misses). This is a tuning signal, not a redesign signal - but it's a place to measure rather than assume.</li> Zig's standard library grows what we built. If a future std</code> ships a small-buffer-optimized bitset or a bitset with an algebraically-updateable hash, the custom type becomes a candidate for deletion. Until then, neither feature is in std</code>, and external dependencies are not worth introducing for a 330-line file.</li> The arena lifetime model goes away. The "allocator on the call site" discipline matters because every bitset in the cache is owned by a per-partition arena. If the checker moves to a different memory strategy - long-lived caches across partitions, persistent search trees, anything - the unmanaged-allocator argument weakens, and a managed type becomes more reasonable.</li> </ol> Symmetrically, the std-bitset choice in assertPartitionIndependent</code> is trivially robust: it would only need to change if the function were promoted out of debug-only mode, at which point the access-pattern analysis would have to be redone from scratch. 4. The general principle</h2> The two call sites in this project illustrate a discipline that is easy to state and consistently hard to follow: data structure choice should be driven by access pattern, not by category. "Bitset" is not a specification. The questions that produce a specification look like: How often is the structure mutated relative to how often it is read?</li> Is it cloned? At what rate? With what allocator?</li> Is it hashed or compared? Is the hash on the critical path?</li> Does the algorithm ask "what would the structure look like if I applied operation X</code>?" without committing to X</code>? (If so, the structure may need primitives for virtual mutation.)</li> What is the lifetime - one stack frame, one DFS subtree, one partition, one process?</li> What hardware-level effects matter - cache lines, branch prediction, allocator contention?</li> </ul> For the DFS in checker.zig</code>, the answers point to a small custom type with SBO, an algebraically-updateable hash, deferred-clone probe primitives, and an arena-friendly allocator interface; given the constraints, that is the simplest and most controllable implementation in this codebase. For assertPartitionIndependent</code>, the answers point to the simplest bit-of-storage primitive in the standard library. These are not opposing aesthetics; they are the same aesthetic - fit the tool to the work - applied to different work. The fact that one site needs a finely-engineered custom type does not mean every other bitset use in the project should reach for it. The fact that the standard library has a perfectly adequate DynamicBitSet</code> does not mean the hot path should give up SBO, an updateable hash, and virtual-mutation probing to use it - those primitives, not a generic "slowdown" headline, are the load-bearing pieces, and the right way to size their value is a benchmark table comparing the current code against (a) a custom Bitset</code> without virtual-mutation probing and (b) std.DynamicBitSetUnmanaged</code> plus clone-then-hash. The two bitsets in this codebase exist because the two call sites have disjoint requirements. Picking one tool for both would either pessimise the algorithm that defines the project's performance, or import a specialised type's invariants into a debug-only assertion that has no use for them. Keeping them separate is the boring, correct answer. Zig Allocation Patterns Unknown — Sun, 17 May 2026 00:00:00 +0000 Zig Allocation Patterns</h1> A reference for the allocation and ownership patterns used in porcupine-zig</code></a>, plus the broader landscape of Zig allocator idioms. Examples cite types and functions in the codebase by name so the patterns can be grounded against real code. But the discussion in this post stands on its own and does not require reading those sources. 1. The two ownership models: aware vs. owning</h2> Both patterns satisfy the project rule "every allocating function takes an explicit std.mem.Allocator</code>." The split is about whether the type also stores a copy of that allocator inside itself. 1.1 Allocator-aware (Unmanaged)</h3> The type holds no allocator field. Every method that allocates or frees re-receives one as a parameter, and the caller is responsible for passing the same allocator on init</code>, clone</code>, and deinit</code>. var bs = try Bitset.init(arena_alloc, 256); defer bs.deinit(arena_alloc); // caller re-supplies arena_alloc const cp = try bs.clone(arena_alloc); // and again</code></pre> Use it when: Many short-lived instances share one allocator. The custom Bitset</code> used by the checker clones constantly inside the DFS - one per cache entry, often hundreds or thousands per partition. All of them live and die under the same per-worker arena. Storing a 16-byte std.mem.Allocator</code> (vtable pointer + state pointer) on every instance would inflate the type for zero information gain - it's the same allocator every time. </li> The type is small and SBO-sensitive. Bitset</code> deliberately keeps a [4]u64</code> inline buffer to dodge heap allocation for histories ≤ 256 ops. Adding 16 bytes of allocator metadata to a struct whose whole point is staying compact would defeat the optimization. </li> The instance is embedded inside another struct that already owns its own lifetime story. Putting an ArrayListUnmanaged</code> inside a struct lets the outer struct drive deinit</code> cleanup with whichever allocator it already has on hand. </li> You want the type to be trivially copyable as a value. A Bitset</code> can be assigned by value (with the documented caveat that the heap</code> slice is a shallow alias - both copies will point at the same heap buffer). A managed type with an embedded allocator complicates the question of who owns what after a copy. </li> </ol> The standard library's ArrayListUnmanaged</code>, HashMapUnmanaged</code>, BufMap</code> family all follow this rule - and as of Zig 0.14+ the unmanaged variants are the default; the managed wrappers were largely removed. 1.2 Allocator-owning (Managed)</h3> The type stores its allocator at construction time and exposes a parameterless deinit()</code>. var arena = try NodeArenaOf(M).fromEntries(gpa, entries); defer arena.deinit(); // no allocator argument needed</code></pre> Use it when: There's exactly one (or a small handful of) long-lived instances. The checker's NodeArena</code> is created once per partition, holds one bulk []Node</code> allocation, and has one well-defined teardown site. The 16-byte cost is paid once; the duplication argument doesn't apply. </li> The deinit site is far from the init site. When a struct travels through several layers of code (returned from a builder, stashed in a context, eventually torn down by a defer</code> near the top), threading the right allocator back to the deinit call is a footgun. Storing it eliminates the question "which allocator did this come from?" entirely. </li> The deinit shape needs to be compatible with defer</code> without a closure. defer arena.deinit()</code> is a one-liner; defer arena.deinit(...)</code> requires the allocator to still be in scope and unambiguous. </li> The owned resource is non-trivial enough that mismatched-allocator bugs would be expensive. Freeing a []Node</code> with the wrong allocator is undefined behavior; storing the allocator removes the chance. </li> </ol> Standard-library precedent: std.heap.ArenaAllocator</code> itself is managed (it stores the child allocator). std.process.ArgIterator</code> on most platforms stores its allocator. std.Build</code> stores allocators throughout. 1.3 The mental model: cost vs. risk</h3> The choice reduces to a tradeoff between two costs: Cost</th> Aware</th> Owning</th></tr></thead> Per-instance memory (allocator field)</td> 0 bytes</td> 16 bytes</td></tr> Risk of mismatched allocator at deinit</td> Caller's burden</td> Eliminated</td></tr> Trivially copyable as a value</td> Yes (with caveats)</td> No (semantic gotcha)</td></tr> Deinit ergonomics</td> x.deinit(alloc)</code></td> x.deinit()</code></td></tr> </tbody></table> Aware wins on memory and copy semantics. Owning wins on safety and ergonomics. Population size and lifetime usually decide it: many short-lived → aware, few long-lived → owning. 2. Other Zig allocation patterns</h2> The aware/owning axis is just one dimension. Several other patterns are in active use across the standard library and this codebase. 2.1 Arena allocation (lifetime-bundled)</h3> std.heap.ArenaAllocator</code> - many objects share one allocator, and freeing happens only in bulk (deinit</code> or reset(.retain_capacity)</code>). Per-object free</code> is a no-op. Used pervasively here: WorkerCtx.run</code> holds one arena per worker, resets it between partitions, throws it away on exit. The arena itself is allocator-owning (it stores its child allocator); the things it allocates for are usually allocator-aware (Bitset</code>, hashmaps). This is orthogonal to aware/owning: an arena is a strategy, while aware/owning is a contract. When arenas are wrong: when individual objects need to outlive each other in unpredictable orders, or when objects hold non-memory resources (file handles, locks) that need explicit release. Arenas reclaim memory only. 2.2 Fixed-buffer / stack-backed allocators</h3> std.heap.FixedBufferAllocator</code> wraps a [N]u8</code> and bump-allocates inside it. No heap touch at all. Useful for parsers, formatters, comptime-bounded scratch - not used here, but the SBO pattern in Bitset</code> is the manual cousin: a [4]u64</code> inline buffer that dodges allocation entirely until you exceed it. FixedBufferAllocator</code> returns error.OutOfMemory</code> when the buffer is exhausted, so callers still need to handle the fallible path; the win is no syscall, no heap fragmentation, and no thread-safety concerns. 2.3 Allocator-free types</h3> Types that allocate nothing: pure value types or types backed by a caller-provided slice. Bitset.data()</code> is a borrowed view over the active chunks; the project's Operation</code>, Event</code>, and CheckResult</code> types are POD. No init</code>/deinit</code> ceremony, no allocator threading. Always prefer this when feasible - the best allocation is no allocation. 2.4 Caller-provided buffer ("BYO storage")</h3> The function takes a []T</code> from the caller and writes into it; ownership stays with the caller. std.fmt.bufPrint</code>, std.fs.File.read</code>. The function never allocates. Useful when the caller already has a buffer or wants tight control. Bitset.dataMut()</code> exposes this shape on the read side. The pattern composes well with fixed buffers: a caller can stack-allocate a [1024]u8</code>, hand a slice of it to a library, and never touch the heap. 2.5 Bulk single-allocation containers (the "index arena")</h3> NodeArenaOf</code> is itself a pattern: instead of N small Node</code> allocations linked by pointers, do one alloc([]Node)</code> and link nodes by u32</code> indices. Halves per-node footprint, makes free a single call, plays nicely with arena resets, and keeps everything cache-local. A none_ref = std.math.maxInt(u32)</code> sentinel replaces ?u32</code> at zero cost - u32</code> and ?u32</code> have different sizes in Zig (the latter carries a discriminator), so picking a value out of the valid range and treating it as "null" recovers the tighter packing while keeping the semantics. This is closer to a layout pattern than an allocator pattern, but it exists because of how Zig encourages explicit allocator thinking. In a language that hides allocation, a pointer-linked list and an index-linked list look the same; in Zig the cost difference is right in the source. 2.6 Two-phase reservation (capacity hints + assume-capacity)</h3> ensureTotalCapacity</code> (fallible) followed by appendAssumeCapacity</code> (infallible). Lets you front-load OOM into one reservation and then write a tight loop that can't fail. The codebase uses it where the final size is known up front - for example, building the per-partition entry array and op-id list inside checkOperations</code>, and the event-renumbering loop in the model layer. This is allocator-discipline rather than ownership, but it's a distinct idiom: it splits "can this fail?" from "will this run?", which composes better with errdefer</code> than a sequence of fallible appends. 2.7 Scratch / two-allocator split</h3> Pass two allocators: one for the long-lived output, one for the temporary working set. Useful whenever a function builds a result via a transient hashmap or buffer that is discarded before return - e.g., a function that renumbers events through an id-remap hashmap and then emits a new slice: the slice must outlive the call, the hashmap must not. The shape is fn foo(out_alloc: Allocator, scratch: Allocator, ...)</code>. Callers who have an arena handy pass it as scratch</code>; callers who don't can pass the same allocator to both arguments and pay the per-allocation cost. The win is that callers who do have an arena get bulk reclamation of the scratch state for free, without the function having to know an arena exists. 2.8 Failing allocator / testing wrappers</h3> std.testing.allocator</code> (leak detector), std.testing.FailingAllocator</code> (OOM injection at the Nth allocation), std.heap.GeneralPurposeAllocator(.{})</code> with .retain_metadata = true</code>. These wrap any other allocator and observe its calls. Their value evaporates if any code on the hot path bypasses the allocator the test handed in - for instance, a worker that hardcodes std.heap.page_allocator</code> instead of taking the allocator from its parent will silently skip leak detection for everything it allocates. The discipline pairs with §3.1 below: every layer must accept an allocator parameter, even when the "obvious" choice would be a global. FailingAllocator</code> is the standard way to test OOM paths: configure it to fail on the Nth allocation, run the function, assert it returns error.OutOfMemory</code> cleanly with no leaks. The HashMapKvModel</code> OOM tests in this project use exactly this pattern to exercise the errdefer</code> unwinding inside step</code> and clone</code>. 2.9 Concrete backing allocators (page / C / smp)</h3> The patterns above are strategies; at the bottom of the stack sits a concrete allocator that actually returns memory. Three from the standard library show up in real code, each picked along a different axis: std.heap.page_allocator</code> goes straight to mmap</code> / VirtualAlloc</code>. No fragmentation tracking, allocations rounded up to page size (typically 4 KiB). Cheap for one big request, expensive per call. Almost always wrong for small, frequent requests - that's what GeneralPurposeAllocator</code> is for - but it shines as the backing store for arenas: the arena amortizes the page-grain cost across many small client requests, so page_allocator</code> is only called when the arena actually grows. The per-worker arenas in this codebase use it for exactly that reason. </li> std.heap.c_allocator</code> calls malloc</code> / free</code>. Available only when linking libc. Useful for FFI scenarios where Zig-allocated memory crosses into C code, since C code expects to call free</code> on it. Outside FFI, no edge over GeneralPurposeAllocator</code>. </li> std.heap.smp_allocator</code> (Zig 0.14+) - a thread-safe, general-purpose allocator suitable for multi-threaded code. The natural fit for "I have N worker threads and I want each to be free to allocate without coordinating": callers can pass smp_allocator</code> directly without wrapping their own GPA in a thread-safe shim. In this project's parallel partition dispatch, the worker threads each hold their own arena, but the backing allocator for those arenas needs to be safe to call concurrently - that's the slot smp_allocator</code> fills. </li> </ul> 3. Anti-patterns and pitfalls</h2> 3.1 Hidden global allocators</h3> Resist the urge to define const gpa = std.heap.page_allocator;</code> at module scope and reach for it from every function. The whole point of Zig's explicit-allocator convention is that the caller decides allocation strategy: arena, GPA, fixed buffer, leak-detecting test wrapper. A hidden global silently defeats all of those. The one defensible exception is a top-level main</code> that picks an allocator and threads it down - but even that is one decision, not a pattern. 3.2 Mismatched allocators on aware types</h3> Bitset.deinit(other_alloc)</code> when init</code> ran with arena_alloc</code> is undefined behavior. The aware contract puts the burden on the caller, and Zig has no compile-time check that catches it. Mitigations: Pair defer</code> with the allocation site so they're visually adjacent.</li> Don't pass aware types across module boundaries without documenting the allocator contract.</li> When in doubt, prefer owning: the cost is 16 bytes, the benefit is the whole class of bug going away.</li> </ul> 3.3 Shallow copies of aware types with heap state</h3> Bitset</code> is a value type; var b2 = b1;</code> shares the heap</code> slice. Freeing either invalidates the other. The codebase deliberately holds bitsets through pointers - for example, iterating with for (entries.items) |e|</code> rather than |e|</code> - to avoid making an aliased copy on every loop step. Bitset</code>'s doc comment flags the hazard; without that comment, the bug class is invisible at the call site. 3.4 Per-element deinit under an arena</h3> If an arena is going to reclaim everything in one call, walking each element and free</code>ing it first is pure waste - and worse, it's misleading documentation: a reader sees the loop and assumes the elements have non-trivial cleanup, which forces them to reason about ordering, partial failure, and reentrancy. The fix is either to delete the loop and document the arena assumption at the allocation site, or - if the type genuinely might hold non-memory resources like file handles - gate the loop behind a comptime marker on the type so the dead version is removed in builds where the elements really are arena-safe. 3.5 Storing an allocator inside a thread-shared struct</h3> If multiple threads share a struct that owns its allocator, and the allocator isn't thread-safe (GeneralPurposeAllocator(.{})</code> defaults are single-threaded), every allocation is a race. Either pick a thread-safe allocator (smp_allocator</code>, c_allocator</code>) or give each thread its own arena, as WorkerCtx</code> does. 3.6 Returning slices from a function whose allocator goes out of scope</h3> The classic: a function creates a FixedBufferAllocator</code> on its stack frame and returns a slice allocated from it. The frame unwinds, the buffer is gone, the slice is dangling. Either return ownership with toOwnedSlice</code> (backed by a heap allocator that outlives the call), or have the caller pass in the buffer. 4. Quick decision table</h2> Pattern</th> When it fits</th></tr></thead> Allocator-aware (Unmanaged)</td> Many small instances; SBO-sensitive; embedded in a parent that drives deinit</td></tr> Allocator-owning (Managed)</td> Few long-lived owners; deinit site distant from init; mismatch is dangerous</td></tr> Arena</td> Many small lifetimes coterminous with a single phase</td></tr> Fixed-buffer / SBO</td> Bounded size known at comptime or by domain heuristic</td></tr> Allocator-free</td> POD / borrowed views</td></tr> BYO buffer</td> Caller already has storage; library doesn't need policy</td></tr> Index-arena (one bulk alloc)</td> Graphs / linked structures of homogeneous nodes</td></tr> Two-phase reserve + assume</td> Hot-loop appends where OOM should be front-loaded</td></tr> Scratch + output split</td> Function with transient working set + persistent result</td></tr> Failing/leak-detecting wrapper</td> Tests; OOM-path coverage</td></tr> Page allocator</td> Backing store for arenas; rare direct use</td></tr> C allocator</td> FFI boundaries with C code</td></tr> Smp allocator</td> Multi-threaded code without a per-thread arena</td></tr> </tbody></table> 5. How the patterns compose in porcupine-zig</code></h2> A single checkOperations</code> call exercises most of the table: Caller's allocator (any) is handed in for the entry array and partition slices - the long-lived outputs that survive the call.</li> Per-worker arena wraps page_allocator</code> and serves every DFS-internal allocation: bitsets, hashmap nodes, cloned states, linearization records.</li> Allocator-aware Bitset</code> is allocated inside the arena, cloned thousands of times during DFS, and never individually freed - the arena reclaims it all on reset</code> between partitions.</li> Allocator-owning NodeArena</code> holds the doubly-linked-list of call/return nodes for the whole partition, allocated from the caller's allocator (one bulk alloc) and freed once at the end.</li> Two-phase reserve + assume in checkOperations</code> and dedupe</code> pre-sizes the entry array and op-id list so the inner loop can't OOM.</li> Index arena layout in NodeArenaOf</code> keeps the linked-list traversal cache-friendly with u32</code> links and a none_ref</code> sentinel.</li> Allocator-free types (Operation</code>, Event</code>, CheckResult</code>) flow through the API boundary without any allocation ceremony.</li> </ol> Each pattern is doing the job it's best at, and the boundaries between them match natural lifetime boundaries in the algorithm. That's the goal: pattern selection should fall out of "what lives how long?" rather than being imposed top-down. 6. Further reading</h2> Zig standard library source: lib/std/heap.zig</code> and lib/std/mem/Allocator.zig</code> for the canonical implementations of every allocator mentioned here. The Allocator</code> interface in particular is worth reading end-to-end - it is short, and seeing how the vtable is laid out makes the "16 bytes" cost in §1 concrete.</li> The Zig language reference's "Memory" chapter, for the rationale behind explicit-allocator passing as a language design choice.</li> </ul> Why Zig’s Io Feels Like an Effect System (Without Being One) Unknown — Mon, 11 May 2026 00:00:00 +0000 Why Zig’s Io</code> Feels Like an Effect System (Without Being One)</h1> Context: What does "effect" even mean?</h2> The word is overloaded, and most arguments about whether something "is" an effect collapse the moment you fix a sense. At least four are in play: A semantic side effect - reading a file, mutating memory, throwing.</li> A type-level effect annotation - Koka-style effect rows, where the function type itself names which effects may occur.</li> An algebraic operation plus handler - a perform</code>/handle</code> mechanism that reifies the continuation and lets a handler decide whether, when, or how often to resume it (OCaml 5, Eff, Koka).</li> A capability value - an explicit value that grants the authority to perform an action, threaded through call sites (object-capability style).</li> </ol> These can stack but they are independent. Koka has all four. OCaml 5 has (1)+(3) but not (2): its manual is explicit that unhandled effects raise Effect.Unhandled</code> at runtime, with no static effect safety. Zig Io</code> is squarely (1)+(4). When this document says "effect" without qualification, it usually means sense (3) - the algebraic, handler-based shape that the term carries in PL theory. The opening question: is Io</code> an effect?</h2> Zig 0.16 (April 2026) introduced an Io</code> interface that you thread through any function that performs I/O. The release notes phrase it bluntly: "all input and output functionality requires being passed an Io</code> instance." The threaded implementation (Io.Threaded</code>) is feature-complete today; evented, io_uring</code>, kqueue</code>, and Grand-Central-Dispatch backends range from work-in-progress to proof-of-concept; std.testing.io</code> is the recommended fake for tests. A typical signature looks like: fn readConfig(io: Io, path: []const u8) ![]u8</code></pre> The same function body is intended to work across different Io</code> implementations - the threaded backend today, evented or platform-specific backends as they mature - without splitting code into async</code> and sync</code> variants. This is the same engineering payoff that effect systems advertise: one body, many interpretations, no function colouring. So: is it an effect? Short answer. Not in the algebraic-effect sense (3), but it is exactly an effect in the capability sense (4), and it captures most of the engineering benefits people normally reach for an effect system to get. Why it is not an algebraic-effect system</h3> No effect row for I/O. Unlike Koka-style effect-typed systems, Zig does not attach an I/O effect row to the return type. Two functions that do wildly different I/O have indistinguishable signatures apart from "they both take an Io</code>."</li> No effect operations plus handlers. There is no perform</code>/handle</code> mechanism that captures a delimited continuation and lets a handler decide whether, when, or how often to resume it. You select an interpretation by passing a different Io</code> value, not by handling an operation.</li> Ordinary interface dispatch. Io</code> is a value-level interface, not a statically elaborated effect operation.</li> </ul> The absence of effect rows is not by itself what disqualifies Zig - OCaml 5 has algebraic-effect handlers without effect rows, and we still call those algebraic effects. The decisive missing piece in Zig is sense (3): operations plus handlers that capture a continuation. Why it nevertheless feels effect-like</h3> It is an explicit capability threaded through call sites - the same pattern as Zig's Allocator</code>. To do I/O, you must be handed the right to do I/O.</li> It decouples description (call io.read(...)</code>) from interpretation (threaded, evented, fake). That is the same separation algebraic effects give you, achieved with an interface value instead of a handler.</li> It avoids Future[T]</code> / async fn</code> colouring: the same function body can be driven under different I/O strategies. It does not eliminate all visible plumbing - idiomatic Zig still accepts an Io</code> parameter or stores it in a context struct - but it does eliminate the language-level split into separate function categories.</li> </ul> The accurate label is capability-passing I/O: a value-level effect, in the same family as Scala's ZIO environment, OCaml's Eio</code> (deliberately designed before OCaml 5 effects landed and still capability-based), or the Reader-monad-of-IO pattern. Andrew Kelley has been explicit that Zig is not adopting an effect system; Io</code> is an interface that gives you most of the engineering benefits of one without the type-system machinery. Unpacking the technical claim</h2> The previous section said Io</code> "is just a parameter" with no propagation up the call graph. Two distinct claims, worth separating. "The function signature doesn't carry an effect row"</h3> In an effect-typed system (Koka), the type of a function lists the effects it can perform: fun read_config() : <io, exn> string</code></pre> That <io, exn></code> is the effect row - a set of effect labels attached to the return type. If read_config</code> calls something that does <net></code>, the compiler forces you to either handle net</code> inside read_config</code> or widen its row to <io, exn, net></code>. Effects compose by union and are visible in every signature. Zig's Io</code> has nothing like that. The signature is just: fn readConfig(io: Io, path: []const u8) ![]u8</code></pre> The fact that it does I/O is encoded as a value (io: Io</code>), not as a tag on the return type. A note in passing: Zig already does track one effect-shaped thing in the type - errors, via error unions (!T</code>). The example above uses ![]u8</code>. So the claim is not "Zig tracks nothing effect-like." The narrower claim is that I/O authority specifically is not tracked as a row or a capture set. "The compiler doesn't enforce that I/O-doing code propagates a marker upward"</h3> In an effect-typed system, effects are contagious through the type checker: fun a() : <io> () // does I/O fun b() : <> () = a() // ERROR — b's row is empty but it called something with <io></code></pre> The fix is forced: widen b</code>'s row, or install a handler. You cannot silently launder an effect away. In Zig, nothing propagates. If you have an Io</code> in scope (e.g. in a struct field), you can call I/O from a function whose signature mentions no Io</code> at all: const std = @import("std"); const Io = std.Io; const Logger = struct { io: Io, fn log(self: Logger, msg: []const u8) !void { // `self.io` is the I/O capability; the Writer is obtained from it. var stderr = self.io.stderr(); try stderr.writeAll(msg); // does I/O } }; fn pureLooking(l: Logger) !void { // signature gives no hint try l.log("hi"); // …yet this does I/O }</code></pre> pureLooking</code>'s signature says nothing about I/O. The compiler is fine with this. The "I can do I/O" capability rode in inside the Logger</code> value, invisibly. An effect-typed system would have rejected this until you either added the effect to pureLooking</code>'s row or handled it locally. The underlying distinction</h3> Effect row in the type → the compiler knows, statically and structurally, which effects every function may perform, and forces that knowledge to flow up the call graph.</li> Capability as a value (Zig's choice) → the compiler only knows "this function received an Io</code>-shaped value." Whether it uses it, hides it, or passes it to ten other functions is invisible at the type level. Discipline is by convention, not by the checker.</li> </ul> Zig gives you the runtime benefit of swappable interpretations without the static benefit of guaranteed effect tracking. Enter Scala 3's experimental capture checking (Caprese)</h2> Scala 3 ships an experimental capture-checking feature, enabled with import language.experimental.captureChecking</code>. The Scala documentation flags it explicitly as a research project, evolving quickly. The associated research effort is sometimes called Caprese. Both Zig's Io</code> and Scala's capture checking converge on the same philosophical move - "the ability to do X is a value, not an ambient power" - and both reject monadic I/O / function colouring. The split is whether the type system tracks where the capability flows. What they share</h3> Capabilities-as-values. Scala's IO</code>, FileSystem</code>, Async</code> and Zig's Io</code> are values you receive as parameters and pass on. In the intended capability discipline I/O authority is explicit rather than ambient; the checker tracks captured capabilities instead of requiring a monadic IO[T]</code> wrapper.</li> Decoupled interpretation. Calling code is identical regardless of which interpreter is plugged in.</li> No function colouring. A function that does I/O looks like any other function - it just takes an extra parameter (Zig) or carries a capture annotation on its arrow (Scala).</li> </ul> What capture checking adds: the static layer</h3> Capture checking encodes "which capabilities did this value or closure capture?" into the type. The current syntax puts capture annotations on the function arrow: -></code> is a pure arrow that captures nothing, ->{c, d}</code> may use capabilities c</code> and d</code>, and =></code> is treated as the impure default that may capture arbitrary capabilities. Capability values themselves carry a ^</code> marker on their type. val pure: String -> Int = s => s.length // pure arrow, captures nothing val impure: String ->{fs} Long = path => fs.size(path) // captures fs class Logger(xfs: FileSystem^): // a class that retains a capability def log(s: String): Unit = xfs.append("log", s) // inferred type carries the captured capability, e.g. Logger^{xfs}</code></pre> That gives Scala things Zig structurally cannot give you: Capability-disciplined purity. A function value with an empty capture set is guaranteed not to capture any tracked capabilities. In a capability-disciplined subset where all I/O flows through tracked capabilities, this gives you a strong purity guarantee - the docs put it conditionally: if capabilities are the only means to induce side effects, then capability polymorphism corresponds to effect polymorphism. On the JVM at large, ambient effects still exist; the guarantee is relative to the discipline. </li> Effect / capability polymorphism. You can abstract over the capabilities a function may use, with explicit capture variables. A higher-order function can be parametric over which capabilities its callback needs, and the result type tells the caller exactly what was captured. Zig has no analogue; an Io</code>-using callback just propagates the Io</code> parameter manually. </li> Scope-bounded capabilities. The type system can guarantee a capability doesn't escape the scope that granted it: Resource.use { fs => // fs : FileSystem^ is local; cannot be returned, stored in // a longer-lived ref, or captured by an escaping closure }</code></pre> This makes structured resource handling enforced, not advisory. In Zig, once you hold an Io</code>, you can stash it in any struct of any lifetime and the compiler is silent. </li> Capability retention is visible in the type. When a class holds a capability in a field, instances acquire a captured type (e.g. Logger^{xfs}</code>). Generic containers introduce nuance - capture information can tunnel through a container's projections rather than always surfacing as a simple outer capture set - but the principle is that retention is tracked. The hidden-Io</code> example from the earlier section is structurally invisible in Zig; its Scala analogue is not. </li> </ol> Two-way comparison</h3> </th> Zig Io</code></th> Scala capture checking</th></tr></thead> Capability is a value</td> yes</td> yes</td></tr> Avoids function colouring</td> yes (with plumbing)</td> yes</td></tr> Runtime-swappable interpreter</td> yes (interface dispatch)</td> yes (different instance)</td></tr> Pure functions distinguishable in type</td> no</td> yes (under the discipline)</td></tr> Capability hidden in struct is tracked</td> no</td> yes (with capture-tunneling nuance)</td></tr> Polymorphism over effects</td> no</td> yes</td></tr> Compiler prevents capability escape</td> no</td> yes</td></tr> Maturity</td> new in 0.16</td> experimental research</td></tr> Learning curve</td> trivial</td> significant</td></tr> </tbody></table> How to think about it</h3> Capture checking is what Zig's Io</code> would become if you bolted an effect-tracking type system on top of it. The runtime model is essentially the same - capabilities are plain values, dispatch is dynamic, there is no monad. Capture checking adds a bookkeeping layer in the type checker that records which capabilities each value transitively holds, and refuses programs where capabilities flow somewhere they should not. The trade-off is honest: Zig gets you most of the engineering benefit (swappable I/O, no colouring, testability) for ~0% of the language complexity. You lose static guarantees: a function's signature does not tell you what it can do.</li> Scala gets you the static guarantees at the cost of a non-trivial type-system feature that interacts with everything else (variance, inference, given/using, etc.).</li> </ul> Io</code> is the runtime substrate of a capability system without the type-level discipline. Capture checking is what you get when you keep the substrate and add the discipline back - without going all the way to algebraic effects with handlers. Enter Kyo</h2> Kyo sits in a third corner of the design space - it is the only one of the three whose public model is directly algebraic-effect-like: effects are pending in the return type, compose as a type-level set, and are discharged by handlers. What Kyo actually is</h3> Kyo encodes effects in the return type using a pending-effects operator <</code>. Its current core effects include Sync</code> (side effects), Async</code>, Scope</code> (resource lifecycle), and Abort[E]</code> (typed failure). The README describes the design as based on algebraic effects and modular handlers, with pending effects represented as unordered type-level sets via intersection. def readConfig(path: String): String < (Sync & Abort[ConfigError]) = ... def parse(s: String): Config < Abort[ConfigError] = ... val program: Config < (Sync & Abort[ConfigError]) = readConfig("conf.json").map(parse) // Handlers discharge effects, peeling them off the pending set: val safe: Result[ConfigError, Config] < Sync = Abort.run(program) // The remaining Sync is normally discharged at the application boundary // (e.g. by extending KyoApp), not by ad-hoc unsafe runs.</code></pre> Three things to notice: Effects appear in the type, not in the parameter list. For ordinary effects such as Sync</code> or Abort</code>, readConfig</code> does not take an io: IO</code> parameter; the requirement is encoded in the pending-effects type. (Kyo can still model environments and services, but those are themselves represented as effects rather than as manually threaded interface values.)</li> Effects compose by intersection in the type (Sync & Abort[E]</code>), not by passing more parameters.</li> Handlers discharge effects. Abort.run</code>, Async.run</code>, and the KyoApp</code> boundary that runs Sync</code>, peel an effect off the pending set, transforming the type. This is the algebraic-effect handler shape - code is interpretation-free until a handler runs it.</li> </ol> Under the hood, Kyo is a single CPS-trampoline encoding (not transformer stacks), so it gets direct-style-ish performance with a uniform composition story. It is a Scala library encoding of algebraic effects, not a language feature like OCaml 5 or Koka - worth keeping in mind when comparing it to those. Three-way table</h3> </th> Zig Io</code></th> Scala capture checking</th> Scala Kyo</th></tr></thead> Where the effect lives</td> parameter value</td> capture set on the type</td> pending set on the return type</td></tr> Effect appears in signature</td> no</td> yes (->{io,fs}</code>)</td> yes (< (Sync & Fs)</code>)</td></tr> Pure code distinguishable</td> no</td> yes (under the discipline)</td> yes (A</code> vs A < S</code>)</td></tr> Effect / capability polymorphism</td> no</td> yes</td> yes (poly over S</code>)</td></tr> Handler-based discharge</td> no</td> no</td> yes (Abort.run</code>, Async.run</code>, KyoApp</code>, …)</td></tr> User-defined non-determinism, backtracking, custom scheduling</td> no</td> no</td> yes (handler controls the continuation)</td></tr> Composition mechanism</td> n/a</td> union of capture sets</td> intersection of effect types</td></tr> Resource scoping enforced by checker</td> no</td> yes (escape-checked)</td> partially, via Scope</code> and disciplined handlers/runtime boundaries</td></tr> Runtime model</td> interface dispatch</td> plain values</td> CPS trampoline</td></tr> Avoids function colouring</td> yes (with plumbing)</td> yes</td> yes-ish (the colour moves into the type)</td></tr> Learning curve</td> trivial</td> significant</td> significant (different shape from capture checking)</td></tr> </tbody></table> The three philosophies in one line each</h3> Zig Io</code> - "the ability is a value; the type system stays out of it."</li> Capture checking - "the ability is a value; the type system tracks where the value flows."</li> Kyo - "the ability is a type; handlers interpret it at the boundary."</li> </ul> Where Kyo gives you something the other two structurally cannot</h3> Both Zig and capture checking let you swap implementations - pass a different Io</code>, pass a different FileSystem</code>. That is enough for "blocking vs. async vs. fake for tests." But neither lets the handler control the control flow of the effectful code. Kyo does, because handlers receive a continuation: Non-determinism: a Choice</code> handler that runs the rest of the computation once per branch and collects results. You cannot express this by swapping a vtable - it requires capturing what comes after the effect.</li> Backtracking / search: handler peeks at intermediate results and decides whether to resume, restart, or abandon.</li> Custom schedulers: an Async</code> handler that decides ordering, fairness, cancellation policy entirely in user code.</li> Effect translation: a handler that converts Abort[E1]</code> into Abort[E2]</code>, or interprets Sync</code> as Trace</code> (logging every operation without performing it) for testing.</li> </ul> Capture checking cannot do this directly because capabilities are just values - they do not reify the continuation of the caller. Zig cannot do this because there is no notion of a handler at all; you only pick the implementation, not the control structure. Where capture checking gives you something Kyo doesn't</h3> Capture checking still wins on one axis: enforced lexical scoping of capabilities. It can prove a FileSystem</code> doesn't escape Resource.use { fs => ... }</code> because the type system refuses to let fs</code> flow into a longer-lived type. Kyo can model this with Scope</code>, but the guarantee comes from the handler/runtime discipline, not from a structural impossibility - a poorly written handler could leak a resource. (In practice Kyo's standard handlers are correct, but the type system is not what is stopping you.) Conversely, capture checking can't express "run this block 17 times under a different interpretation each time," which Kyo gets for free. Synthesis: a spectrum, not a hierarchy</h2> Mapping back to the original question — is Zig's Io</code> an effect? — the honest answer is that "effect" is a continuum, and these three systems sit at distinct points along it: Zig Io ──► capture checking ──► Kyo ──► Koka / Eff and OCaml 5 (none) (capture sets) (lib AE) (language AE; Koka/Eff are effect-typed, OCaml 5 has handlers without static effect safety and only one-shot continuations) │ │ │ │ │ │ │ └─ Effects + handlers as │ │ │ a first-class language │ │ │ feature. │ │ │ │ │ └─ Effects in types, handlers in user │ │ code, implemented as a library on a │ │ CPS runtime. │ │ │ └─ Capabilities as values, with type-system tracking of │ where capabilities flow. No handlers — interpretation │ is chosen by which capability value you pass. │ └─ Capabilities as values. No type tracking, no handlers. Engineering benefits of effects without the formal machinery.</code></pre> Each step rightward buys more guarantees and more expressive control flow at the cost of more language or library complexity, more annotations, and (often) more runtime indirection. A final philosophical reading</h3> There is a useful way to see the progression: Zig Io</code> treats effects as a runtime engineering problem: "we need the ability to swap implementations and avoid colouring functions." It solves exactly that and refuses to pay for anything more. </li> Capture checking treats effects as a flow-of-authority problem: "if a piece of code holds the ability to do X, the type system should know, and should be able to prevent that ability from leaking past where it was granted." It keeps the value-level model and adds a static accounting layer on top. </li> Kyo (and Koka, OCaml 5) treats effects as a control-flow problem: "operations are syntactic markers whose meaning is supplied by the surrounding handler, including how - or whether - to resume." Effects stop being about who has permission and become about who decides what happens next. </li> </ol> Zig's Io</code> is, by this lens, the minimal viable point in the design space: enough effect-shape to fix function colouring and enable testing, no more. Whether that is "really" an effect depends on which sense of "effect" from §0 you have in mind - and on whether you are asking a language theorist or a working systems programmer. The theorist will say no in sense (3); the systems programmer will note that they got most of the benefit they cared about, with none of the syntactic or runtime cost - and that, for a language whose target audience is largely the second group, is probably the correct trade. Appendix: status as of writing</h2> Zig std.Io</code> is new in 0.16.0 (April 2026). The threaded backend is feature-complete; evented, io_uring</code>, kqueue</code>, and Grand-Central-Dispatch backends range from work-in-progress to proof-of-concept. std.testing.io</code> is the recommended fake for tests.</li> Scala 3 capture checking is enabled via import language.experimental.captureChecking</code> and is documented as a research project that is evolving quickly. Function-arrow syntax (-></code>, ->{c}</code>, =></code>) and capture markers (T^</code>, T^{c}</code>) reflect the current shape; details may shift.</li> Kyo's core effects today are Sync</code>, Async</code>, Scope</code>, and Abort</code> - not the older IO</code> / Resource</code> terminology. The README treats KyoApp</code> as the standard application boundary that discharges Sync</code>.</li> </ul> Evaluating PBT Frameworks: How Proptest and Hegel Differ in Algebraic Expressivity Unknown — Mon, 04 May 2026 00:00:00 +0000 Evaluating PBT Frameworks: How Proptest and Hegel Differ in Algebraic Expressivity</h1> Context</h2> I have recently been working on porcupine-rust</code></a> is a Rust port of Porcupine</a>, a linearizability checker for concurrent and distributed systems, with APIs over timestamped Operation</code> histories and raw Event</code> histories, optional timeout-bounded checking, P-compositional partitioning, and support for nondeterministic step semantics through NondeterministicModel</code> and PowerSetModel</code>. In the same codebase, the runtime invariant layer explicitly ties implementation checks back to INV-</code> identifiers in docs/spec.md</code>, including well-formed history, minimal-call frontier, partition independence, and cache soundness. That mix of algebraic laws, history-shape constraints, and incremental trace reasoning is exactly the kind of workload that exposes the real design trade-offs between testing libraries. The project’s testing setup from the inside reveals two property-based testing implementations - tests/property_tests.rs</code> (using proptest</a>) and tests/hegel_properties.rs</code> (using hegel</a>). They are deliberately near-mirror suites over the same invariants, with shared builders under tests/common/</code>, while also documenting the handful of places where the Hegel suite genuinely goes beyond the proptest one. That makes this repository a particularly good benchmark for “same domain, same models, same assertions, different property-testing architecture”. Why porcupine-rust makes this comparison unusually revealing</h2> The repository’s core problem is linearizability</a>: given a partially ordered history of calls and returns, the checker has to decide whether those operations can be linearized into some sequential history consistent with the model. The README makes clear that the implementation is, in essence, a search engine over histories, with DFS/backtracking, bitset-based state tracking, partition-aware checking, and optional nondeterministic model support. Those characteristics matter for testing because failures are rarely “one bad scalar”, they are usually tangled histories with subtle timing relationships, partition boundaries, or branching semantics. That is also why the invariant layer matters so much. In src/invariants.rs</code>, the code documents that every macro or function corresponds to an INV-</code> identifier in docs/spec.md</code>, and the partition-related checks nail down strong contracts: partitions must be disjoint, complete, in-bounds, and for event histories, keep each (Call, Return)</code> pair together. Those are higher-order obligations over histories, not merely per-field sanity checks. A property-testing library used here therefore has to be good at constrained structured generation, shrinking of closely-related values, and, in the stateful case, construction of histories that remain meaningful as they are simplified. There is a subtle but crucial caveat though: neither test suite is executing live concurrent Rust code with threads or async runtimes. Both suites are checking linearizability properties over pre-built histories. So the argument is not “which library is better at race testing by itself”; it is “which library better expresses the generation and shrinking of concurrent-looking histories and prefixes”. That framing is important, because it is exactly where Hegel’s state-machine API starts to matter and where proptest’s batch-generated histories remain perfectly adequate for a large class of checks. Proptest and Hegel in one paragraph each</h2> Proptest is the established Rust-native member of the QuickCheck family. Its central abstraction is the Strategy</code>, which defines both how values are generated and how they are shrunk. The Proptest book describes generation and shrinking as the two defining duties of a strategy, and the project describes the crate as feature-complete enough to be in long-term passive maintenance. In practical Rust terms, Proptest’s worldview is: build a typed generator graph, compose it with strategy combinators such as prop_map</code>, prop_flat_map</code>, prop_filter</code>, and prop_compose!</code>, and then feed the resulting values into a test body. Hegel, by contrast, is a Rust property-testing library built on Hypothesis through the Hegel protocol</a>. Its centre of gravity is the live TestCase</code>: you draw values imperatively with tc.draw(...)</code>, reject inputs with tc.assume(...)</code>, attach debugging information with tc.note(...)</code>, and build dependent generators with #[hegel::composite]</code>. The official documentation also makes stateful testing a first-class module, with #[hegel::state_machine]</code>, #[rule]</code>, #[invariant]</code>, stateful::run</code>, and Variables<T></code> for pools of previously generated values. A concise way to describe the difference is this: Proptest localises randomness in strategy values, while Hegel keeps randomness live inside the test execution itself. That is not a matter of syntax alone. It changes whether generation is a prelude to the test or part of the test algorithm. In a repository like porcupine-rust</code>, that architectural distinction ends up being more important than slogans like “stateless versus stateful”. Where both libraries are equally strong</h2> The strongest point in favour of keeping both libraries is that a large part of the repository does not actually force a choice. The project's strategy for enforcing invariants across spec and implementation suggests that for every INV-</code> invariant in docs/spec.md</code>, the two suites contain near-mirror tests using the same shared builders, and it names several such pairs explicitly: prop_time_shift_invariance</code> versus hegel_time_shift_invariance</code>, prop_slice_order_invariance</code> versus hegel_slice_order_invariance</code>, prop_concurrent_write_overlap_read_matches_membership</code> versus its Hegel counterpart, prop_powerset_eq_hashed_powerset</code> versus hegel_powerset_eq_hashed_powerset</code>, and cache-soundness checks on both sides. In other words, for a large class of finished-history properties, the repository itself already demonstrates that the two frameworks are expressively interchangeable. The best concrete example is dependent generation. Let us walk through arb_overlap_write_read</code> in the proptest suite and its Hegel mirror in tests/hegel_properties.rs</code>. In proptest, the pattern is the familiar multi-stage prop_compose!</code> form: first draw write_dur</code>, then use Just(write_dur)</code> to make that value available while drawing read_call</code> from 0..write_dur</code>. That two-stage form is essentially sugar for prop_flat_map</code>: the first stage produces a value, and the second stage builds a new strategy parameterized by it. Generation is still a function of strategies, evaluated before the test body runs. In Hegel, the same constraint is written directly with sequential draws: first write_dur</code>, then read_call</code> with its upper bound computed from that earlier draw. Both are principled encodings of a dependent generator - neither is resorting to rejection sampling. Rejection Sampling: A naive way to enforce read_call < write_dur</code> would be to draw both freely and then throw away cases that violate the bound (rejection sampling), which wastes work and confuses shrinking. Both libraries avoid that by making the dependency structural. The difference is purely ergonomic: proptest expresses it through strategy combinators, Hegel through ordinary control flow.</h2> A distilled version of that contrast looks like this: // proptest style: dependency staged through the strategy. prop_compose! { fn overlap_history() (write_dur in 5u64..30, write_value in -50i64..50) (write_dur in Just(write_dur), write_value in Just(write_value), read_call in 0u64..write_dur) -> History { // build overlapping write/read history } }</code></pre>// Hegel style: dependency expressed inline at the draw site. #[hegel::composite] fn overlap_history(tc: TestCase) -> History { let write_dur = tc.draw(gs::integers::<u64>().min_value(5).max_value(30)); let write_value = tc.draw(gs::integers::<i64>().min_value(-50).max_value(50)); let read_call = tc.draw(gs::integers::<u64>().min_value(0).max_value(write_dur - 1)); // build overlapping write/read history }</code></pre> Those snippets differ in ergonomics, not in logical power for this category of problem. both cleanly encode read_call ∈ [0, write_dur)</code> without rejection sampling - but proptest forces the dependency into the shape of the strategy (two parameter lists, Just re-binding), while Hegel lets it look like ordinary imperative code. The proptest book explicitly documents prop_compose!</code> and prop_flat_map</code> as the mechanism for value-dependent strategies, and the Hegel docs explicitly document feeding the result of one draw</code> into later draw</code> calls in #[hegel::composite]</code> generators. This equivalence matters because it prevents an overstatement that would otherwise be tempting: Hegel is not “more expressive” in the repository’s dominant workload of stateless or value-level history generation. For overlap windows, shaped reads, algebraic invariants, powerset/hash equivalences, and deterministic cache checks, proptest’s strategy algebra and Hegel’s imperative draws are simply two different surfaces over the same basic generator logic. It would be right to say that, in this category, nothing in the repository is inherently beyond one framework or the other. Where Hegel is materially stronger in this repository</h2> The expressive gap opens only when the history is not generated all at once, but grown step by step while the property is being checked. The test suite identifies three Hegel-only tests in porcupine-rust</code>: IncrementalKv</code>, ConcurrentWritesChain</code>, and IncrementalRegister</code>. In IncrementalKv</code>, random writer and reader rules are selected repeatedly and the checker is asserted to return Ok</code> on every prefix of the accumulated history. In ConcurrentWritesChain</code>, the next operation’s legal call-time range is computed from state accumulated by previous rule firings. In IncrementalRegister</code>, a newly generated read derives its expected output from a backward scan over prior history before the operation is appended. Those three tests are not just “stateful”; they interleave random generation, model-state inspection, mutation, and assertion in one loop. Hegel’s stateful API is built exactly for that shape. The official stateful docs say that #[rule]</code> methods mutate &mut self</code>, #[invariant]</code> methods are checked after each successful rule application, and stateful::run</code> repeatedly applies random rules. The StateMachine</code> trait itself is defined in terms of rules()</code> and invariants()</code>, and Variables<T></code> adds a symbolic pool of previously generated values when a state machine needs to carry handles or references forward across rule applications. In other words, the official abstraction and the repository’s most sophisticated tests line up almost perfectly. A representative Hegel pattern for this repository looks like the following: #[hegel::state_machine] impl IncrementalKv { #[rule] fn write_random_key(&mut self, tc: TestCase) { // draw key/value/timestamps, append operation } #[rule] fn read_random_key(&mut self, tc: TestCase) { // draw key/timestamps, compute expected read from current history, append } #[invariant] fn prefixes_are_linearizable(&self, _: TestCase) { assert_eq!(check_operations(&KvModel, &self.history, None), CheckResult::Ok); } }</code></pre> This is not just pleasantly ergonomic. It preserves the semantic unity between “how the next move is drawn”, “how state evolves”, and “what must hold now”. That unity is exactly what makes Hegel support stateful, rule-scheduled, prefix-asserting traces with mid-test adaptive draws. Proptest can approximate parts of that story, but it does so by moving complexity into the strategy layer. The proptest state-machine chapter requires a ReferenceStateMachine</code> with init_state</code>, transitions(state)</code>, and apply</code>, plus a StateMachineTest</code> with separate init_test</code>, apply</code>, and check_invariants</code>. The same chapter explains that the sequential strategy ultimately generates a Vec<Transition></code> that is then interpreted against the SUT. That is a perfectly respectable design, and it has real strengths, but it is a different shape from the Hegel rule body that can keep calling tc.draw(...)</code> after inspecting current machine state. The subtle gain Hegel gets here is not merely “stateful testing exists”. Proptest also has state-machine testing. The gain is that Hegel allows generation to remain imperative and state-sensitive inside the rule. The ConcurrentWritesChain</code> example in the tests is the clearest illustration: the next call time is drawn from bounds computed from self.last_call</code> and self.last_return</code>, and that choice sits in the same body as the subsequent assertion over the whole history. In proptest-state-machine, the same logic would have to be split across transitions(state)</code> for generation, apply</code> for mutation and local postconditions, and check_invariants</code> for global assertions. That is doable; it is just materially more fragmented. There is also a shrinker story behind this. The incremental, prefix-sensitive failures tend to benefit from Hypothesis-style shrinking because the counterexample is simplified at the same granularity as rule firings rather than only as a final batch-produced move list. I would phrase that carefully: not “Hegel always shrinks better”, but “for the failure shape produced by incrementally constructed histories, Hegel’s model fits the bug more closely”. In a linearizability checker, that can be the difference between a five-step witness history and a forty-operation hairball. Where Proptest is still the better tool</h2> The first proptest advantage is operational, not theoretical: speed. A benchmark done reports a project-specific measurement at 100 cases per test of roughly 0.05 s</code> for cargo test --test property_tests</code> versus roughly 2.2 s</code> for cargo test --test hegel_properties</code>. That gap is entirely plausible given the official Hegel installation model: hegeltest</code> uses uv</code>, may install or download its server component, and Hegel’s own maintainers explicitly say that the current Python dependency is the main performance limiter. If you care about fast inner-loop feedback while editing the checker, proptest is the obvious default. The second advantage is ecosystem maturity. Proptest describes itself as close to feature-complete and largely in passive maintenance, which is usually exactly what infrastructure code wants from a test dependency: low surprise, stable idioms, and broad community familiarity. Hegel’s crate documentation, by contrast, carries an explicit beta disclaimer and reserves the right to make breaking changes if that improves the library. That is not a criticism of Hegel’s design; it is a concrete engineering cost. For CI-critical test suites in production Rust repositories, this stability differential matters. The third advantage is that proptest’s “generator first” model is often the right abstraction when the generated object is the thing you want to persist, replay, or compose. Its failure-persistence mechanism stores failing cases in proptest-regressions</code> so later runs replay them before searching for fresh failures, and its state-machine API is explicit about the split between the abstract reference state and the SUT. This split points to a genuine strength when there is a real effectful system under test—say a database, a service process, or an external component—because it forces the model/implementation distinction into the type shape of the harness. The final Proptest advantage is that its combinator algebra rewards careful modelling. The official docs show that prop_map</code> preserves the relationship to the original strategy so shrinking still happens in terms of the underlying source values, and prop_flat_map</code> allows a later strategy to depend on an earlier drawn value while keeping invariants such as “the index stays within the vector”. In other words, Proptest is not weak at dependency management; it just insists that you make those dependencies explicit in the strategy graph. For teams that value typed, reusable, independently testable generators, that discipline is often a feature rather than a burden. Which design patterns belong to which library</h2> Choose Hegel when the trace is the test. If your property grows a history incrementally, checks every prefix, and must let later draws depend on already-mutated model state, Hegel is the more natural fit in Rust today. porcupine-rust</code>’s IncrementalKv</code>, ConcurrentWritesChain</code>, and IncrementalRegister</code> are exemplary here: the next operation is not merely sampleable from a static strategy; it is computed in dialogue with the current history. That design pattern maps directly onto #[hegel::state_machine]</code>, #[rule]</code>, #[invariant]</code>, tc.draw</code>, and tc.assume</code>. </li> Choose Hegel as well when you want imperative generator code that still shrinks coherently. The official #[hegel::composite]</code> docs explicitly bless feeding one draw into subsequent draws, and the this is exactly how the Hegel versions of the porcupine history generators are expressed. In Rust terms, Hegel feels most at home when your generator wants to read like straight-line business logic with branches, helper calls, local calculations, and late assumptions. </li> Choose proptest when the input object is a value, not an interaction. That includes algebraic laws over finished histories, finished operation vectors, transformations over timestamps, partitioning rules, and cross-checks over pure model functions. It also includes cases where you want generators to be exported as reusable functions, composed through prop_map</code>, prop_flat_map</code>, prop_oneof!</code>, and prop_compose!</code>, and reused across many tests. That pattern is heavily represented in porcupine-rust</code>’s mirrored value-level properties, and the repository’s own comparison note says the two frameworks are effectively interchangeable there. </li> Choose proptest when execution economics or harness architecture matter more than the last ounce of stateful elegance. If you want the fastest edit-run loop, a pure-Rust dependency with no sidecar, stored regression seeds, and a state-machine API that explicitly separates model from SUT, Proptest is the better day-to-day engineering choice. That is why it's recommended for inner-loop CI work even while crediting Hegel for stronger stateful coverage. </li> </ul> The most useful general rule I can extract from porcupine-rust</code> is this: proptest is strongest when generation is a specification artefact; Hegel is strongest when generation is part of the execution semantics of the property itself. That is why both libraries look equally capable on overlap-window generators, yet Hegel pulls ahead once the history is allowed to evolve under a live scheduler with per-prefix assertions. It is also why proptest remains the saner choice for fast, stable, repository-wide coverage over law-like properties. The right conclusion, then, is not that one library wins. The right conclusion is that porcupine-rust</code> exposes two different testing geometries. For finished-history algebra, typed generator combinators, and cheap CI repetition, proptest is superb. For incremental history construction, stateful linearizability prefixes, and rule bodies whose random choices must remain entangled with current model state, Hegel is the more expressive instrument. In a repository that checks linearizability, partition independence, and nondeterministic model behaviour, appreciating the power of both is not compromise; it is architectural honesty. The RAII Drop-Guard Pattern in Rust Unknown — Thu, 30 Apr 2026 00:00:00 +0000 The RAII Drop-Guard Pattern in Rust</h1> Context</h2> In Rust, any value that owns a resource and frees it in its Drop</code> implementation acts as a guard: the resource lives for exactly as long as the value is in scope. This is RAII - Resource Acquisition Is Initialization - and in Rust it is the primary, statically-enforced mechanism for resource management. A drop guard is the pattern of binding such a value to a name whose sole purpose is to keep it in scope. It looks like you're creating an unused variable. You're not - you're creating a lexical region in which a resource is held. The compiler uses the variable's scope to schedule the exact point at which the resource is released. This doc explains the mechanics, why the pattern exists, the benefits it offers, and three concrete examples from real reviews where readers misread the pattern on first contact. 1. The Rust mechanics</h2> There are three syntactic forms people confuse. They behave differently. 1.1 Three bindings that look similar</h3> // Form A: bind to a proper name let permit = sem.acquire().await?; // Form B: bind to a name starting with underscore let _permit = sem.acquire().await?; // Form C: destructure with a bare underscore pattern let _ = sem.acquire().await?;</code></pre> Semantically: Form</th> Binding</th> Drop runs at</th></tr></thead> A - permit</code></td> yes</td> end of enclosing scope (}</code>)</td></tr> B - _permit</code></td> yes</td> end of enclosing scope (}</code>)</td></tr> C - _</code></td> no binding</td> immediately on this statement</td></tr> </tbody></table> Forms A and B are identical in terms of when Drop runs. They differ only in how the compiler treats the name for lint purposes: permit</code> - a used-looking name. If you never read it, unused_variables</code> warns.</li> _permit</code> - a name starting with _</code> suppresses unused_variables</code> but is still a real binding. The value is stored; it lives until the scope ends.</li> _</code> (bare) - a pattern, not a name. The value is matched, then immediately dropped. No storage, no name, no lifetime extension.</li> </ul> This distinction is critical for RAII. If you write let _ = sem.acquire().await?;</code>, you've released the permit on that very line - you held the slot for a nanosecond. If you write let _permit = sem.acquire().await?;</code>, you hold the slot until the closing brace. 1.2 When does Drop actually run?</h3> Scope exit, in LIFO order of creation. Temporaries within an expression drop at the end of the enclosing statement; let</code>-bound values drop at the end of their block. fn work() { let a = Guard::new("a"); // acquired let b = Guard::new("b"); // acquired do_stuff(); // runs with both guards held // implicit: b dropped, then a dropped }</code></pre>fn work() { { let _inner = Guard::new("inner"); do_inner(); } // _inner dropped here do_outer(); // runs with no guards held }</code></pre> The compiler inserts the drops at the }</code>. You cannot observe them in the source, but they are deterministic and guaranteed barring abort/panic-during-drop pathologies. 1.3 Explicit release</h3> If you need to release a guard earlier than end-of-scope, call drop(x)</code>: let mut guard = mutex.lock().unwrap(); modify_protected_state(&mut guard); drop(guard); // release before the function returns expensive_computation_that_doesnt_need_the_lock();</code></pre> Or nest a block: let result = { let _guard = mutex.lock().unwrap(); read_protected_state() }; // _guard dropped here use(result);</code></pre> Both idioms are explicit and legitimate. What you cannot do is "forget" to drop - the compiler will drop at scope exit whether you want it to or not, which is the whole point. 2. What makes a type a "guard"</h2> Any type with a Drop</code> impl that releases a resource is a guard. The Rust ecosystem is full of them: Type</th> Resource it guards</th> Who releases on drop</th></tr></thead> std::sync::MutexGuard<'_, T></code></td> an acquired Mutex</code> lock</td> releases the lock</td></tr> tokio::sync::SemaphorePermit<'_></code></td> a semaphore slot</td> releases one permit</td></tr> tempfile::TempDir</code></td> an on-disk temp directory</td> deletes the directory</td></tr> std::fs::File</code></td> an OS file descriptor</td> closes the fd</td></tr> Box<T></code></td> heap allocation</td> deallocates</td></tr> Vec<T></code></td> heap buffer + len</td> deallocates buffer, drops elements</td></tr> A custom struct</code> with manual impl Drop</code></td> whatever you coded</td> whatever you coded</td></tr> </tbody></table> Because the resource lifetime is tied to a value's scope, you get deterministic, leak-safe, exception-safe cleanup by construction - without try/finally</code>, without defer</code>, without a garbage collector. 3. Three flavors of the pattern in practice</h2> These are three examples that came up in real code review comments on a single PR. Each uses the same underlying Rust mechanic; each looked surprising or wrong to a reviewer encountering it in a context they hadn't previously mapped to RAII. 3.1 Semaphore permits as concurrency gates</h3> join_set.spawn(async move { let _global_permit = match global_sem.acquire().await { Ok(p) => p, Err(_) => return (rid, Err(sem_closed_error())), }; let _local_permit = match local_sem.acquire().await { Ok(p) => p, Err(_) => return (rid, Err(sem_closed_error())), }; let result = do_work().await; (rid, result) // _local_permit dropped here → local slot released // _global_permit dropped here → global slot released });</code></pre> Two caps - a global one enforced across all spawned groups, a local one per group - are implemented as two semaphores. Each task must hold one permit from each to proceed. The permits are bound to _global_permit</code> and _local_permit</code> so they survive until the task finishes, then release on scope exit. A reviewer on this PR called the pattern "clever." What they likely noticed was that two values appear to be "unused" - nothing reads _global_permit</code> or _local_permit</code> - yet the code depends critically on their existence. The existence of the value is the work; the value itself is never consumed. That is the distinctive shape of an RAII guard. Two things are worth noting beyond the scope-trick itself: Acquisition order is consistent: every task acquires global before local. That is deliberate - it's the standard anti-deadlock discipline for nested locks. Reverse ordering in some paths (acquire local before global on the error branch, say) would let two tasks each hold one and wait on the other.</li> Release order is LIFO: _local_permit</code> drops before _global_permit</code>, matching acquisition. No hand-coded drop()</code> calls are needed.</li> </ul> 3.2 TempDir</code> as a filesystem lifetime anchor</h3> fn create_test_workspace() -> (tempfile::TempDir, Workspace) { let tmp = tempfile::TempDir::new().unwrap(); let ws = Workspace::create_at(tmp.path(), ExecutionStrategy::Sequential) .expect("Failed to create test workspace"); (tmp, ws) } #[test] fn test_something() { let (_tempdir, mut workspace) = create_test_workspace(); workspace.set_session("/github", SessionId::new("…".into())).unwrap(); workspace.save().unwrap(); // ...assertions... // _tempdir dropped here → directory deleted }</code></pre> tempfile::TempDir</code> guards a temporary directory on disk. It deletes the directory when it drops. TempDir::path</code> returns a &Path</code> borrow; the workspace likely converts it to an owned PathBuf</code> internally for storage, but it does not own the TempDir</code> itself - so if the TempDir</code> drops mid-test, subsequent workspace.save()</code> will fail because the backing directory is gone. So the helper returns both - the Workspace</code> you operate on and the TempDir</code> whose scope must outlive it. The caller binds the TempDir</code> to a name. Any name works, but _tempdir</code> is conventional because: It communicates "this is a lifetime anchor, not something you'll read."</li> It silences the unused-variable lint without the bare _</code> that would drop it immediately.</li> </ul> A reviewer on this PR objected that "the first element of the tuple isn't being used." The literal claim is a misread - _tempdir</code> is used, via its Drop</code> side-effect, which is the point of the binding. But the misread is informative: the pattern's central move (a binding whose value is never read) looks like dead code to readers not tuned to RAII. The cheapest mitigation is naming: _tempdir</code> reads intent better than _tmp</code>. 3.3 MutexGuard</code> for scoped critical sections</h3> static STATE: Mutex<Counters> = Mutex::new(Counters::new()); fn observe_and_reset() -> Counters { let mut guard = STATE.lock().unwrap(); let snapshot = guard.clone(); guard = Counters::new(); snapshot // guard dropped here → mutex released }</code></pre> MutexGuard</code> is probably the most textbook RAII guard in Rust. Acquired by lock()</code>, released on drop, and Deref</code>s to the protected value. The snapshot</code> returned is derived from the protected state under the lock; the guard releases immediately after the expression snapshot</code> is evaluated, at the end of the function. If you want to shrink the critical section, nest a block: fn observe_and_reset() -> Counters { let snapshot = { let mut guard = STATE.lock().unwrap(); let s = guard.clone(); *guard = Counters::new(); s }; // guard dropped here expensive_post_processing(&snapshot); snapshot }</code></pre> Same pattern. The guard anchors the critical section to a lexical region. 4. Why Rust uses this pattern</h2> Rust deliberately doesn't have try/finally</code>, defer</code>, or runtime-managed cleanup. RAII via Drop</code> is the uniform mechanism, and that is a design decision worth understanding. 4.1 Static, deterministic cleanup</h3> The compiler knows when every value's scope ends. Drop code is inserted at those points. There is no runtime scheduling, no "when the GC feels like it," no race between cleanup threads. The same source produces the same drop ordering on every run. This means: Cleanup cost is accounted for at compile time - you know where the drop()</code> calls are even if they're implicit.</li> Drop order is observable and portable - tests that exercise cleanup side effects are reproducible.</li> </ul> 4.2 Exception-safety is free</h3> If a function panics mid-way, stack unwinding drops every value in every stack frame being unwound, in reverse order of creation. A MutexGuard</code> held across the panic point releases the lock during unwind. A TempDir</code> deletes itself. A File</code> closes. Contrast with C++ (where you get RAII in destructors but must combine it with careful exception types) or Go (where a forgotten defer</code> leaks silently). There is no path through a Rust function, however exceptional, that skips Drop. The exceptions are all explicit and named: std::process::abort()</code> - terminates without unwinding.</li> std::mem::forget()</code> and ManuallyDrop</code> - opt out of running Drop</code> for a specific value.</li> Box::leak</code> / Vec::leak</code> / String::leak</code> - intentionally turn an owned value into a 'static</code> reference, leaking the allocation.</li> A Drop</code> impl that itself panics during unwinding - aborts the process.</li> Reference cycles via Rc</code> / Arc</code> - if two Rc</code>s reference each other, neither's strong count ever reaches zero, so neither value is dropped. This is memory-safe (no UB, as mem::forget</code> is also memory-safe) but it is the most common "my Drop</code> never ran" footgun in real code. The standard remedy is Weak</code> for the back-edge.</li> </ul> 4.3 Coupling lifetime to semantic scope, not to control flow</h3> The shape of the scope - function body, block, match arm - becomes the shape of the resource's lifetime. You don't have to hand-thread release calls through every early return, every error branch, every loop break. The compiler does it. That removes a whole class of bugs: "forgot to unlock on the error path" - impossible; the guard drops on unwind.</li> "released the permit twice" - can't; you can only drop an owned value once.</li> "used after release" - caught at compile time; the borrow checker rejects uses after drop(guard)</code> or after scope exit.</li> "resource leaked because of an early return" - the return point runs the implicit drops.</li> </ul> 4.4 Composable</h3> A struct that owns a guard inherits the guard's behavior. If MyService</code> holds a MutexGuard<'_, State></code>, dropping MyService</code> releases the lock. You build larger transactional units by composition, and Rust's drop ordering gives you deterministic control - but note that the rule for struct fields differs from the rule for local let</code> bindings: Locals in a block drop in reverse declaration order (LIFO), as shown in §1.2.</li> Struct fields drop in declaration order (top to bottom).</li> Tuple and array elements likewise drop in declaration / index order.</li> </ul> So to release the lock last (after the audit log is finalized and pending entries are released), the lock field must be declared last: struct Transaction<'a> { pending: Vec<Entry>, _log: AuditLog, _lock: MutexGuard<'a, Bank>, } // Fields drop in declaration order: // 1. pending drops - entries released // 2. _log drops - audit entry finalized // 3. _lock drops - bank unlocked (released last, as intended) // To change the order, reorder the fields.</code></pre> If a struct itself has a manual impl Drop</code>, the compiler runs that drop</code> body first, then drops the fields in declaration order. 4.5 Tooling recognizes the pattern</h3> Clippy and rustc have lints specifically for drop-guard misuse: #[must_use]</code> on MutexGuard</code> / SemaphorePermit</code> / TempDir</code> makes rustc warn when the value is dropped as a bare expression statement (e.g., mutex.lock();</code> with no binding at all). Note: it does not fire for let _ = mutex.lock();</code> - the underscore pattern counts as a use.</li> let_underscore_lock</code> (clippy) covers exactly that gap - it warns on let _ = mutex.lock();</code> because the bare _</code> pattern drops the guard immediately, almost always a bug.</li> let_underscore_must_use</code> (clippy) is the more general form: warns when let _</code> discards any #[must_use]</code> value.</li> used_underscore_binding</code> (clippy) warns if a _name</code> binding is actually read (reasonably - _name</code> signals intent-not-to-read).</li> </ul> The ecosystem is aware of the pattern and the failure modes adjacent to it. 5. Benefits, restated as a checklist</h2> When you reach for an RAII guard over an explicit acquire/release pair, you get: Cleanup on every path, including panics. No try/finally</code>.</li> One acquisition site, one name, one scope. No "did I call release on every branch?"</li> Static drop-order guarantees. No runtime reordering; no surprises.</li> Composable resource ownership. Guards nest into structs, get returned from functions, and still respect their scope. (Caveat for async: a std::sync::MutexGuard</code> is !Send</code> and cannot be held across an .await</code> in a Send</code> future. Even when the type system permits it, holding a sync mutex across .await</code> is almost always a bug - it can stall the runtime. Clippy's await_holding_lock</code> flags this. Use tokio::sync::Mutex</code> if a guard genuinely needs to live across an await point.)</li> Lint-enforced correctness. The compiler and clippy catch the common mistakes.</li> Self-documenting lifetime. The region where the resource is held is visible as a block or a function body - no reader has to trace control flow to find the release.</li> </ol> The cost, compared to a "just call release()</code> manually" approach, is essentially zero at runtime and slightly more vocabulary at read time: you must recognize that _guard</code> is a real binding, that Drop runs at the }</code>, and that a bare _</code> is different from _name</code>. Once internalized, the pattern fades into background idiom. 6. Common misreadings</h2> These are the shapes that trip readers who haven't fully absorbed the pattern. Each has a cheap remedy. Misreading 1: "This variable is unused."</h3> let _permit = sem.acquire().await?;</code></pre> The name starts with _</code>, there is no read of _permit</code> anywhere, the linter is silent. To a reader used to scanning for reads, the binding looks dead. They may suggest removing it or changing it to let _ = sem.acquire().await?;</code>. Both changes break the program - the permit would release immediately. Remedy: give the binding a name that telegraphs intent to hold, and/or add a one-line comment the first time the pattern appears in a file. let _permit = sem.acquire().await?; // RAII guard – keeps permit alive until end of task</code></pre> Or, for strictly lifetime-anchoring values: let _permit_keep_alive = sem.acquire().await?;</code></pre>Misreading 2: "Drop isn't happening where I expect."</h3> fn work() { let _guard = mutex.lock().unwrap(); // ...20 lines... expensive_pure_computation(); // still holds the mutex! }</code></pre> _guard</code> drops at the end of the function, not at the end of the work that actually needs it. A reviewer might assume the mutex is released before expensive_pure_computation</code>; it isn't. Remedy: use explicit drop(_guard);</code>, or nest a block to bound the critical section: fn work() { { let _guard = mutex.lock().unwrap(); protected_step(); } expensive_pure_computation(); // now runs unlocked }</code></pre>Misreading 3: "The first tuple element isn't used."</h3> let (_tempdir, workspace) = create_test_workspace();</code></pre> This is misreading 1 in disguise, but dressed up as a tuple destructure. The first element is a TempDir</code> whose Drop</code> impl deletes the on-disk directory. Binding it to _tempdir</code> keeps the directory alive for the test body. Destructuring with let (_, workspace) = …</code> would delete the directory immediately and break every subsequent I/O operation on workspace</code>. Remedy: the name. _tempdir</code> reads better than _tmp</code>; a doc comment on the helper that returns the tuple seals the intent. Misreading 4: "Just use drop()</code> at the right place - guards are magic."</h3> The concern here is that Drop at scope exit is "implicit" and therefore opaque. In practice the placement is predictable (closing brace of the enclosing block) and tools can show you: rust-analyzer</code>'s "inlay hints" will display drop points; cargo clippy</code> catches the main misuse shapes. You don't need to treat Drop as magic - you need to treat it as a compile-time scheduling rule. 7. When to use the pattern - and when not to</h2> Use a drop guard when: You hold a resource whose release is deterministic and tied to a lexical region.</li> The release is infallible or best-effort (dropping a MutexGuard</code> can't fail meaningfully; deleting a TempDir</code> is best-effort - errors are swallowed).</li> You want the release to happen on every exit path, including panic, without threading cleanup calls through every branch.</li> You want the guard to compose into larger owned units (structs, tuples, closures captured by move</code>).</li> </ul> Avoid the pattern or supplement it when: The release is fallible in a way the caller must observe. Drop</code> cannot return errors and cannot be await</code>ed. Flushing a buffered writer, for instance, should usually be explicit (writer.flush()?</code>) with the guard as a safety net for the panic path - not as the primary mechanism. BufWriter</code>'s drop flushes best-effort and ignores errors.</li> The resource needs async cleanup. Drop</code> is synchronous. For async cleanup, use explicit close().await</code> and let the guard handle only the emergency-panic path. The AsyncDrop</code> RFCs are still in flux.</li> The scope-exit timing is wrong. If the "natural" scope is larger than the time you want to hold the resource, use drop(x)</code> or a nested block.</li> </ul> For ad-hoc "run this on scope exit" needs that don't justify a dedicated type - restoring some thread-local, undoing a temporary state mutation in a test - the scopeguard</code></a> crate provides defer!</code> and defer_on_unwind!</code> macros. They are RAII guards underneath (a generated struct with a Drop</code> impl) and complement, rather than replace, named guard types: reach for defer!</code> when the cleanup is one-off and inline; reach for a named guard when the resource has a real type that consumers should see in signatures. 8. Quick reference</h2> // Standard: bind, use implicitly via Drop, scope-exit releases. let _permit = sem.acquire().await?; work().await; // drop here // Explicit early release. let permit = sem.acquire().await?; hot_path().await; drop(permit); cold_path().await; // Nested scope to bound release. let result = { let _guard = mutex.lock().unwrap(); compute_under_lock() }; // _guard released here use_result(result); // Tuple-returning helper with an anchor. let (_tempdir, workspace) = make_workspace(); // _tempdir lives as long as this frame; don't rebind to `_`. // WRONG: immediate drop. let _ = sem.acquire().await?; // permit dropped NOW let _ = mutex.lock().unwrap(); // unlocked NOW let _ = TempDir::new().unwrap(); // directory deleted NOW</code></pre> 9. Further reading</h2> Rust reference on drop order: https://doc.rust-lang.org/reference/destructors.html</a></li> std::mem::drop</code> and destructor semantics: https://doc.rust-lang.org/std/ops/trait.Drop.html</a></li> Tokio Semaphore</code> - "Using a semaphore as a limiter" example: https://docs.rs/tokio/latest/tokio/sync/struct.Semaphore.html</a></li> tempfile::TempDir</code> - guarantees and pitfalls: https://docs.rs/tempfile/latest/tempfile/struct.TempDir.html</a></li> Clippy lint let_underscore_lock</code>: https://rust-lang.github.io/rust-clippy/master/#let_underscore_lock</a></li> Article: "Using Drop</code> to write idiomatic Rust" by Jon Gjengset and others - covers the pattern under the name "drop guards."</li> </ul> Takeaway</h2> A let _name = value;</code> binding where value</code> has a meaningful Drop</code> is not an "unused variable." It is a lexical declaration of a resource's lifetime. The value's type - MutexGuard</code>, SemaphorePermit</code>, TempDir</code>, File</code>, custom Drop</code> impl - tells you what's being held. The scope - function body, block, match arm - tells you for how long. The scope's }</code> is the release point. Once that triad (type, scope, brace) is legible to a reader, the pattern stops being "clever" and becomes background mechanism. Until then, the cheapest thing to do in a codebase is name the bindings well (_tempdir</code>, _permit</code>, _guard</code>) and document the pattern at its first load-bearing use in the file. Dyn-Compatible Async Traits in Rust: Why the Manual Boxed Future Idiom is Required Unknown — Sun, 26 Apr 2026 00:00:00 +0000 Pin<Box<dyn Future + Send>></code> in Traits - Why the Manual Desugar?</h1> Context</h2> When a Rust trait needs async methods and must be usable as a trait object (Box<dyn Trait></code>, &dyn Trait</code>, Arc<dyn Trait></code>), you cannot just write async fn</code> and move on. async fn</code> in traits returns an opaque impl Future</code> whose concrete type differs per implementation - which breaks vtable dispatch. The compiler's own error-recovery hint spells it out: to make this trait dyn-compatible, use -> Pin<Box<dyn Future<Output = ...> + Send>></code> instead. </blockquote> The manual form - write a synchronous fn returning Pin<Box<dyn Future<Output = T> + Send>></code>, implement it with Box::pin(async move { ... })</code> - exists to preserve dyn-compatibility. It is what #[async_trait]</code> generates under the hood. It is what tower::Service</code>, hyper::Service</code>, and many service-oriented abstractions use (sometimes with an associated future type, sometimes with the boxed form). It looks unfamiliar at first; it is mechanically load-bearing. This doc explains why the pattern exists, what it gives you, and the lifetime subtlety that turns every "clone Arc fields before async move</code>" line in the implementation into a consequence of the trait's signature. 1. The setup: async + polymorphism + pluggable backends</h2> A common design: You define a trait that abstracts some capability with async methods - an executor, a service, a storage backend, a message bus.</li> Production uses one implementation (a real HTTP executor, a real database backend).</li> Tests use another (a mock that records calls and returns canned responses).</li> Consumers of the trait want to swap between them at construction time without recompiling.</li> </ul> The most natural Rust encoding is: pub struct Dispatcher { executor: Box<dyn StepExecutor>, // production or mock, decided at construction }</code></pre> For this to compile, StepExecutor</code> must be dyn-compatible (the compiler's current term for what used to be called "object-safe"). That is: you can build a vtable for it, because every method's signature is known at compile time regardless of the Self</code> type that implements it. Synchronous methods are trivially dyn-compatible. Async methods are not - and that is where the whole pattern starts. 2. What async fn</code> actually is</h2> async fn foo(...) -> T { body }</code> is sugar for: fn foo(...) -> impl Future<Output = T> { async move { body } }</code></pre> The return type is impl Future<Output = T></code> - an anonymous existential type whose concrete identity is "whatever compiler-generated state machine this particular function body produced." The caller gets something that implements Future</code>, but the caller does not know (and cannot name) the concrete type (that's what existential types are for). This is excellent for monomorphized, generic code. The compiler specializes each call site against the concrete future type and inlines aggressively. It is fatal for dyn dispatch. 2.1 Why impl Future</code> and dyn dispatch are incompatible</h3> A vtable stores function pointers with fixed signatures. For a sync method fn run(&self, x: u32) -> bool</code>, the vtable slot is a pointer to a function taking (&dyn Trait, u32)</code> and returning bool</code>. Every implementation of the trait must produce a function with that exact signature. For fn run(&self, x: u32) -> impl Future<Output = bool></code>: Implementation A's impl Future</code> is some compiler-generated Running<A></code> state machine.</li> Implementation B's impl Future</code> is a different Running</code> state machine with a different size, different layout, and different poll</code> code.</li> They are not the same type. There is no single function signature the vtable can store.</li> </ul> You cannot put Running<A></code> and Running</code> behind the same function pointer. Polymorphism through dispatch requires a stable, type-erased return type. impl Future</code> is the opposite of type-erased. 2.2 What the compiler does today (Rust 1.75+)</h3> You can write async fn</code> in traits. The compiler added that support in Rust 1.75 (late 2023). But the same method on dyn Trait</code> is rejected with an error that explicitly instructs you to switch to the manual form. Consider this minimal example - a trait with one async fn</code>, then an attempt to use it behind a trait object: type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>; pub trait StepExecutor { async fn execute(&self) -> Result<()>; } fn build() -> Box<dyn StepExecutor> { todo!() }</code></pre> The trait declaration compiles fine on its own. The line Box<dyn StepExecutor></code> is what trips the dyn-compatibility check, producing: error[E0038]: the trait `StepExecutor` is not dyn compatible --> src/lib.rs:10:18 | 10 | fn build() -> Box<dyn StepExecutor> { todo!() } | ^^^^^^^^^^^^^^^^^ `StepExecutor` is not dyn compatible | note: for a trait to be dyn compatible it needs to allow building a vtable help: consider boxing the future: | - async fn execute(&self) -> Result<()>; + fn execute(&self) -> Pin<Box<dyn Future<Output = Result<()>> + Send + '_>>;</code></pre> The compiler is telling you the lowered form. Adopting it is the way to keep dyn compatibility. 3. The manual form, line by line</h2> Here is the paired trait declaration and impl, written the manual way: // Trait declaration pub trait StepExecutor: Send + Sync { fn execute_prepared( &self, requests: Vec<PreparedRequest>, max_concurrency: usize, ) -> Pin<Box<dyn Future<Output = Vec<Result<ExecutionResult>>> + Send>>; } // Implementation impl StepExecutor for ParallelExecutor { fn execute_prepared( &self, requests: Vec<PreparedRequest>, max_concurrency: usize, ) -> Pin<Box<dyn Future<Output = Vec<Result<ExecutionResult>>> + Send>> { let executor = self.http_executor.clone(); // (1) snapshot fields let handler = self.adapter_handler.clone(); // before the async block Box::pin(async move { // (2) build the future let sem = Arc::new(Semaphore::new(max_concurrency.max(1))); let mut set = JoinSet::new(); for req in requests { let exec = executor.clone(); let hdl = handler.clone(); let s = sem.clone(); set.spawn(async move { let _p = s.acquire().await.ok()?; exec.dispatch(req, hdl).await }); } set.join_all().await }) // (3) return the pinned box } }</code></pre> What each piece does: (1) Clone fields before async move</code>. The returned future is Send + 'static</code>. By the default object bounds rule (Rust Reference, Trait object types), when a dyn Trait</code> appears inside Box<…></code>, Rc<…></code>, or Arc<…></code> without an explicit lifetime, the bound defaults to 'static</code>. So Pin<Box<dyn Future<Output = …> + Send>></code> is shorthand for Pin<Box<dyn Future<Output = …> + Send + 'static>></code>. async move</code> captures everything it references by move. If it captured &self.http_executor</code>, the future would need to outlive self</code>, which means its type would need a lifetime parameter (+ '_</code>). The trait said no such parameter. So the body cannot touch self</code> directly - it must capture owned clones of Arc-wrapped fields. (2) Box::pin(async move { ... })</code>. The async move { ... }</code> block produces some compiler-generated future type with the moved-in captures. Box::pin</code> heap-allocates it and pins it - pinning is required because futures may self-reference (an await point can borrow locals that live in the future's state machine), and moving such a future invalidates those borrows. Pinning prevents the move. Once allocated and pinned, we erase the concrete type by coercing to Pin<Box<dyn Future<Output = T> + Send>></code>. (3) The return type unifies every impl's future. ParallelExecutor::execute_prepared</code> returns one heap-allocated, pinned future. MockStepExecutor::execute_prepared</code> returns a different heap-allocated, pinned future. To the caller - through dyn StepExecutor</code> - they both look like Pin<Box<dyn Future<Output = ...> + Send>></code>. The vtable stores one function pointer; each impl's pointer builds and returns a different underlying future, but all wearing the same type-erased wrapper. That is the whole trick. The return type is the lingua franca that makes dispatch possible. 4. Why the .clone()</code> dance is load-bearing, not just style</h2> The trait signature controls what the returned future may and may not do. Two constraints matter: 4.1 + Send</code></h3> The future will cross thread boundaries - it will be spawned on a tokio runtime, passed to a JoinSet</code>, select!</code>-ed against other futures, etc. Send</code> is mandatory for that to be sound. Every value captured by the async move</code> must itself be Send</code>. If self.http_executor</code> is Arc<HttpExecutor></code> and HttpExecutor: Send + Sync</code>, then Arc<HttpExecutor>: Send</code>, and cloning it into a local gives you a Send</code> owned handle. If you tried to capture &self.http_executor</code>, you'd capture a &HttpExecutor</code> - which is Send</code> only if HttpExecutor: Sync</code>, and even then you'd tie the future's lifetime to self</code>. 4.2 'static</code> (implicit in the absence of + '_</code>)</h3> Inside Pin<Box<dyn Future<Output = T> + Send>></code>, the trait-object bound is 'static</code> by default. There is no lifetime parameter on the return type. The future must not contain any borrows that outlive the enclosing function - it must own all its captures. If you write: fn execute_prepared(&self, ...) -> Pin<Box<dyn Future<Output = T> + Send>> { Box::pin(async move { do_work(&self.http_executor).await // captures &self! }) }</code></pre> The compiler rejects this: self</code> has an anonymous lifetime bounded by the call, but the returned future needs 'static</code>. The error will be some variant of "self</code> does not live long enough" or "captured variable cannot escape FnMut</code> closure body" or "argument requires that '1</code> must outlive 'static</code>," depending on the exact shape. The fix is to replace the borrow with an owned clone: fn execute_prepared(&self, ...) -> Pin<Box<dyn Future<Output = T> + Send>> { let executor = self.http_executor.clone(); // owned Arc, 'static Box::pin(async move { do_work(&executor).await }) }</code></pre> This is the same idiom Tokio documentation shows for tokio::spawn</code>, which has the same constraint (Send + 'static</code>). The spawn body clones Arcs before move</code> into the task. Trait-level dyn-safe async works the same way for the same reason. 4.3 If you want to borrow self</code>, say so</h3> You can write a dyn-safe async trait method that does borrow self</code>, by adding the lifetime: fn execute_prepared<'a>( &'a self, ... ) -> Pin<Box<dyn Future<Output = T> + Send + 'a>>;</code></pre> Now the future is permitted to borrow self</code> for the duration 'a</code>. The cost is that callers cannot store the future beyond the borrow; it cannot be tokio::spawn</code>ed as-is without some form of owned-handle wrapping. Most service-oriented traits choose the 'static</code> flavor (no 'a</code>) precisely so the futures are spawnable; they accept the clone-before-move cost as the price of spawn-ability. 5. #[async_trait]</code> is the same thing, macroed</h2> The async-trait</code> crate (proc macro) lets you write the sugared form: #[async_trait] pub trait StepExecutor: Send + Sync { async fn execute_prepared( &self, requests: Vec<PreparedRequest>, max_concurrency: usize, ) -> Vec<Result<ExecutionResult>>; }</code></pre> At expansion time, the macro rewrites both the trait and every #[async_trait] impl</code> into the manual form we just showed. The return type becomes Pin<Box<dyn Future<Output = ...> + Send + 'async_trait>></code>, where 'async_trait</code> is a generated lifetime bound to &self</code>. That is the §4.3 borrowing form, not the 'static</code> form from §3 - the macro lets the future borrow self</code> for the duration of the call, so impls can write self.field.do_work().await</code> without cloning. By default the macro adds + Send</code>; opt out with #[async_trait(?Send)]</code> for futures that don't cross thread boundaries. That is: #[async_trait]</code> style and the §4.3 manual form produce equivalent desugaring; the macro is a purely syntactic shortcut and buys you nothing at runtime.</li> It is not equivalent to the §3 'static</code> form. If you want a spawnable, owns-its-data future, the macro alone won't give you that - you still write the clone-before-async move</code> dance inside the impl, just without the outer Pin<Box<…>></code> ceremony.</li> The cost of the macro is an extra dependency, some compile-time overhead, and a layer of obfuscation when reading expanded diagnostics.</li> The cost of the manual form is ~5 more lines per method: one fn</code> signature with Pin<Box<…>></code>, one Box::pin(async move { … })</code>, one set of field clones.</li> </ul> Choosing between them is largely a judgment call about dependency hygiene and explicitness. For a library with a handful of dyn-safe async methods on one trait, the manual form is often preferable: no proc macro in the dep graph, the reader sees exactly what's happening, and rustc diagnostics are about your real code rather than generated code. For a codebase with dozens of such traits, #[async_trait]</code> amortizes its dep cost and reduces ceremony. Common Rust advice: prefer the manual form when it's small and contained; prefer #[async_trait]</code> when the volume justifies it. 6. Alternatives and when to reach for them</h2> The boxed form is not the only way to make async work across trait implementations. It is the most common one when you need dyn Trait</code>. Here are the alternatives worth knowing. 6.1 Associated future type (the "tower" style)</h3> pub trait Service { type Response; type Error; type Future: Future<Output = Result<Self::Response, Self::Error>> + Send; fn call(&mut self, req: Request) -> Self::Future; }</code></pre> Each implementation names its own future type. No boxing, no allocation per call. The catch: it is not dyn-compatible on its own - to use dyn Service</code>, you typically wrap it with a helper like tower::util::BoxService</code> that boxes the future at the object boundary. tower</code> takes this approach precisely to decouple the trait's abstract shape (zero-cost associated future) from the erasure-at-the-edge concern (box only when you need dyn). Good choice when you are writing a service that will usually be composed generically and occasionally boxed. Overkill for traits that always want dyn dispatch anyway. 6.2 Native async fn</code> in traits (without dyn</code>)</h3> If you do not need dyn Trait</code>, post-1.75 you can simply write: pub trait StepExecutor: Send + Sync { async fn execute_prepared(&self, requests: Vec<PreparedRequest>) -> Vec<Result<()>>; }</code></pre> Callers use generics: fn run<E: StepExecutor>(e: E)</code> rather than fn run(e: Box<dyn StepExecutor>)</code>. The compiler monomorphizes, inlines, and produces tight code with no boxed futures. Downside: you lose type erasure. Everywhere StepExecutor</code> appears in a signature, the concrete impl leaks through as a generic parameter. If you were using dyn</code> specifically to hide the impl from downstream code (test vs production, feature-gated variants), that hiding is gone. There is a second, more subtle downside - the Send bound problem. With native async fn</code> in traits, you cannot directly write "the returned future must be Send</code>" in the trait declaration. The desugared impl Future</code> is opaque; there is no straightforward syntax to constrain its auto-traits at the trait-definition site. A consumer that needs to tokio::spawn</code> the future therefore has no way to demand Send</code> from the trait alone. This is exactly why many codebases on Rust 1.75+ still reach for #[async_trait]</code> even when they use generics: the macro's expansion to Pin<Box<dyn Future + Send + 'async_trait>></code> makes Send</code> a contractual part of the trait signature. The language has partial answers - return-type notation (where T::method(..): Send</code>, still being stabilized) lets a caller demand Send</code> on a per-method basis at the use site, and the trait-variant</code></a> crate generates a parallel Send</code>-flavored trait from a single declaration - but neither is as ergonomic as the boxed form for the "every future must be Send</code>" case. If your trait is intended for spawning, the boxed dyn</code> form remains the path of least resistance. 6.3 Both, via a helper Boxed<T></code> wrapper</h3> The pattern tower::util::BoxService</code> uses: a thin wrapper that implements the trait by boxing the associated future at each call. You define the trait with the native/associated form, then provide a Boxed</code> adapter for the erasure case. More code to maintain, but you get both generic and dyn dispatch from the same abstraction. 6.4 Returning impl Future</code> directly (no dyn, no macro)</h3> pub trait StepExecutor { fn execute_prepared(&self, requests: Vec<PreparedRequest>) -> impl Future<Output = Vec<Result<()>>> + Send; }</code></pre> Available since 1.75. Equivalent to async fn</code> in a trait. Same dyn-incompatibility. Same monomorphization behavior. 7. How to read a method written in the manual form</h2> When you encounter code like this in a codebase: fn do_thing( &self, arg: SomeArg, ) -> Pin<Box<dyn Future<Output = Result<Thing>> + Send>> { let state = self.state.clone(); let client = self.client.clone(); Box::pin(async move { let s = state.lock().unwrap(); client.request(s.derive_key(&arg)).await }) }</code></pre> Read it as: Signature: "This is async fn do_thing(&self, arg: SomeArg) -> Result<Thing></code>, written in the dyn-compatible desugared form. The returned future is Send + 'static</code> - spawnable, cross-thread-safe, owns its data."</li> The field clones: "These are the captures the future needs. They are cloned because the future is 'static</code> and cannot borrow self</code>."</li> Box::pin(async move { ... })</code>: "This is the function body of the async fn</code>. It allocates one pinned future on the heap and erases its concrete type."</li> The async block's contents: the real logic.</li> </ol> Once you have done this translation a few times, the extra ceremony recedes into background pattern. The first time, it looks like ritual; the second time, it looks like the compiler speaking through the code. 8. When the pattern is right and when it is not</h2> Use the manual Pin<Box<dyn Future + Send>></code> form when: You need a trait object (Box<dyn Trait></code>, &dyn Trait</code>, Arc<dyn Trait></code>) with async methods.</li> You want to avoid the async-trait</code> proc-macro dependency for a small number of methods.</li> You want the desugaring explicit in the source so readers can inspect the lifetime and Send</code> bounds directly.</li> </ul> Use #[async_trait]</code> when: You have many async trait methods and the repetition overhead is real.</li> You want the sugared async fn</code> reading experience across a team that's less practiced in manual futures.</li> You don't mind one proc-macro dep.</li> </ul> Skip the dyn form entirely (use generics + async fn in traits</code>) when: You never need dyn Trait</code>.</li> All your trait consumers are generic code that monomorphizes.</li> You want peak performance - no per-call boxing, no vtable indirection.</li> </ul> Use associated futures (type Future: Future<Output = …>;</code>) when: You are writing a library that wants to support both generic and boxed use without committing to one, at the cost of more scaffolding.</li> You have performance-critical hot paths where the per-call Box::pin</code> allocation measurably matters.</li> </ul> 9. Quick reference</h2> // Dyn-safe async method, manual desugar: fn foo(&self, x: T) -> Pin<Box<dyn Future<Output = R> + Send>> { let state = self.state.clone(); // clone captures (no &self in body) Box::pin(async move { state.do_work(x).await }) } // Dyn-safe async method, borrowing self: fn foo<'a>(&'a self, x: T) -> Pin<Box<dyn Future<Output = R> + Send + 'a>> { Box::pin(async move { self.state.do_work(x).await // can touch self; future tied to 'a }) } // #[async_trait] equivalent - compiler expands to the dyn-safe form: #[async_trait] trait Foo { async fn foo(&self, x: T) -> R; } // Native async fn in trait (Rust 1.75+) - works for generics, NOT for dyn: trait Foo { async fn foo(&self, x: T) -> R; } // Tower-style associated future - no box by default: trait Foo { type Future: Future<Output = R> + Send; fn foo(&self, x: T) -> Self::Future; }</code></pre> 10. Takeaway</h2> async fn</code> in traits and Box<dyn Trait></code> are two features that Rust individually supports and jointly conflict. Every implementation of an async trait method produces a different anonymous future type; vtables cannot dispatch over anonymous types; therefore dyn-safe async methods must return a concrete, type-erased representation of a future. The only such representation the language offers is Pin<Box<dyn Future<Output = T> + Send>></code> - a pinned, boxed, erased trait object of Future</code>. Everything else in the manual form is a consequence of that single decision: The Box::pin(async move { ... })</code> wrap produces the erased value.</li> The clone-before-move on Arc</code>-wrapped fields is because the future must be 'static</code> (no lifetime parameter on the return type) and therefore cannot capture &self</code>.</li> The #[async_trait]</code> macro expands to exactly this shape; reading it as generated code makes the behavior predictable.</li> </ul> Once the return type is recognized for what it is - the compiler's recommended lowering for "async in a dyn trait" - the rest of the ceremony reads as mechanics, not cleverness. References</h2> Crates</h3> tower</code></a> (repo</a>) - modular service/middleware abstraction built around the Service</code> trait with an associated future type. Source of the "tower style" in §6.1.</li> tower::util::BoxService</code></a> - the adapter that boxes the future at the dyn boundary, so a tower service can be used behind dyn</code>.</li> async-trait</code></a> (repo</a>) - the proc macro that expands sugared async fn</code> in traits into the manual Pin<Box<dyn Future + Send>></code> form.</li> hyper</code></a> and tonic</code></a> - HTTP and gRPC libraries built on top of tower::Service</code>.</li> </ul> Language and standard library</h3> Rust 1.75 release notes</a> - the release that stabilized async fn</code> in traits and -> impl Trait</code> in trait methods (RPITIT).</li> Announcing async fn</code> and return-position impl Trait</code> in traits</a> - the lang team post explaining the feature, its limitations, and explicitly why it does not work with dyn Trait</code>.</li> E0038: the trait is not dyn compatible</a> - the error code emitted when a trait cannot be made into a trait object, with the full list of reasons.</li> Reference: object safety / dyn compatibility</a> - the language reference's definition of which traits can be used as dyn Trait</code>.</li> std::pin</code></a> - module documentation for Pin</code>, including why self-referential futures need pinning.</li> std::future::Future</code></a> - the trait being boxed.</li> </ul> Background reading</h3> The Async Rust Book</a> - foundational material on Future</code>, async</code>/await</code>, executors, and pinning.</li> Tokio tutorial: Spawning</a> - explains the Send + 'static</code> requirement for tokio::spawn</code>, which is the same constraint that drives the clone-before-async move</code> idiom in §4.</li> </ul>

Cache-Conscious Data Layout in Rust: Field Zoning, False Sharing, and the 128-Byte Rule

Two counterintuitive corollaries</h2>
Once you're thinking at the level of cache lines and prefetchers, two pieces of "obvious" performance folklore turn out to be wrong.</p>

Architectural Decomposition: Remove Contention by Design

Scalar Replacement of Aggregates: How "Copy to Locals" Unlocks the Compiler

Why x86 Zeroes a Register With `xor eax, eax`

Set-Associative Caches: Trading Global Optimality for Predictable Speed

3. Memory Layout: Hash Map vs. Set-Associative Cache</h2>

5. If Set-Associative Caches Are So Good, Why Use Ordinary Caches?</h2>
Set-associative caches win on raw lookup predictability and hot-path locality. General-purpose caches win on flexibility, richer policies, and often better hit ratios for messy application workloads.</p>

Building a custom Bitset - A study in access-pattern-driven data structure choice

Zig Allocation Patterns

1. The two ownership models: aware vs. owning</h2>
Both patterns satisfy the project rule "every allocating function takes an explicit `std.mem.Allocator</code>." The split is about whether the type also stores a copy of that allocator inside itself.</strong></p>`

2. Other Zig allocation patterns</h2>
The aware/owning axis is just one dimension. Several other patterns are in active use across the standard library and this codebase.</p>

3. Anti-patterns and pitfalls</h2>

Why Zig’s Io Feels Like an Effect System (Without Being One)

Unpacking the technical claim</h2>
The previous section said `Io</code> "is just a parameter" with no propagation up the call graph. Two distinct claims, worth separating.</p>`

Enter Kyo</h2>
Kyo sits in a third corner of the design space - it is the only one of the three whose public model is directly algebraic-effect-like: effects are pending in the return type, compose as a type-level set, and are discharged by handlers.</p>

Evaluating PBT Frameworks: How Proptest and Hegel Differ in Algebraic Expressivity

Evaluating PBT Frameworks: How Proptest and Hegel Differ in Algebraic Expressivity</h1>

The RAII Drop-Guard Pattern in Rust

The RAII Drop-Guard Pattern in Rust</h1>

1. The Rust mechanics</h2>
There are three syntactic forms people confuse. They behave differently.</p>

3. Three flavors of the pattern in practice</h2>
These are three examples that came up in real code review comments on a single PR. Each uses the same underlying Rust mechanic; each looked surprising or wrong to a reviewer encountering it in a context they hadn't previously mapped to RAII.</p>

4. Why Rust uses this pattern</h2>
Rust deliberately doesn't have `try/finally</code>, defer</code>, or runtime-managed cleanup. RAII via Drop</code> is the uniform mechanism, and that is a design decision worth understanding.</p>`

6. Common misreadings</h2>
These are the shapes that trip readers who haven't fully absorbed the pattern. Each has a cheap remedy.</p>

Dyn-Compatible Async Traits in Rust: Why the Manual Boxed Future Idiom is Required

6. Alternatives and when to reach for them</h2>
The boxed form is not the only way to make async work across trait implementations. It is the most common one when you need `dyn Trait</code>. Here are the alternatives worth knowing.</p>`

References</h2>

Ruminations of a Programmer

Zero-Copy on the Hot Path: Reserve/Commit and Fast-Path/Slow-Path Splitting

Amortizing Cross-Core Coordination: Cursor Caching and Batch Processing

The Cross-Core Contract: Memory Ordering and Single-Writer State in Lock-Free Rust

Ruminations of a Programmer

Zero-Copy on the Hot Path: Reserve/Commit and Fast-Path/Slow-Path Splitting

Two taxes that have nothing to do with atomics</h2> Parts 1 through 3 were all about the cost of coordination</em> - cache lines, memory ordering, and how often cores have to talk. But a naive hot path pays two more taxes that no amount of clever atomics will fix:</p>

Amortizing Cross-Core Coordination: Cursor Caching and Batch Processing

Amortizing Cross-Core Coordination: Cursor Caching and Batch Processing</h1> Part 3 of Low-Level Systems Design in Rust</strong> - a series on writing high-throughput, low-latency systems code, using a single-producer / single-consumer (SPSC) ring buffer as the running example.</p>

The Cross-Core Contract: Memory Ordering and Single-Writer State in Lock-Free Rust

Cache-Conscious Data Layout in Rust: Field Zoning, False Sharing, and the 128-Byte Rule

Two counterintuitive corollaries</h2> Once you're thinking at the level of cache lines and prefetchers, two pieces of "obvious" performance folklore turn out to be wrong.</p>

Architectural Decomposition: Remove Contention by Design

Scalar Replacement of Aggregates: How "Copy to Locals" Unlocks the Compiler

Why x86 Zeroes a Register With `xor eax, eax`

Set-Associative Caches: Trading Global Optimality for Predictable Speed

3. Memory Layout: Hash Map vs. Set-Associative Cache</h2>

5. If Set-Associative Caches Are So Good, Why Use Ordinary Caches?</h2> Set-associative caches win on raw lookup predictability and hot-path locality. General-purpose caches win on flexibility, richer policies, and often better hit ratios for messy application workloads.</p>

Building a custom Bitset - A study in access-pattern-driven data structure choice

Zig Allocation Patterns

1. The two ownership models: aware vs. owning</h2> Both patterns satisfy the project rule "every allocating function takes an explicit std.mem.Allocator</code>." The split is about whether the type also stores a copy of that allocator inside itself.</strong></p>

2. Other Zig allocation patterns</h2> The aware/owning axis is just one dimension. Several other patterns are in active use across the standard library and this codebase.</p>

Two counterintuitive corollaries</h2>
Once you're thinking at the level of cache lines and prefetchers, two pieces of "obvious" performance folklore turn out to be wrong.</p>

5. If Set-Associative Caches Are So Good, Why Use Ordinary Caches?</h2>
Set-associative caches win on raw lookup predictability and hot-path locality. General-purpose caches win on flexibility, richer policies, and often better hit ratios for messy application workloads.</p>

1. The two ownership models: aware vs. owning</h2>
Both patterns satisfy the project rule "every allocating function takes an explicit `std.mem.Allocator</code>." The split is about whether the type also stores a copy of that allocator inside itself.</strong></p>`

2. Other Zig allocation patterns</h2>
The aware/owning axis is just one dimension. Several other patterns are in active use across the standard library and this codebase.</p>