Principles of Mechanical Sympathy

Principles of Mechanical Sympathy

Modern hardware is remarkably fast, but software often fails to leverage it. Mechanical sympathy - a concept borrowed from racing and popularized in software by Martin Thompson - is the practice of creating software that is sympathetic to its underlying hardware. This practice can be distilled into a set of everyday principles: Predictable memory access, awareness of cache lines, the single-writer principle, and natural batching. Together, these principles can be used to optimize everything from an AI inference server to a distributed data platform.

07 April 2026

Over the past decade, hardware has seen tremendous advances, from unified memory that's redefined how consumer GPUs work, to neural engines that can run billion-parameter AI models on a laptop.

And yet, software is still slow, from seconds-long cold starts for simple serverless functions, to hours-long ETL pipelines that merely transform CSV files into rows in a database.

Back in 2011, a high-frequency trading engineer named Martin Thompson noticed these issues, attributing them to a lack of Mechanical Sympathy. He borrowed this phrase from a Formula 1 champion:

You don't need to be an engineer to be a racing driver, but you do need Mechanical Sympathy.

-- Sir Jackie Stewart, Formula 1 World Champion

Although we're not (usually) driving race cars, this idea applies to software practitioners. By having sympathy for the hardware our software runs on, we can create surprisingly performant systems. The mechanically-sympathetic LMAX Architecture processes millions of events per second on a single Java thread.

Inspired by Martin's work, I've spent the past decade creating performance-sensitive systems, from AI inference platforms serving millions of products at Wayfair, to novel binary encodings that outperform Protocol Buffers.

In this article, I cover the principles of mechanical sympathy I use every day to create systems like these - principles that can be applied most anywhere, at any scale.

Not-So-Random Memory Access

Mechanical sympathy starts with understanding how CPUs store, access, and share memory.

Most modern CPUs - from Intel's chips to Apple's silicon - organize memory into a hierarchy of registers, buffers, and caches, each with different access latencies:

• Each CPU core has its own high-speed registers and buffers which are used for storing things like local variables and in-flight instructions.
• Each CPU core has its own Level 1 (L1) Cache which is much larger than the core's registers and buffers, but a little slower.
• Each CPU core has its own Level 2 (L2) Cache which is even larger than the L1 cache, and is used as a sort of buffer between the L1 and L3 caches.
• Multiple CPU cores share a Level 3 (L3) Cache which is by far the largest cache, but is much slower than the L1 or L2 caches. This cache is used to share data between CPU cores.
• All CPU cores share access to main memory, AKA RAM. This memory is, by an order of magnitude, the slowest for a CPU to access.

Because CPUs' buffers are so small, programs frequently need to access slower caches or main memory. To hide the cost of this access, CPUs play a betting game:

• Memory accessed recently will probably be accessed again soon.
• Memory near recently accessed memory will probably be accessed soon.
• Memory access will probably follow the same pattern.

In practice, these bets mean linear access outperforms access within the same page, which in turn vastly outperforms random access across pages.

Prefer algorithms and data structures that enable predictable, sequential access to data. For example, when building an ETL pipeline, perform a sequential scan over an entire source database and filter out irrelevant keys instead of querying for entries one at a time by key.

Cache Lines and False Sharing

Within the L1, L2, and L3 caches, memory is usually stored in “chunks” called Cache Lines. Cache lines are always a contiguous power of two in length, and are often 64 bytes long.

CPUs always load (“read”) or store (“write”) memory in multiples of a cache line, which leads to a subtle problem: What happens if two CPUs write to two separate variables in the same cache line?

You get False Sharing: Two CPUs fighting over access to two different variables in the same cache line, forcing the CPUs to take turns accessing the variables via the shared L3 cache.

To prevent false sharing, many low-latency applications will “pad” cache lines with empty data so that each line effectively contains one variable. The difference can be staggering:

• Without padding, cache line false sharing causes a near-linear increase in latency as threads are added.
• With padding, latency is nearly constant as threads are added.

Importantly, false sharing only appears when variables are being written to. When they're being read, each CPU can copy the cache line to its local caches or buffers, and won't have to worry about synchronizing the state of those cache lines with other CPUs' copies.

The Single Writer Principle

False sharing isn't the only problem that arises when building multithreaded systems. There are safety and correctness issues (like race conditions), the cost of context-switching when threads outnumber CPU cores, and the brutal overhead of mutexes (“locks”).

These observations bring me to the mechanically-sympathetic principle I use the most: The Single Writer Principle.

In concept, the principle is simple: If there is some data (like an in-memory variable) or resource (like a TCP socket) that an application writes to, all of those writes should be made by a single thread.

Let's consider a minimal example of an HTTP service that consumes text and produces vector embeddings of that text. Most AI runtimes can only execute one inference call to a model at a time. In the naive architecture, we use a mutex to work around this problem. Unfortunately, if multiple requests hit the service at the same time, they'll queue for the mutex and quickly succumb to head-of-line blocking.

We can eliminate these issues by refactoring with the single-writer principle. First, we can wrap access to the model in a dedicated Actor thread. Instead of request threads competing for a mutex, they now send asynchronous messages to the actor.

Natural Batching

Using the single-writer principle, we've removed the mutex from our simple AI service, and added support for batching. Because the actor is the single-writer, it can group independent requests into a single batch inference call to the underlying model, and then asynchronously send the results back to individual request threads.