80386 Memory Pipeline

The FPGA 386 core I've been building now boots DOS, runs applications like Norton Commander, and plays games like Doom. On DE10-Nano it currently runs at 75 MHz. With the core now far enough along to run real software, this seems like a good point to step back and look at one of the 80386's performance-critical subsystems: its memory pipeline.

32-bit Protected Mode was the defining feature of the 80386. In the previous post, I looked at one side of that story: the virtual-memory protection mechanisms. We saw how the 80386 implements protection with a dedicated PLA, segment caches, and a hardware page walker. This time I want to look at virtual memory from a different angle: the microarchitecture of the memory access pipeline, how address translation is made efficient, how microcode drives the process, and what kind of RTL timing the design achieves.

On paper, x86 virtual memory management looks expensive. Every memory reference seems to require effective address calculation, segment relocation, limit checking, TLB lookup, and, on a miss, two page-table reads plus Accessed/Dirty-bit updates. Yet Intel's own 1986 IEEE ICCD paper, Jim Slager's Performance Optimizations of the 80386, describes the common-case address path as completing in about 1.5 clocks. How did the 386 pull that off?

The answer is that virtual memory is not really a serial chain of checks, even if the diagrams make it look that way. It is a carefully overlapped memory pipeline that uses pre-calculation, pipelining, and parallelism to keep the common case surprisingly short.

Microcode for memory accesses

Intel's 80386 Programmer's Reference Manual describes 80386 address translation like this:

"The 80386 transforms logical addresses (i.e., addresses as viewed by programmers) into physical address (i.e., actual addresses in physical memory) in two steps: segment translation... and page translation..."

Before looking at the hardware, it helps to start from the microcode. Here is the microcode for an ALU instruction that reads memory, modifies it, and writes it back, for example ADD [BX+4], 8:

; ADD/OR/ADC/SBB/AND/SUB/XOR m,i
039 EFLAGS -> FLAGSB FLGSBA RD 9
03A DLY
03B OPR_R -> TMPB WRITE_RESULT JMP UNL
03C TMPB IMM +-&|^
; WRITE_RESULT
046 SIGMA -> OPR_W RNI WR 0
047 DLY

Three things matter here:

• RD starts a memory read
• WR starts a memory write
• DLY is where the microcode waits for the result to become available

Intel's answer was to build a dedicated address path that usually adds only about one extra cycle, or in their own phrasing, about 1.5 clocks for the address pipeline itself.

Efficient segmentation

Segmentation is mandatory and active in both protected and real mode. The first optimization is simply to avoid repeating descriptor lookup on every access. When a selector is loaded into a segment register, the processor also loads the descriptor's base, limit, and attributes into the register's invisible part (descriptor caches).

For real mode, the microcode uses a special operation SBRM (set-base-real-mode) to modify the base value in the hidden descriptor cache. Interestingly, the real-mode routine does not touch the limit value. This design choice is what makes the famous "unreal mode" trick possible: by entering protected mode, setting a large limit, and returning to real mode, software can access a 4 GB data segment while remaining in real mode.