Math Is Hard

A deep dive into the technical challenges of handling arithmetic exceptions on the VAX architecture, where a simple division by zero could lead to infinite loops in the kernel.
When you're on the other side of the fence, and working in the kernel, all the gory details which will heavily differ across platforms can no longer be ignored, and sometimes, the shortcomings of a given processor architecture can become a real pain in the arse.
For example, if you have read the m88k saga, you might remember that the need, for the operating system exception handler, to perform all the pending load and stores before returning from exception processing, had been a source of problems for years.
The 88100 processor is not the only processor which sometimes makes the kernel developer's life harder than it could have been.
Let me tell you about a processor design choice which turned out to have a significant cost in the kernel (but in a rare situation.)
The exception model of the VAX was also quite simple, with the ``Exceptions and Interrupts'' chapter of the VAX Architecture Reference Manual being only 36 pages long in the first edition (and 43 in the second edition, mostly because of a slightly larger font rather than extra text.)
Quoting from it:
So far, this is textbook processor design. If the processor encounters a situation which is not recoverable (and will cause your process to be killed), it's a trap.
If, however, there is a chance that some recovery action can be done and the offending instruction given another chance, then it's a fault.
For example, accessing a memory page which is not mapped will cause a fault. If the address is legitimate, the appropriate page and its contents will be fetched from swap (or from the binary file you are running), and the operation can be restarted. If the address is not legitimate, then your process will be sent a SIGSEGV signal and die.
Dividing by zero, on the other hand, is a trap. No matter what one may try to bend the laws of mathematics, there is no way for such a computation to ever deliver a meaningful result. Your process will be sent a SIGFPE (Floating-Point Exception) signal - even if this was an integer divide. (The siginfo_t extra information will let an hypothetical signal handler tell integer divide by zero (FPE_INTDIV) and floating-point divide by zero (FPE_FLTDIV) apart.)
So far, so good - the VAX exception handler (trap() in sys/arch/vax/vax/trap.c) would let the VM system recover the missing page faults, and would send a SIGFPE signal down the throat of your process, for arithmetic traps. This code has been almost unchanged since 3BSD.
Source: Hacker News















