Exposing Floating Point – Bartosz Ciechanowski (2019)

Despite everyday use, floating point numbers are often understood in a hand-wavy manner and their behavior raises many eyebrows. Over the course of this article I’d like to show that things aren’t actually that complicated.

This blog post is a companion to my recently launched website – float.exposed. It’s intended to be a handy tool for inspecting floating point numbers. On a technical note, by floating point I’m referring to the ubiquitous IEEE 754 binary floating point format. Types half, float, and double are understood to be binary16, binary32, and binary64 respectively. Whatever device you’re reading this on is pretty much guaranteed to use IEEE 754.

We’ll begin with the very basics of writing numeric values. Consider the number 327.849. Digits to the left of the decimal point represent increasing powers of ten, while digits to the right represent decreasing powers of ten. Even though this notation is very natural, it has a few disadvantages regarding very small or very large numbers and precision management.

Scientific notation solves all these problems. It shifts the decimal point to right after the first non-zero digit and sets the exponent accordingly. It has three major components: the sign (+), the significand, and the exponent.

Binary numbers can use scientific notation as well. Since the only non-zero digit in base-2 system is 1, every non-zero binary number in scientific notation starts with a 1. Floating points numbers are just numbers in base-2 scientific notation with two restrictions: a limited number of digits in the significand and a limited range of the exponent.

Unfortunately, because the number of bits is limited, some real values may not be perfectly representable. A decimal number 0.2 has a forever repeating value in binary. The 25th and later significant digits won’t fit in a float and have to be accounted for by rounding. This explains why some decimal numbers don’t have their exact floating point representation.

To encode these into binary (like a 32-bit float), IEEE 754 uses 1 bit for the sign, 8 bits for the exponent (using a bias of 127 to avoid negative numbers), and 23 bits for the significand (dropping the leading '1' as an implicit bit to save space).