Endian wars and anti-portability: this again?

Note: This is another really technical post. Those who do not know the difference between a PowerPC and an ARM can probably skip this one – we’ll be back to general audience posts tomorrow!

One of my friends made the mistake this evening of linking me to a thread about endianness on that bastion of free expression, Hacker News. (Feel the shade dripping from my virtual pen.). I had drafted this article for Old Blog, but I never published it because I was never quite satisfied that I had a good ending. Well, today Zach has gotten me on with it. I rewrote the ending.

I grow weary, dear reader, of the hostility in the broader open source software community against portability. It is exhausting to hear the same tired arguments against portability, especially when in the end most projects end up realising the errors of their ways. So, here are some of the most common arguments I hear and my rebuttals to them.

Against “old” architectures

The most disturbing, and frequent, argument that I hear is that whatever port I have suggested (or indeed written) is for an “old” architecture that “is not relevant today”.

The only architectures supported by Linux that have no commercial viability left are the DEC Alpha AXP and the Itanium. The Itanium has been removed from the Linux kernel with 6.7, which warranted its own article on Old Blog, so I will remove that one from the discussion.

In my opinion, the Alpha is a valuable resource for learning some of the core concepts of a RISC architecture. In fact, the team behind RISC-V said they took a significant amount of inspiration from Alpha, and had it not been for some of the oddities around storing quadwords, they may have tried to resurrect it instead of bringing up their own ISA. I think there are still more valuable lessons to learn from it, which is why I’ve been quite happy to see development progressing on it! This has been primarily driven by the Gentoo community around it.

As for MIPS, it is still sold and used in products with commercial support. SPARC is an open architecture with no patents, which is highly valuable. PowerPC is continuing to evolve, and older PPC systems are a very inexpensive way to obtain a RISC workstation with actual expandable RAM, PCI Express slots, and so on. Even the Motorola 68000 architecture continues to see embedded use today.

If the community is offering you a port to an architecture, whether it is 4 days old or 40 years old, that means the community actively wants to use your software on it – otherwise, nobody would put in the effort. Ports like this are hard, and authors like me already know we are fighting an uphill battle just trying to make upstream projects care. We aren’t doing it for our health or as some sort of game. We are doing it because it actually solves a problem we have.

I’d also like to point out that I found an actual security bug in the Linux kernel by booting it on a “real” 586 (Intel Pentium, ca 1995). This was caused by an invalid use of the kernel static_key API, which allowed non-root access to a protected API on the 586, but went silent on newer architectures. It was also silent on QEMU emulation of the 586, which is what kernel developers were using, because of buggy CR4 emulation.

Against big endian

I regularly hear things like “little endian won” or “big endian does not exist any more”. This is not only wrong, but it frames things like a war. It is from this rhetoric on Hacker News that I brought this article draft back out and finished it. I fully believe that both types of endian are necessary for a healthy computing ecosystem, and I wouldn’t want to participate in software engineering without either one existing. Full stop.

Endian correctness is almost a meme in my friend circle because of how adamant I am about testing all code on both big and little endian systems. For those who don’t know, endianness is simply how the computer stores numbers. Big endian systems store numbers the way us humans do: the largest number is written first. Little endian systems store numbers opposite: the smallest number is written first. Hence, ‘big’ vs ‘little’.

I happen to prefer big endian systems in my own development life because they are easier for me to work with, especially reading crash dumps. Big endian systems also catch out some classes of bugs far quicker and easier by “failing fast” vs little endian systems where bugs can lurk. Such classes include invalid type casts which can lead to security bugs.

Recently, a memory corruption bug was found in Git. And it was found because the test suite wasn’t working properly on a big endian computer. The bug was there in little endian, too, but it wasn’t making the test suite crash. I additionally found an invalid memory read/store in Clang that was wrong everywhere, but only crashing on 32-bit big endian hardware.

It is usually easy to write code that is endian-safe. Any code that is not endian-safe is poorly written and harder to maintain at best, and possibly obscuring security bugs at worst. Any project maintainer should be jumping for joy when they receive a patch adding a big-endian port of their project, especially if it includes reports that tests pass and the software works. That is the sign of a codebase that has a level of sanity that should not be noteworthy, yet is.

Popular architectures that exist today and are exclusively big endian including the System/390 (s390x) mainframe, and the SPARC. Bi-endian architectures, which can be either big or little, include PowerPC (yes, even the Power9 and 10!) and Arm. You can actually boot a Pine A64 and most Raspberry Pi boards in big endian mode, if you need an easy-to-obtain and inexpensive BE CI computer! This is an area of development I hope to focus on eventually.

Against 32-bit

A lot of arguments against 32-bit are variants on the “old” argument, as most 32-bit architectures predate 64-bit counterparts. RISC-V introduced a 32-bit variant of their ISA, so it isn’t entirely true, but I accept that most 32-bit architectures are older than 64-bit ones.

As I’ve noted, “old” does not always equate to “bad” nor “worse”. I’ll take this moment to note that in the developing world, and in poorer communities even in the developed world, many people are getting by with 32-bit ARM systems or Celerons for desktop hardware.

There is another argument made against 32-bit architectures, and that is their memory space. You are limited to 4 GB – in most cases, typically a bit less than that. There is something to be said for a lack of 32-bit support in the HPC or machine learning space, considering the massive datasets used there. However, most software programs are not in those spaces. There is no real reason that the 4 GB memory barrier should in any way prevent a Web browser, email client, or program compiler from starting.

For that reason, 32-bit support can sometimes be a little harder because it requires optimisation and efficiency, something far too many software programs lack. This has always been a challenge in our industry, and it will continue to be. If someone has managed to make your software work on 32-bit, you should accept that port as it will help ensure your software remains efficient even on 64-bit systems. After all, if every program is required to fit in 4 GB, that means the 32 GB RAM workstation you are spoiled with can run a full 8 programs on it! Think of the multitasking you can do with that sort of power!

In summary

Open source and Libre software projects are almost always a passion project for those involved. Even when libre software is controlled by commercial interest, the employees working on it feel some sort of personal connection to the software they maintain. Linux didn’t grow as big as it has because of any one single technical quality; it grew big because of the passionate people behind it.

In closing, let me reiterate this point so it is crystal clear. If you are a maintainer of a libre software proj