The end of the AArch64 desktop experiment

After about eleven months of using an Ampere Altra AArch64 system as a desktop, the author decided to end the experiment due to hardware errata, GPU driver issues, and software ecosystem limitations.

This post is part 7 of the "Let me try to use an AArch64 system as a desktop" series:

AArch64 desktop: day one
AArch64 desktop: day two
AArch64 desktop: last day
Arm desktop: 2025 attempt, part one
Arm desktop: emulation
Arm desktop: so many cores, not enough speed
The end of the AArch64 desktop experiment

After about eleven months of using an AArch64 desktop, I decided to end that experiment.

Hardware used

About a year ago, I bought myself an Ampere Altra system. After moving some hardware around and making a few extra orders, the final setup was:

To be fair, I should mention that this is a server motherboard, not a desktop one, and Altra systems were never meant to be desktops (despite companies selling them as such). Naturally, the list of tested/approved devices (Qualified Vendor List (QVL for TLA fans)) is quite short and for Ampere Altra systems, it does not contain AMD Radeon GPU cards. They can be made to work, but this often requires additional effort.

The extra USB 3.2 controller allowed me to have more USB devices than the motherboard alone supported, and gave me some 10Gbps ports for connecting external NVMe drives.

The whole system was running just fine* under Fedora 42–44.

The first issue

Have you noticed the small “*” at the end of the previous paragraph? The system I used was not quite Fedora — I had to use my own, self-built kernel.

You see, the PCI Express controller in the Ampere Altra has some issues. Let me quote the description of the Ampere Altra erratum 82288 patches:

Per Altra family erratum, PCIE_65 may cause invalid addresses to be generated on PCIe mmio writes, impacting certain device types, notably AMD GPUs, and thus the Altra family is not generally compatible with those device types.

And longer description from patch itself:

PCIe device drivers may map MMIO space as Normal, non-cacheable memory attribute (e.g. Linux kernel drivers mapping MMIO using ioremap_wc). This may be for the purpose of enabling write combining or unaligned accesses. This can result in data corruption on the PCIe interface’s outbound MMIO writes due to issues with the write-combining operation.

The workaround modifies software that maps PCIe MMIO space as Normal, non-cacheable memory (e.g. ioremap_wc) to instead Device, non-gathering memory (e.g. ioremap). And all memory operations on PCIe MMIO space must be strictly aligned.

So, to have a working Linux system, I had to rebuild the kernel on every package update. Which usually meant “weekly”. Each Monday or Tuesday, I would update the local copy of the Fedora kernel package repository and build it using my own versioning scheme, like “7.0.2-200.fc44.pcie65.6”. The “pcie65” part reminded me which patches I had applied, and the “6” was a counter for the patch rebases.

I cloned the repository from GitHub and then rebased patches, adapting them whenever they needed work. The side effect was that I often used a newer kernel than the official Fedora release — there is a “stabilisation” branch in the Fedora kernel package repo where the soon-to-be-pushed version is present. So, when Fedora had 6.19.y kernel, I had 7.0.z one.

So many cores, not enough speed

As I wrote in my previous post, having eighty CPU cores does not mean that the system is a good, fast desktop machine.

AMD GPU started failing

As I mentioned above, to get my AMD Radeon RX6700XT running properly I had to alter kernel with the out-of-tree patches. It worked, I could play some games, watch videos with hardware-assisted video decode acceleration.

Until one day, around the Linux 7.0 release, when it started to fail. Running a game ended with:

kernel: amdgpu 0000:03:00.0: Fence fallback timer expired on ring vcn_dec_0
kernel: amdgpu 0000:03:00.0: Fence fallback timer expired on ring vcn_dec_0
kernel: amdgpu 0000:03:00.0: Fence fallback timer expired on ring vcn_dec_0

Over and over again. Watching YouTube videos became impossible due to 720 out of 750 frames being dropped, etc.

Normally I would start to bisect the kernel to find out where the problem is. But I was running a tainted kernel due to PCIE65 patches so who knew where the problem actually was…

Let’s get Nvidia

I bought an Nvidia RTX 2060 graphics card and put it in place of the AMD Radeon. It turned out that if I wanted to use it with the nouveau kernel driver I still needed PCIE65 patches applied…

So I tried default Fedora kernel with Nvidia binary driver. And it worked fine. Video decoding was accelerated, some games under Wine worked as well.

But then I started FreeCAD. And OrcaSlicer. And in both cases I got crash and exit…

It turned out that there was no org.freedesktop.Platform.GL.nvidia in Flatpak repositories for AArch64. And I used both of those tools quite often.

Powering up the old x86-64…

At that point, I gave up. And booted my x86-64 system, which had been powered off all that time. There were a lot of cables to move, some new ones to arrange, and now I have both “wooster” (Ampere Altra) and “puchatek” (Ryzen 5 3600) systems running under my desk.

Moving from 80 cores to 6 cores (12 threads) was a weird experience. A much smaller number, yet things work fine. I can load all threads and the music still plays. All games from my Steam library are playable. A working FreeCAD allows me to finish designing cases for my home projects and I can 3D print prototypes straight from OrcaSlicer.

The “wooster” system stays powered on, churning through RISC-V package builds. It may be weak in single-thread, but it flies when it comes to multi-core load.

Conclusion

As for the Ampere Altra, I am not planning to repeat this experiment. Another AArch64 desktop attempt would require a completely new hardware platform. And I have no plans to spend over twenty thousand PLN to buy an Nvidia DGX Spark system.

Source: Hacker News