Scrying the AMD GFX1250 LLVM Tea Leaves

Scrying the AMD GFX1250 LLVM Tea Leaves

In just a few short days, AMD will be showing off their brand new MI400 series of Datacenter Accelerators at their Advancing AI event but before that event comes, we thought it would be fun to attempt to scry the tea leaves that are LLVM commits to see what we can ascertain about this next generation of AMD accelerator.

LLVM currently has preliminary support for two new accelerators from AMD, under the codenames GFX1250 and GFX1251.

GFX1250 is an accelerator chip oriented at the machine learning market called MI455X and will be what powers the Helios rack. While the GFX1251 is oriented at the HPC market, named the MI430X and is projected by AMD to deliver over 200 TFLOPs of native double-precision compute.

We’ll focus on the GFX1250 for now though and save the GFX1251 for another day.

We’ll compare the architecture to the consumer variant, the previous generation of CDNA accelerators, and to some extent Nvidia’s Blackwell.

A curious note here is that while we know that these accelerators are not APUs, they are listed as such in LLVM.

Similarities to RDNA4

The biggest similarity with the consumer architectures is that we now have a WGP (Workgroup Processor) that are built up from two pairs of two SIMD32 arrays, but there is also four SIMDs per CU just like there was in GCN and CDNA. This seeming discrepancy is because there is no longer any need for the software to separate the two CUs in the WGP and AMD seems to use the terms interchangeably.

This “change” is probably not as structural as it might at first seem and might not even exist in hardware, the changes in the caching hierarchy simply makes the distinction pointless for the compiler as there is now a shared vector L0 cache for the whole WGP.

GFX1250 operates only in Wave32 mode, unlike RDNA that can run in Wave64 in addition to Wave32 mode. Prior CDNA GPUs can only run in Wave64 mode and we can expect to see some oddness there from a performance engineering point of view when porting things to the new accelerators and a lot of kernels will have to be re-evaluated for the new architecture.

It also seems like GFX1250 can run 20 waves per SIMD, which is four more than RDNA4 can do.

We do not know if any of these characteristics are a sign for where AMD goes on their GPUs, but I personally guess that it is a sign of times to come and a future generation of AMD GPUs will be locked to Wave32 and the “WGP” as it currently exist goes the way of the dodo but it seems like this is not happening anytime soon.

Similarly to RDNA4 we now get “WMMA” support, unlike the older “MFMA” featured in CDNA and this is where most of the machine learning capabilities of the GFX1250 will come from and we’ll look into it more later.

The dynamic VGPR allocation feature that was heavily touted in RDNA4 is missing, which is a bit strange to me since I would imagine it is even more important for machine learning workloads that are under much higher register pressure due to working with whole tensors instead of just individual vectors or matrices. The mode-switch is still there but the instruction to allocate new registers is documented as a no-op.

A Change in (C)DNA

One of the largest changes found in GFX125X is that each wave in a SIMD can now address up to 1024 Vector General Purpose Registers (VGPRs) from the Register File. This is a major improvement over prior CDNA generations where you had a split allocation of registers with up to 256 VGPRs and 256 Accumulation VGPRs, which can only be used by the matrix units, for a total of up to 512 registers allocated to a single wave at a time.

This is an even larger improvement over the register allocation behavior of the RDNA series of GPUs, RDNA can only allocate up to 256 registers to a single Wave with no allocation split. This is a major update that might at some point be part of a future generation GPU IP but I wouldn’t bet on it coming soon. We will just have to wait to see if the feature is going to stay locked to the Instinct line or if this ability to allocate up to 1024 registers to a single wave will make an appearance in the Radeon line up as well.

The maximum addressable local memory (LDS) is now 320kB per wavefront which is twice as much as CDNA4 and dwarfs the 64kB addressable in RDNA. But the actual LDS can actually be even larger, a major change in GFX125X is the merging of the LDS and vector L0 caches into what AMD is calling the WGP Cache (WGP$). While both nVidia and Intel have had a single structure for both cache and shared memory for a long time, AMD has been the odd vendor out with a split cache and LDS. GFX1250 now has a single 448KB structure that can be broken up into different allocations depending on what the compiler or programmer requests.

GFX1250 still supports packed fp32 operations allowing a single lane in the SIMD to process two fp32 operations at once just like CDNA2 and later, this practically means that when using packed instructions it has twice the vector width of RDNA.

It also inherits the more numerous and powerful SDMA units from CDNA.

Graphics Support

It is notable that nVidia and Intel do have some vestigial graphics hardware left on their accelerators and GFX125x differs a little from their offerings on this front.

No export instructions and therefore no rasterizer (like CDNA)

No image (texture) instructions (like CDNA)

No BVH (raytracing) instructions (like CDNA)

No vector parameter interpolation instructions that are used to access and interpolate vertex data in pixel shaders (like CDNA)

No MTBUF (typed buffer) instructions are supported (unlike CDNA and RDNA3)

No MUBUF (untyped buffer) instructions are supported (unlike CDNA and RDNA)

No LDS parameter and parameter direct loads (like CDNA)

We’re seeing that this GFX125x is even more of a pure compute accelerator than even prior CDNA architectures with nearly all graphics features having been removed. This probably saves a little bit of die space and removes features no one will miss, so it makes a lot of sense to me.

Tensor Operations

The headline feature of the MI455X is its AI capabilities, and the tensor units are what powers that. This seems to be a best of both worlds situation where AMD is combining the simpler programming model from RDNA4 with the higher performance and capabilities of CDNA4.

On RDNA4 WMMA operations are M = N = K = 16 (matrices are 16x16) except for the 4-bit ones that have K = 32 (arguments are 16x32 or 32x16 and accumulated into 16x16).

GFX1250 keeps M = N = 16, but K instead is the same as it was on CDNA4. K = 4 for 64-bit and 32-bit floating-point elements, K = 32 for floating-point 16-bit elements, K = 64 for fixed-point 8-bit elements and K = 128 for floating-point 8-bit elements. Interestingly the i4 operations stay at RDNA4 level at K=32 and are no longer a very popular data format in large AI inference scenarios due to the adoption of MX floating point operations.

Unlike on RDNA4 the sparse versions of WMMA seem to do less work per instruction than the dense ones by keeping K the same.

RDNA4 supports the following matrix of formats for WMMA

While CDNA4 supports the following for MFMA

And GFX1250 seems to support all the datatypes that are found on both CDNA4 and RDNA4 except for fp64 which seems to be restricted to GFX1251, including the support of OCP MX-style scaling that was previously available solely on CDNA4.

In addition there’s new instructions that allow you to transpose submatrices in LDS to complement the ones already in RNDA4. The fp4/fp6 versions of these have their own feature flag, that might be a sign that these will show up on consumer GPUs or it is just a coincidence.

It’s HIP to be … CUDA compatible?

Clusters

This seems to be AMD’s version of “Thread Block Clusters” that nVidia introduced in their Hopper architecture, and it seems like they allow you to schedule a cluster to a specific shader engine and seems to be a feature t