Hardware Image Compression

One of the things I’ve always lamented about hardware image formats is the slow pace of innovation. Developers were usually unwilling to ship textures in a new format unless that format was widely available. That is, the format had to be supported in the majority of the hardware they were targeting, and must be supported across all vendors.

For example, even though ATI introduced the 3Dc formats in 2004 with the Radeon X800 (R420) and exposed them through D3D9 extensions, in practice their use did not become widespread when Direct3D 10 standardized them as BC4 and BC5 in 2007, but only when Direct3D 10 became the minimum hardware requirement.

Crysis was the first major game which shipped with BC5 textures, but most games were not willing to have such a steep hardware requirement until many years later. To avoid these adoption delays, the BC6 & BC7 formats were designed in collaboration between ATI and NVIDIA for inclusion in Direct3D 11.

Hardware development cycles are already long, and for a new format to gain adoption it needs to be proposed for standardization, which often makes the process even longer.

This is one of the reasons why I find real-time texture compression so exciting. When the encoder runs in real-time it’s a lot easier to introduce new hardware formats, because adopting a new format no longer requires waiting for content to be created targeting it.

In a previous post I mentioned hardware compression as an alternative to real-time compression. The details of these formats are not documented anywhere and their use is completely transparent, applications do not need to target these formats explicitly, instead the driver compresses textures dynamically during rendering and image uploads.

Today, there are three competing hardware image compression formats: ARM’s AFRC, ImgTec’s PVRIC4, and Apple’s ‘lossy’ (for lack of a better name). In this post I’ll take a closer look at how these formats are used, what quality we can expect from them, and how they perform compared with Spark, my real-time texture compression library.

Let’s start with Apple’s implementation.

Metal

Apple introduced lossy texture compression in the A15 and M2 chipsets (which share the same GPU generation). Enabling it results in a 1:2 compression ratio.

Metal’s lossy compression is remarkably easy to opt into. The API surface is minimal, the compressionType property on MTLTextureDescriptor takes a value from the MTLTextureCompressionType enum, and setting it to MTLTextureCompressionTypeLossy is often the only required change.

MTLTextureDescriptor *descriptor = [MTLTextureDescriptor
texture2DDescriptorWithPixelFormat:MTLPixelFormatRGBA8Unorm
width:width
height:height];
descriptor.usage = MTLTextureUsageRenderTarget | MTLTextureUsageShaderRead;
descriptor.storageMode = MTLStorageModePrivate;
descriptor.compressionType = MTLTextureCompressionTypeLossy;
id<MTLTexture> texture = [device newTextureWithDescriptor:descriptor];

The Metal Feature Set Tables indicate that all ordinary pixel formats support lossy compression, this includes 10 bit and floating point formats, which I think is quite remarkable. I ran some tests and can confirm that this is indeed the case, but so far I’ve focused my tests on R8, RG8 and RGBA8 formats.

In terms of quality the R and RG formats perform better than the Spark EAC codecs, but worse than the BC4 and BC5 codecs.

In terms of performance the lossy formats perform very well and tend to saturate memory bandwidth if the size of the texture is large enough. I ran some tests on my M4 Pro (16 GPU core). The throughput of the standard blits remains fairly consistent regardless of the texture size. On the other hand, the Spark codecs appear to have a fixed overhead that becomes more significant as the texture sizes decrease.

Vulkan

On Vulkan, the VK_EXT_image_compression_control extension gives applications a way to request fixed-rate compression for images. This extension is already available on flagship devices from Arm and Imagination.

As you would expect enabling lossy image compression in Vulkan is a bit more verbose than in Metal, but in practice, not much more complicated. The only thing we need to do is to extend the VkImageCreateInfo structure by chaining the VkImageCompressionControlEXT structure to it.

We can use the VK_IMAGE_COMPRESSION_FIXED_RATE_DEFAULT_EXT flag to let the implementation choose any fixed rate compression setting. BPC stands for “bits per component” which is a bit unusual, but is intended to allow you to specify the compression ratio in a uniform way regardless of the number of channels.