Easily Build and Share ROCm Kernels with Hugging Face

Easily Build and Share ROCm Kernels with Hugging Face

Custom kernels are the backbone of high-performance deep learning, enabling GPU operations tailored precisely to your workload; whether that’s image processing, tensor transformations, or other compute-heavy tasks. But compiling these kernels for the right architectures, wiring all the build flags, and integrating them cleanly into PyTorch extensions can quickly become a mess of CMake/Nix, compiler errors, and ABI issues, which is not fun. Hugging Face’s kernels library makes it easy to build (with kernel-builder) and share these kernels with the kernels-community, with support for multiple GPU and accelerator backends, including CUDA, ROCm, Metal, and XPU. This ensures your kernels are fast, portable, and seamlessly integrated with PyTorch.

In this guide, we focus exclusively on ROCm-compatible kernels and show how to build, test, and share them using kernels. You’ll learn how to create kernels that run efficiently on AMD GPUs, along with best practices for reproducibility, packaging, and deployment.

The RadeonFlow GEMM kernel is a high-performance, FP8 block-wise matrix multiplication implementation optimized for the AMD Instinct MI300X GPU. GEMM (General Matrix Multiplication) is the core building block behind most deep learning workloads. This kernel was developed for the AMD Developer Challenge 2025, it was awarded the 🏆 Grand Prize in June 2025, recognizing its excellence in performance and innovation on AMD hardware.

The kernel operates on quantized inputs using the e4m3fnuz floating-point format and applies per-block scaling to preserve accuracy during low-precision computation. The e4m3fnuz format is an FP8 variant with 4 exponent bits and 3 mantissa bits, designed to be efficient for neural network workloads. Because FP8 has a much smaller dynamic range than FP16/FP32, we apply per-block scaling factors (a_scale and b_scale) so that each block of values is rescaled into a numerically “comfortable” range before and after computation, which helps preserve accuracy despite the low precision.

The Hugging Face Kernel Builder expects your files to be organized like this:

gemm/
├── build.toml
├── gemm/
│   └── gemm_kernel.h
├── flake.nix
└── torch-ext/
    ├── torch_binding.cpp
    ├── torch_binding.h
    └── gemm/
        └── __init__.py

build.toml: The project manifest; it’s the brain of the build process. gemm/: Your raw CUDA/HIP source code where the GPU magic happens. flake.nix: The key to a perfectly reproducible build environment. torch-ext/gemm/: The Python wrapper for the raw PyTorch operators.

This file orchestrates the entire build. It tells the kernel-builder what to compile and how everything connects. For ROCm, you specify the backend and the target architectures (e.g., "gfx942" for MI300 series).