Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation

Developing high-performance NPU kernels is a major bottleneck due to strict hardware constraints. Hawk, a training-free framework, addresses this by leveraging hardware-aware knowledge, boosting generation accuracy to 80% and achieving up to a 2.2x speedup.

Computer Science > Artificial Intelligence

Title:Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation

View PDF HTML (experimental)Abstract:Developing high-performance kernels for Neural Processing Units (NPUs) is a critical industry bottleneck, requiring developers to manually navigate implicit hardware constraints and strict memory hierarchies. While large language models offer immense automation potential, they fail catastrophically on NPUs due to a fundamental lack of hardware-specific priors. Naively transplanting code snippets from similar NPU kernels may pass the compiler, but it consistently triggers runtime crashes and performance degradation by blindly violating underlying hardware constraints. To overcome this, we introduce Hawk, a training-free framework that harnesses hardware-aware knowledge through three core modules: (1) Run-Time Knowledge Synthesis Module, which employs a Triple-Part Executable Knowledge Representation to inherently couple the error context with executable semantics; (2) Bottleneck-Aware Knowledge Retrieval Module, which implements a 2D-Retrieval paradigm to project queries into orthogonal syntactic and hardware-aligned semantic spaces; and (3) Effect-Driven Knowledge Distillation Module, which leverages LLM-driven semantic arbitration to continuously distill the knowledge by pruning errors and consolidating redundancies based on the empirical execution feedback. Extensive evaluations on real-world NPU workloads demonstrate that Hawk elevates generation accuracy from 49.4% to 80.0%, while achieving up to a 2.2x execution speedup over state-of-the-art baselines.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

SemHash-LLM: A Multi-Granularity Semantic Hashing Framework for Document Deduplication

Researchers have introduced SemHash-LLM, a multi-granularity semantic hashing framework designed for efficient large-scale document deduplication. By combining LLMs with advanced hashing techniques, it reduces neural verification costs to under 1% while maintaining high accuracy.

agentic-systems

Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows

Researchers demonstrate how Reinforcement Learning with Verifiable Rewards (RLVR) can bridge the gap between next-token prediction and complex API execution, significantly boosting the performance of small language models on Atlassian workflows.

agentic-systems

Discrete Diffusion Language Models for Interactive Radiology Report Drafting

Researchers have adapted a mixture-of-experts diffusion language model for medical applications, matching or exceeding traditional autoregressive models while decoding 3.5 to 4.4 times faster and enabling flexible, non-linear report drafting.

agentic-systems

Profit-Based Counterfactual Explanations for Product Improvement: A Case Study of Manga Sales in Japan

Researchers propose a novel Profit-Based Counterfactual Explanation (PBCE) framework that integrates machine learning with business profit maximization, demonstrated through a case study on Japanese manga sales.

agentic-systems

Safe and Adaptive Cloud Healing: Verifying LLM-Generated Recovery Plans with a Neural-Symbolic World Model

Researchers have introduced PASE, a breakthrough cloud self-healing framework that combines Large Language Models (LLMs) with a Neural-Symbolic World Model. This novel approach optimizes autonomous recovery planning, reducing system downtime by over 40% compared to state-of-the-art methods.

agentic-systems

Scaling Trends for Lie Detector Oversight in Preference Learning

A new study evaluates Scalable Oversight via Lie Detectors (SOLiD) on larger LLMs, showing that scaling reduces undetected deception to 14% and can eliminate the need for expensive human labelers during fine-tuning.

EXPLORE TOPICS