ITNet: A Learnable Integral Transform That Subsumes Convolution, Attention, and Recurrence

Researchers have introduced ITNet, a unified neural network architecture that mathematically subsumes convolution, self-attention, and recurrence under a single learnable integral transform, matching or exceeding specialized baselines across multiple modalities.

Computer Science > Artificial Intelligence

Title:ITNet: A Learnable Integral Transform That Subsumes Convolution, Attention, and Recurrence

View PDF HTML (experimental)Abstract:Convolutional networks, recurrent networks, and transformers each encode different inductive biases -- locality, sequential memory, and content-dependent pairwise interaction -- and have remained mathematically distinct since their inception. We show that this fragmentation reflects not a fundamental diversity in how signals should be processed, but rather incomplete views of a single underlying mathematical object: a learnable integral transform. We introduce the Integral Transform Network (ITNet), a unified architecture built around a learnable kernel that depends jointly on positions and features. This kernel is implemented as a small neural network, specifically an MLP, that models pairwise interactions, enabling the model to adapt its behavior from data. We show that convolution, self-attention (including multi-head), and autoregressive recurrence (including LSTM, GRU, S4, and Mamba) arise as special cases under appropriate parameterizations, and that ITNet is a universal approximator of continuous operators. To make this practical, we develop tiled kernel fusion, importance-weighted Monte Carlo integration, and learned low-rank factorization, enabling efficient and scalable computation. A single ITNet architecture with a shared operator and lightweight modality-specific encoders matches or exceeds specialized baselines on ImageNet-1K , GLUE, ModelNet40, VQA,v2 and NLVR2. The results demonstrate that a single learned interaction mechanism can recover the behavior of all three architectural families from data.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Analyzing the Narration Gap in LLM-Solver Loops

A new study highlights the 'narration gap' in hybrid LLM-solver systems, revealing that while the formal solver produces sound results, adversaries can still manipulate the LLM to invert the final answer via prompt injection.

agentic-systems

Uncertainty Decomposition for Clarification Seeking in LLM Agents

A new study proposes a prompt-based uncertainty decomposition method that enables LLM agents to detect ambiguous user requests and proactively seek clarification. This approach significantly outperforms existing methods across multiple large language models, including GPT-5.1 and DeepSeek-v3.2.

NOW LET US Related – LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data

agentic-systems

LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data

A recent study reveals that Large Language Models (LLMs) struggle to recognize their own knowledge limits when processing structured clinical tabular data. By comparing Qwen 2.5 7B with XGBoost, researchers identified critical epistemic blind spots and proposed a cross-model calibration method to address this limitation.

agentic-systems

Deontic Policies for Runtime Governance of Agentic AI Systems

Autonomous agentic AI systems introduce novel security and compliance challenges that exceed the capabilities of current policy engines. To address this, researchers propose AgenticRei, a runtime governance framework utilizing deontic policies to strictly control AI behavior outside the LLM.

agentic-systems

AI4SE and SE4AI Exploration: A Decade Looking Back and Forward

A new study traces a decade of progress in the intersection of AI and Systems Engineering (SE) across three developmental phases. By identifying five critical research gaps, the authors provide a roadmap for AI adoption, assurance, and workforce transformation in the field.

agentic-systems

Emergent Alignment

Researchers have introduced "Emergent Alignment," a method enabling Large Language Models (LLMs) to detect and self-correct their own unethical outputs. By integrating a "conscience step" and DPO optimization, this technique helps AI maintain ethical standards without relying on external judge models.

EXPLORE TOPICS