AGENTIC-SYSTEMSMarch 20, 20261 min read25 views

Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism

Li-Net is a novel architecture for multi-channel time series forecasting that leverages a sparse Top-K Softmax attention mechanism and multi-modal fusion to achieve superior accuracy and computational efficiency.

Computer Science > Artificial Intelligence

Title:Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism

Abstract:The task of multi-channel time series forecasting is ubiquitous in numerous fields such as finance, supply chain management, and energy planning. It is critical to effectively capture complex dynamic dependencies within and between channels for accurate predictions. However, traditional method paid few attentions on learning the interaction among channels. This paper proposes Linear-Network (Li-Net), a novel architecture designed for multi-channel time series forecasting that captures the linear and non-linear dependencies among channels. Li-Net dynamically compresses representations across sequence and channel dimensions, processes the information through a configurable non-linear module and subsequently reconstructs the forecasts. Moreover, Li-Net integrates a sparse Top-K Softmax attention mechanism within a multi-scale projection framework to address these challenges. A core innovation is its ability to seamlessly incorporate and fuse multi-modal embeddings, guiding the sparse attention process to focus on the most informative time steps and feature channels. Through the experiment results on multiple real-world benchmark datasets demonstrate that Li-Net achieves competitive performance compared to state-of-the-art baseline methods. Furthermore, Li-Net provides a superior balance between prediction accuracy and computational burden, exhibiting significantly lower memory usage and faster inference times. Detailed ablation studies and parameter sensitivity analyses validate the effectiveness of each key component in our proposed architecture.

Keywords: Multivariate Time Series Forecasting, Sparse Attention Mechanism, Multimodal Information Fusion, Non-linear relationship

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Generative Ontology Induction: Domain-Agnostic Schema Discovery from Document Corpora Using Large Language Models

Researchers introduce Generative Ontology Induction (GOI), a domain-agnostic framework that automatically extracts structured ontologies from document corpora using LLMs. Achieving 95-100% structural coverage, GOI addresses a major bottleneck in knowledge-intensive AI systems.

agentic-systems

JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models

Researchers have proposed JUMP, a novel single-pass membership inference attack designed for fine-tuned discrete diffusion language models (dLLMs). By leveraging the unique properties of dLLMs, JUMP significantly improves detection accuracy while drastically reducing the number of required queries.

agentic-systems

Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

Researchers demonstrate that Masked Diffusion Language Models (MDLMs) serve as highly effective, steerable text-based world models for agentic reinforcement learning. By leveraging bidirectional denoising, MDLMs outperform autoregressive models four times their size in coherence, groundedness, and rollout diversity.

NOW LET US Related – Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment

agentic-systems

Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment

A new study demonstrates that small language models (SLMs) under 3 billion parameters can serve as highly capable local experts for specialized tasks. By combining structured benchmarking with low-cost parameter-efficient fine-tuning (PEFT), institutions can achieve AI autonomy without relying on expensive hardware.

agentic-systems

PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection

Researchers have introduced PlanFlip, a novel prompt injection attack framework targeting the planning phase of multi-agent LLM systems. The study reveals critical security blind spots in homogeneous agent pipelines and demonstrates that reasoning-augmented models like DeepSeek-R1 exhibit strong resistance.

agentic-systems

Rater State Bias in RLHF Preference Data: An Audit Framework

A new study identifies a structured bias in Reinforcement Learning from Human Feedback (RLHF) caused by the psychological state of human raters. Under stress, raters' shifting preferences can propagate through reward modeling, potentially compromising AI policy optimization.