AGENTIC-SYSTEMSApril 3, 20261 min read15 views

Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

Researchers propose the 'Compress-Add-Smooth' (CAS) framework, enabling AI agents to achieve continual learning without neural networks or backpropagation, optimized for resource-constrained hardware.

Computer Science > Machine Learning

Title: Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

An agent that operates sequentially must incorporate new experience without forgetting old experience, under a fixed memory budget. We propose a framework in which memory is not a parameter vector but a stochastic process: a Bridge Diffusion on a replay interval [0,1], whose terminal marginal encodes the present and whose intermediate marginals encode the past. New experience is incorporated via a three-step Compress–Add–Smooth (CAS) recursion.

We test the framework on the class of models with marginal probability densities modeled via Gaussian mixtures of fixed number of components K in d dimensions; temporal complexity is controlled by a fixed number L of piecewise-linear protocol segments whose nodes store Gaussian-mixture states. The entire recursion costs O(LKd^2) flops per day -- no backpropagation, no stored data, no neural networks -- making it viable for controller-light hardware.

Forgetting in this framework arises not from parameter interference but from lossy temporal compression: the re-approximation of a finer protocol by a coarser one under a fixed segment budget. We find that the retention half-life scales linearly as a_1/2 ≈ cL with a constant c > 1 that depends on the dynamics but not on the mixture complexity K, the dimension d, or the geometry of the target family. The constant c admits an information-theoretic interpretation analogous to the Shannon channel capacity.

The stochastic process underlying the bridge provides temporally coherent “movie” replay -- compressed narratives of the agent's history, demonstrated visually on an MNIST latent-space illustration. The framework provides a fully analytical “Ising model” of continual learning in which the mechanism, rate, and form of forgetting can be studied with mathematical precision.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Semi-Supervised Text-Attributed Graph Distillation

Researchers propose a unified semi-supervised framework guided by Wasserstein Distance to tackle scalability bottlenecks in Text-Attributed Graphs (TAGs). The approach achieves a state-of-the-art trade-off between performance and data compression for both GNN- and LLM-based downstream tasks.

agentic-systems

Benchmarking the Personalization Capabilities of Large Language Models

A new study introduces SDR-Bench and SDR-Arena to benchmark the personalization capabilities of LLMs in two-party persuasion scenarios, revealing a personalization plateau among frontier models.

agentic-systems

SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification

Researchers introduced SonicSampler, a unified suite of tile-aware Triton kernels designed to accelerate LLM sampling and speculative verification. By fusing the entire pipeline into a single CUDA Graph-compatible kernel, SonicSampler achieves up to 16x speedups over state-of-the-art baselines.

agentic-systems

Enabling Scalable Topology Inference in Distribution Systems via Constrained Multi-Source Inference

A new research study introduces a constrained multi-source inference framework to accurately reconstruct power distribution system topology. Validated on over 8,000 smart meters in the U.S., the method achieves over 95% accuracy while drastically reducing computational overhead.

agentic-systems

Incomplete Prompt Jailbreaks in Large Language Models

Researchers have conceptualized Incomplete Prompt Jailbreaks (IPJ), a vulnerability where incomplete harmful prompts bypass LLM safeguards due to delayed refusal behaviors. The discovery of functional neurons offers a novel path for precise neuron-level defense interventions.

agentic-systems

Robust Critics: Defending LLMs Against Multi-Turn Attacks

Researchers introduce Dialogue Critic Guided Sampling (DCGS), a new framework designed to defend Large Language Models against complex, multi-turn adversarial attacks by inferring user intent throughout the conversation history.