AGENTIC-SYSTEMSMay 28, 20261 min read6 views

Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems

Researchers have proposed a breakthrough multi-agent architecture that shifts data analytics from reactive to proactive. By combining LLMs, Apache Kafka, and Apache Flink, the system autonomously generates hypotheses and processes real-time data streams to deliver valuable insights.

Computer Science > Artificial Intelligence

Title:Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems

View PDF HTML (experimental)Abstract:Modern analytics systems are fundamentally reactive, requiring users to define queries over increasingly complex and continuously evolving data. In real-time streaming environments, this paradigm breaks down, as the space of potential insights becomes too large to enumerate manually. We present a multi-agent architecture for autonomous insight discovery over real-time data streams. The system implements a continuous discovery loop in which agents generate hypotheses, compile them into executable analytics, validate generated artifacts, and produce visualizations and deployable applications. The architecture leverages Apache Kafka for event-driven coordination, Apache Flink for stream processing, and large language models to implement specialized agents. A key contribution is a contract-driven design based on typed intermediate artifacts, enabling modularity, observability, lineage, and safer execution of dynamically generated analytics. Through use cases in retail, finance, and public data, we show how this architecture supports a shift from query-driven analytics to proactive, discovery-driven systems.

Current browse context:

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

HG-RAG: Hierarchy-Guided Retrieval-Augmented Generation for Structured Knowledge Graphs

HG-RAG (Hierarchy-Guided Retrieval-Augmented Generation) is a novel framework designed to overcome the limitations of traditional flat RAG systems by traversing hierarchical knowledge graphs, significantly improving multi-hop reasoning and reducing hallucinations in LLMs.

agentic-systems

Enhancing Small Language Models Reasoning through Knowledge Graph Grounding

Researchers have proposed a neuro-symbolic framework that enhances the reasoning capabilities of Small Language Models (SLMs) like Gemma 3 and Llama 3.2 using Knowledge Graphs, achieving up to a 2x performance boost.

agentic-systems

Orchestrating Power Grid Studies with Multi-Agent AI and MCP Servers

This paper explores how Agentic AI and Model Context Protocol (MCP) can support power-grid studies by integrating LLMs with numerical simulation tools under human supervision.

agentic-systems

Human AI Construction of Bayesian Networks for Operational Decision Support -- A Virtual Survey Approach

Researchers propose a new methodology using Large Language Models and AI agents to construct Bayesian Belief Networks, bridging the gap between expert opinion and data-driven learning. The framework is demonstrated through a healthcare decision-making model, revealing key insights into customer behavior.

agentic-systems

Interpretable Language Model for Closed-Loop Type 1 Diabetes Control

Researchers have developed LLM-T1D, a novel approach combining reinforcement learning and large language models to automate insulin delivery for Type 1 Diabetes. The system not only outperforms traditional methods but also explains its decisions in plain language, addressing the 'black-box' challenge in medical AI.

agentic-systems

RegNetAgents: A Multi-Agent Framework for Cross-Network Regulatory Driver Identification in Cancer Genomics

Researchers have introduced RegNetAgents, an AI-oriented multi-agent framework designed to identify regulatory driver candidates across heterogeneous gene networks in cancer genomics. By integrating bulk tumor and single-cell data, the system offers a powerful tool for discovering therapeutic targets in breast and colorectal cancers.