AGENTIC-SYSTEMSJuly 28, 20261 min read

Concept-based Visual Counterfactual Explanations with Diffusion Models

Researchers introduced C-VCE, a novel diffusion framework that generates human-interpretable visual counterfactual explanations by building concept bottleneck layers directly into the generative model.

Computer Science > Artificial Intelligence

Title:Concept-based Visual Counterfactual Explanations with Diffusion Models

View PDF HTML (experimental)Abstract:Visual counterfactual explanations aim to answer "what minimal change to this image would flip the model's prediction?", and are increasingly important as vision models are deployed in safety-critical domains (e.g., medicine). Existing diffusion-based methods can produce realistic edits, but they rely on external classifiers that must work reliably on noisy images, which makes them fragile and hard to deploy for robust explanations. We introduce C-VCE, a new diffusion framework that builds the classifier directly into the generative model via a concept bottleneck layer, so that counterfactuals are guided by human-interpretable features (concepts) instead of a separate noise robust classifier that works with pixel-level edits. Our model lets users to toggle on/off semantic concepts during sampling, then minimally adjusts relevant image regions, while preserving the rest of the image, respecting feature correlations. To keep edits small and controlled, we add a simple probabilistic regularizer that balances "change the prediction" against "stay close to the original", plus a gradient-based mask that confines modifications to the most relevant regions. On benchmarks such as CelebA, C-VCE matches or improves flip rates while producing counterfactuals that are visually closer to the input and less distorted than baselines that depend on separate noisy-image classifiers. These properties make C-VCE a practical tool for vision systems where users need concrete "what-if" images without having to trust an additional, noise-robust classifier. More broadly, our results suggest that exposing and controlling an internal concept layer is a promising way to make powerful generative models easier to understand and safer to use.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Schema-Aware Localisation (SAL): Live Schema Grounding and Hallucination Validation for Oracle NL2SQL

Researchers introduce Schema-Aware Localisation (SAL), a lightweight middleware for Oracle NL2SQL that connects LLMs to live database catalogs. SAL dramatically improves SQL execution accuracy from 2.2% to 62.6% while drastically reducing hallucination-induced execution failures without requiring model retraining.

agentic-systems

Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits

Researchers have proposed a loss-aware structured feature-map pruning framework for convolutional neural networks using multi-armed bandits. The approach significantly reduces computational complexity and storage costs while maintaining accuracy levels close to original unpruned models.

agentic-systems

SCAIR: Schema-Conditioned Agentic Iterative Reasoning for Enterprise Knowledge Graphs

Researchers have introduced SCAIR, a training-free agentic framework that enhances Knowledge Graph-based RAG (KG-RAG) for enterprise applications by injecting schema constraints into multi-hop iterative reasoning.

agentic-systems

Synthetic Scenario Generation for Evaluation of Industry 4.0 Agents

Researchers have developed ScenarioGeneratorAgent, an automated pipeline that generates synthetic evaluation scenarios for Industry 4.0 AI agents. Tested on Smart Grid Transformers, the approach accelerates benchmark generation by 8x while maintaining high quality and adherence to IEC standards.

agentic-systems

Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

As LLMs tackle longer tasks, retaining complete interaction histories becomes contextually and computationally expensive. The ABBEL framework addresses this by isolating summaries into natural-language 'belief states' and supervising them via belief grading, recovering accuracy while significantly cutting memory usage.

agentic-systems

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

Researchers have introduced DC-Leap, a training-free framework that accelerates inference in Diffusion Large Language Models (dLLMs) by up to 105x when combined with KV-Cache while maintaining generation quality.