← Key papers

Paper review · November 21, 2026

Probing 'conscious' access in VLMs: Selective access and a global workspace in vision-language models

Probing the "Conscious" Mind: Selective Access and Global Workspace in Vision-Language Models

Safety & alignmentMultimodal Advanced Upvotes 34 cs.AI University College London

The authors built MC-Lens, which reads the intermediate states of the vision encoder and language decoder in the same vocabulary space, and showed that concepts inside a VLM are selectively accessed depending on the task and are actually used in reasoning.

Problem

A VLM's outputs alone cannot reveal where and how concepts are represented inside the model. The existing J-Lens was designed for language models, so it is hard to apply directly to VLMs, where image tokens far outnumber text tokens.

Method

The paper proposes MC-Lens, which extends Jacobian-based lenses to VLMs. V-Lens maps intermediate states of the vision encoder, and L-Lens maps intermediate states of the language decoder, into the final decoder state space using an average Jacobian, then reads them out in the output vocabulary. L-Lens balances the weights of image, text and template tokens and restricts output positions to text positions. It also uses reference centring, which subtracts components that appear commonly across images. Concepts are represented as sets of tokens instead of a single name token for counterfactual intervention experiments. The authors also created two J-space metrics to track cross-modal alignment during controlled VLM training.

Results

Limitations

The authors state explicitly that 'conscious' refers only to functional properties from global workspace theory and that they do not claim the models are conscious. The lens is a first-order linear approximation based on an average Jacobian, so context-specific usage is averaged out. The provided text cuts off partway through the method description, so quantitative figures and experimental scale could not be checked.

Why it matters

It is an interpretability tool for examining why a VLM gives wrong answers even when it internally holds the information it saw in the image. It can also serve as a metric for monitoring the progress of cross-modal alignment training.

Key terms

Global workspace theory (GWT)
A theory of consciousness holding that information from the brain's many parallel processes is selected and integrated into a shared workspace, where it becomes available to other specialized processes.
J-Lens(Jacobian Lens)
An interpretability technique that approximates the effect of intermediate-layer Hidden States on the final-layer state with an average Jacobian, so that intermediate states can be read out as vocabulary distributions.
Selective access
The phenomenon in which, for the same image, different concepts stand out in the model's internals depending on the question (e.g., object name vs. color).
Counterfactual intervention
An experiment that swaps in a specific concept direction in an intermediate state and observes the change in output, to check whether that concept is actually used in computation.

Original

Authors: Jiahao Huang, Jun Wang · Posted 2026-11-19 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.