Paper review · November 21, 2026
Probing 'conscious' access in VLMs: Selective access and a global workspace in vision-language models
Probing the "Conscious" Mind: Selective Access and Global Workspace in Vision-Language Models
The authors built MC-Lens, which reads the intermediate states of the vision encoder and language decoder in the same vocabulary space, and showed that concepts inside a VLM are selectively accessed depending on the task and are actually used in reasoning.
Problem
A VLM's outputs alone cannot reveal where and how concepts are represented inside the model. The existing J-Lens was designed for language models, so it is hard to apply directly to VLMs, where image tokens far outnumber text tokens.
Method
The paper proposes MC-Lens, which extends Jacobian-based lenses to VLMs. V-Lens maps intermediate states of the vision encoder, and L-Lens maps intermediate states of the language decoder, into the final decoder state space using an average Jacobian, then reads them out in the output vocabulary. L-Lens balances the weights of image, text and template tokens and restricts output positions to text positions. It also uses reference centring, which subtracts components that appear commonly across images. Concepts are represented as sets of tokens instead of a single name token for counterfactual intervention experiments. The authors also created two J-space metrics to track cross-modal alignment during controlled VLM training.
Results
- The authors reported that multimodal concept information can be read from intermediate states and is selectively accessed depending on the question. Patch-level readouts revealed not only object identity but also attributes such as color and material.
- In causal reasoning VQA, intervening on intermediate concept representations changed the answers to multi-step reasoning. The authors reported that this effect generally grew stronger as the set of concept directions was enlarged.
- In controlled alignment training, the proposed J-space metrics showed the development of cross-modal alignment more clearly than a Hidden State cosine baseline. Concept accessibility improved early in training, but stable selective access emerged later.
- Internal organization differed across model families and sizes, but common patterns appeared: concept accessibility, query-dependent selectivity and cross-task intervention effects.
Limitations
The authors state explicitly that 'conscious' refers only to functional properties from global workspace theory and that they do not claim the models are conscious. The lens is a first-order linear approximation based on an average Jacobian, so context-specific usage is averaged out. The provided text cuts off partway through the method description, so quantitative figures and experimental scale could not be checked.
Why it matters
It is an interpretability tool for examining why a VLM gives wrong answers even when it internally holds the information it saw in the image. It can also serve as a metric for monitoring the progress of cross-modal alignment training.
Key terms
- Global workspace theory (GWT)
- A theory of consciousness holding that information from the brain's many parallel processes is selected and integrated into a shared workspace, where it becomes available to other specialized processes.
- J-Lens(Jacobian Lens)
- An interpretability technique that approximates the effect of intermediate-layer Hidden States on the final-layer state with an average Jacobian, so that intermediate states can be read out as vocabulary distributions.
- Selective access
- The phenomenon in which, for the same image, different concepts stand out in the model's internals depending on the question (e.g., object name vs. color).
- Counterfactual intervention
- An experiment that swaps in a specific concept direction in an intermediate state and observes the change in output, to check whether that concept is actually used in computation.
Original
Authors: Jiahao Huang, Jun Wang · Posted 2026-11-19 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.