Paper review · October 27, 2026
Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models
HAD-VLM combines autoregressive text generation and diffusion-based continuous numerical prediction in a single VLM, reducing echocardiography measurement error by 21.4–37.6% compared with existing VLMs.
Problem
Ultrasound measurements (distances, thicknesses and so on) are continuous values. VLMs, however, split numbers into token sequences and predict them through next-token classification, so they cannot account for the size of an error. Getting a single digit or decimal point wrong can change the value a lot. Existing measurement models handle only predefined tasks, which makes it hard to change the measurement target through a prompt.
Method
The model adapts CLIP ViT-L/14 to ultrasound with Visual Prompt Tuning and uses Vicuna-7B as the backbone. Each measurement is represented as a 5-tuple made up of 4 coordinates for the two caliper endpoints and 1 length. When the model outputs '[', it stops text generation, uses the LLM as the denoising backbone to predict the 5-tuple in 4 DDIM steps, and then resumes text generation. If the difference between the length computed from the coordinates and the directly predicted length is at or below a threshold set at the 85th percentile on the validation set, the two are averaged. The median of 20 sampled results is used as the final value.
Results
- On EchoNet-LVH, MAE was 1.36 mm for IVSd, 2.77 mm for LVIDd, 3.05 mm for LVIDs and 1.16 mm for LVPWd, reducing error by 21.4–37.6% compared with a Vicuna-based autoregressive VLM.
- Across all 4 tasks, MAE was lower than both a fine-tuned Qwen3-VL and VLM+MLP regression (e.g., LVIDd 2.77 mm vs. 3.00 mm for MLP regression and 3.71 mm for Qwen3-VL).
- LVIDd and LVIDs MAE were numerically lower than the inter-observer variability reported in other studies (2.97 and 3.60 mm).
- On X-ray spine measurement, MAE dropped from 11.4 pixels to 6.7 pixels, and question-answering performance was maintained on the medical QA dataset evaluated.
Limitations
The ultrasound evaluation is limited to 4 measurements in a single dataset, EchoNet-LVH. The inter-observer comparison uses figures from separate studies. Preservation of medical QA performance was also confirmed on only one dataset. The need for 20 samples and threshold calibration adds inference cost and dependence on settings.
Why it matters
This is an attempt to let users request measurements in natural language while still achieving the numerical precision clinical use requires. Using a continuous prediction head to make up for VLMs treating numbers as tokens could also apply to quantitative measurement tasks outside medicine.
Key terms
- Hybrid autoregressive-diffusion model
- An architecture that generates text through token-by-token autoregression and continuous values through a diffusion (denoising) process, both from the same backbone.
- 5-tuple measurement representation
- A representation that bundles the caliper's start and end coordinates and the length into a single continuous vector. It makes the model predict position and value together.
- Coordinate-length consistency
- A post-processing rule: if the length computed from the endpoint coordinates is close to the directly predicted length, the two are averaged; if they differ a lot, the directly predicted value is used.
- Visual Prompt Tuning(VPT)
- A method that adapts a pretrained vision encoder to a new domain by keeping the encoder frozen and training only added, learnable prompt tokens.
Original
Authors: Dianye Huang, Jiamei Guo, Yue Zhou, Mingde Li, Yi Xiong, Wei Wang, Nassir Navab, Zhongliang Jiang · Posted 2026-10-23 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.