Paper review · October 7, 2026
LMBuild: Evaluating LLM agents that build structures that can actually be assembled and work
LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures
The authors built an interactive environment for building structures, including parts, joints, materials and assembly order, along with a four-level evaluation framework, and evaluated 30 systems. They found that function and physical operability are the biggest weaknesses.
Problem
LLM agents can now build complex 3D structures, but existing evaluations focus on geometric and visual quality and semantic alignment. Whether the result can actually be assembled, performs its intended function and can be built in the physical world has not been systematically evaluated.
Method
It provides three things. ① Interactive environment: the agent uses tools to retrieve or newly create parts, place and modify them, and specify joints, materials and assembly order. ② Benchmark: 2,549 objects were selected from about 368K existing 3D and CAD assets (BrickNet, PartNeXt, Fusion 360 Gallery, Artiverse, 27 open-source hardware projects, etc.) to form the Full set, and 200 of them were designated the Core set. Each object was enriched with functional-part and kinematics information using Wikipedia knowledge. ③ Evaluation: it scores four dimensions—structural soundness (S), functional affordance (A), design quality (D) and physical realization (R)—and 12 sub-metrics on a 0–100 scale. Variant settings such as retrieval-only, generation-only, multi-round revision and detailed-description prompts are also evaluated.
Results
- On the Core set, commercial models scored high on the structural-soundness sub-metric S.2, at 90.3–97.4. By contrast, on A.1, which concerns functional parts, GPT-6 Astra scored highest at 63.0, while Claude Haiku 4.5 reached only 36.7.
- The physical-realization sub-metric R.3 was the lowest even for commercial models, at 11.0–22.1 (Claude Opus 5 22.1, GPT-6 Astra 19.2). The open model Qwen3.5-27B scored 10.5.
- The authors reported that stronger models make better use of new-part generation, while weaker models rely on retrieval and find the generation capability more of a burden than a help.
- The authors reported that adding functional specifications to the prompt substantially improves functional-part completeness, kinematic accuracy and physical operability.
Limitations
Because of cost, all experiments were run only on the 200-object Core set. Evaluated objects were limited to at most one per object category. Domain-specific systems that output only geometry are evaluated by estimating joint information with Particulate, so results may differ from their original capabilities. Functional and kinematic criteria rely on textual knowledge enriched from Wikipedia, and operability is assessed through simulation.
Why it matters
It broadens 3D generation evaluation from 'good-looking shapes' to 'objects that can be built and work.' By showing in numbers that functional reasoning and new-part design are the next bottlenecks, it can serve as a reference point for setting the direction of agents that automate design and manufacturing.
Key terms
- Functional affordance
- Whether an object has the parts, joints and motion it needs to perform its intended function.
- Kinematics
- A concept that deals with the directions and ranges in which parts connected by joints move.
- Part retrieval vs. generation
- The approach of choosing a suitable part from a given part pool, versus the approach in which the agent itself creates a new part with the required shape.
Original
Authors: Jiateng Liu, Rushi Wang, Cheng Qian, Xuejun Zhang, Sun Li, Jiayu Liu, Yifan Shen, Xu Cao, et al. · Posted 2026-10-03 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.