Scene Layout via Conceptual Design

01Abstract

We introduce a novel approach for designing 3D scene layouts, focusing on a reason-driven generation from text to conceptual design. Our Iterated Design System (IDS) offers a hierarchical structure that incorporates both geometric and semantic information in the scene, reflecting the intricate process of human design reasoning. We leverage Large Language Models (LLMs) to break down the complex task of conceptual design into more manageable components, mirroring human cognitive methods. Additionally, we propose an LLM-in-the-loop optimization strategy to resolve conflicting constraint challenges which often emerge in traditional geometric layout optimization. Once the traditional algorithm detects and suggests solutions for conflicts, the LLM interprets the semantics of these solutions within the design context to make the best decision. Experimental findings highlight our framework's capability to create satisfactory and complex scene layouts across various scene categories.

Conceptual design Layout generation CAD Large language model AI agents

02Method Overview

From bubble graphs to an Iterated Design System

Human designers rarely jump straight from an idea to detailed geometry — they first perform conceptual design: analyzing requirements and sketching a rough layout before working out precise placements (detail design). Our framework automates this conceptual step, recursively decomposing a textual idea into the Iterated Design System (IDS) — an adaptive hierarchy in which every node, from broad functional areas ("rest area") down to concrete objects ("bed"), is a bounding box with both geometric attributes (size / position / orientation) and semantic attributes (description, requirements).

IDS hierarchical decomposition example for a bedroom — An example IDS. **(a)** The root node "Bedroom" recursively unfolds into "Rest Area", "Exercise Area", "Workspace" and "Storage Area", each further detailed by child nodes. **(b)** The resulting node tree, focused on the desk. **(c)** Boxes replaced with assets to obtain the final detailed design.

Expanding a node with an LLM multi-agent pipeline

Each unfolding step is handled by a pipeline of specialized LLM agents, following the Semantic Analysis → Geometric Analysis → LLM-aided Optimization workflow:

Semantics Analyzer — imagines the characters and story behind the scene, expands a requirement list, and groups requirements into substructures (with a checker to remove redundant ones).
Geometry Analyzer — turns the substructure list into a scene graph, using a custom DSL to express size, connection, and layout constraints.
LLM-in-the-loop Optimization — solves for a feasible, non-overlapping layout; when constraints conflict, an LLM picks the best resolution.

LLM multi-agent pipeline: Semantics Analyzer, Geometry Analyzer, LLM-in-the-loop Optimization — Expanding one IDS node ("Bedroom") with our LLM multi-agent framework. The Semantic Analyzer produces a story-grounded requirement and substructure list; the Geometry Analyzer builds a scene graph annotated with DSL constraints; layout optimization converts the graph into a concrete geometric layout, with an LLM stepping in whenever the optimizer hits contradictory constraints.

Substructure analyzer groups a requirement list into substructures — The substructure analyzer clusters the requirement list (blue triangles) into groups (purple triangles), each defining one substructure — e.g. a rest area for sleeping, a desk area for studying and night reading, an exercise corner for a treadmill.

03A DSL for Layout Constraints

Precise 3D geometry is hard for an LLM to reason about directly, so we give it a small toolbox of layout operators instead — a domain-specific language (DSL) that reduces each placement decision to as few free parameters as possible. Operators either relate a substructure to its parent (e.g. on_the_ground, at_the_corner) or to each other (e.g. in_front_of, on_top_of). Each operator maps to a term in the layout optimization objective, or to a constraint.

Illustration of DSL layout operators relative to a parent box and between sibling boxes — Operators under our DSL. Left, relative to the parent box: **(a1)** `at_the_corner`, **(a2)** `hang_on_upper_surface`, **(a3)** `align_along_the_right_side`, **(a4)** `on_the_ground`. Right, between substructures: **(b1)** `in_front_of`, **(b2)** `on_top_of`, **(b3)** `around`, **(b4)** `on_the_left_side`.

Resolving conflicting constraints with an LLM-in-the-loop

Since substructures and constraints are proposed by LLMs, the optimization can become infeasible — e.g. box A is required to be both left of B and right of B. We locate the conflicting constraints with Gurobi's ComputeIIS, generate a few candidate relaxation plans, and let an LLM agent judge which one best preserves the design intent.

LLM chooses between two candidate plans to resolve a layout constraint conflict for a bed and nightstand — A conflict between `at_the_corner` and `on_the_right` for a nightstand next to a bed. Two candidate plans are proposed; the LLM reasons about space usage, corner utilization, and the room's intended dynamic use before deciding on Plan 2.

04Results

Because conceptual design is not tied to a fixed scene type, our framework generalizes well beyond furnished rooms — from small indoor spaces to gardens and multi-purpose venues — while staying faithful to whatever detail the input text provides.

Gallery of twelve generated indoor scene layouts across diverse room types — A gallery of scenes generated from short textual descriptions — conservatory, indoor garden, pantry, dining room, music studio, laundry room, dress room, bedroom, playroom, reception room, study room, living room. Conceptual design lets our method scale to arbitrary scene types and complexity.

Diverse scenes generated within the same scene category — Diversity within one category. Given only *“create a bedroom with a storage unit”*, the model infers a plausible character profile and generates varied substructures each time; given a terrace-garden description, both indoor and outdoor gardens can be produced.

Comparison with prior scene-layout generators

Against LayoutGPT, ATISS, HOLODECK, DiffuScene and FlairGPT on the same indoor-room prompts, our layouts score highest on CLIP and rank first or second on GPT-4o- and human-rated aesthetics and layout rationality — with the biggest margin on richness of detail, a direct benefit of IDS's recursive decomposition.

Comparison of scene layouts generated by our method versus LayoutGPT, ATISS, HOLODECK, DiffuScene and FlairGPT — Comparison with prior indoor layout generation methods on the same textual prompt. Our optimization-backed layouts avoid overlap and out-of-bounds placement while providing more detailed substructure decomposition.

05BibTeX

@article{wu2026scenelayout,
  title    = {Scene layout via conceptual design},
  author   = {Wu, Wenzheng and Xiang, Chucheng and Lin, Zhi and Guan, Yirui and
              Bao, Ruchao and Liu, Zhongyuan and Wang, Ziqi and Liu, Ligang},
  journal  = {Computers \& Graphics},
  volume   = {136},
  pages    = {104553},
  year     = {2026},
  issn     = {0097-8493},
  doi      = {10.1016/j.cag.2026.104553},
  url      = {https://doi.org/10.1016/j.cag.2026.104553},
  publisher = {Elsevier}
}