InfiniDream: Synthesizing 100 Objects and 20 Scenes in Procedural 3D World Generation

InfiniDream is a VLM-driven (vision-language-model-guided) framework for open-ended procedural generation of photorealistic 3D worlds. Developed by researchers at Alaya Lab (Yi-Chuan Huang, Hao-Jen Chien, Zheng-Hui Huang, Yu-Lun Liu, Kaipeng Zhang, Zhixiang Wang), the method addresses a critical bottleneck in synthetic data generation and game development: existing VLM-based methods typically produce only simple 3D assets, while complex, photorealistic procedural systems still demand months of manual engineering. Procedural generation has historically been favored over static 3D models for its greater controllability, editability, and diversity — but automating it with AI has proven difficult, since existing VLM methods remain tied to predefined APIs and code factories, limiting their ability to create fundamentally new types of assets.

Two Key Components of the Methodology

The InfiniDream framework rests on two core innovations:

Symbolically Constrained Asset Generation fundamentally reframes the task. Rather than building 3D models from scratch or working within the confines of rigid predefined APIs, the system uses existing procedural algorithms as structural constraints and steers them through a VLM. This makes it possible to generate genuinely novel, never-before-seen assets while preserving photorealism, per-element editability, and seed-based diversity — the ability to reproduce variants and vary outputs based on random seeds.

Text-to-World Pipeline is an end-to-end process that converts a text description into a finished 3D world. It integrates several key stages: semantic scene planning (determining scene content from text input), Deterministic Frustum-Guided Grid Placement (placing objects in 3D space using the camera's view frustum for precise control), procedural population (automatically filling the scene with multiple objects), and animation (adding dynamics and motion to scene elements). This pipeline enables the construction of large-scale, highly detailed 3D worlds with precise control over every object's placement.

Testing Results and Evaluation

The authors conducted an extensive evaluation of the method across two main task types:

• 100 objects for synthesis — generating a variety of individual 3D assets • 20 full scenes — creating integrated 3D environments containing multiple objects

Across all four perceptual metrics used, InfiniDream showed measurable improvements. The team also ran a diversity analysis of the generated assets, ablation studies examining the contribution of each framework component, and a human preference evaluation, all of which confirmed the approach's effectiveness and practicality. The results demonstrate that language models can indeed steer procedural generation while still producing photorealistic outcomes that previously required months of manual work from experienced developers.

Resources and Credits

The project was published on October 19, 2026. The full source code is available in the GitHub repository (https://github.com/yichuanH/InfiniDream), where the community can reproduce the results and apply the method to its own projects. The authors have also prepared a detailed project page (https://yichuanh.github.io/infinidream-page/) featuring visualizations of the results and additional information about how the framework works. The research was conducted under Alaya Lab and marks a significant step toward automating procedural 3D content generation through artificial intelligence.