Gemini and Semantic Search: AI Picks Furniture from Architectural Drawings with 87% Accuracy

Choosing furniture from an architectural drawing is a task that construction and interior design still handle by hand. A specialist takes the drawing, opens a supplier's catalog and methodically works through the items, comparing dimensions, materials and style. This step alone takes 20 to 40 person-hours per project. With several projects, the cost multiplies. A Russian development team set out to close this gap by building an AI system that automates the entire selection process and delivers recommendations with 87% accuracy.

The problem sits at the intersection of computer vision and information retrieval. An architectural drawing is a specialized document with notations, symbols, scale grids and multiple layers. Simply recognizing an object in an image is not enough: the system has to understand the layout as a whole, identify functional zones, pick out spots for specific categories of furniture and account for the real constraints of the space.

The core of the system is a multi-model architecture with Gemini as the main coordinator. This model handles the understanding of the drawing: it recognizes the layout, identifies rooms, determines where the bedroom and the living room are, where the walkways run, and where there are limits on height, lighting or wall configuration. This is not just OCR and not plain object recognition. Gemini works with the semantics of the architectural document and extracts structured data for the next layer of the system.

Once the drawing is broken down into meaningful units, semantic search over the catalog takes over. Every item has been vectorized in advance: product attributes such as dimensions, material, style, color palette and price segment are converted into vector embeddings. The system matches the drawing's requirements against this vector representation and finds the closest matches. The output is not just a list of hundreds of suitable items but a ranked selection that explains why a particular model is recommended for a specific spot on the plan.

The technical architecture does not stop at two components. Besides Gemini and semantic search, the pipeline includes models for drawing preprocessing: scale normalization, layer separation and cleanup of scanning artifacts. Real drawings from design firms arrive in varying condition, from clean CAD exports to scanned paper documents with tears and stains. The system has to work reliably across this whole range without manual preprocessing.

Reaching 87% accuracy did not happen on the first try. The team iterated on several key components: the quality of parsing drawings in different formats, the vectorization strategy for catalog data and the final ranking mechanism. Unusual layouts proved a separate difficulty, such as when an architect uses non-standard notation or a drawing shows only a fragment of a room. For these edge cases, the team added fallback logic and an extra layer of result validation.

The practical value of the system lies in scaling specialists' working time. Where a designer could previously work through 2–3 projects a week in detail, with the AI system they review and adjust ready-made recommendations instead of building them from scratch. For construction companies working with standard residential complexes, this means being able to run dozens of projects in parallel without a proportional increase in headcount.

The project shows how multi-model AI systems are starting to automate operational tasks long considered too specialized for machine processing. Architectural drawings are a complex type of unstructured data, and the fact that Gemini can handle their semantic parsing opens the way for similar solutions in adjacent areas: engineering diagrams, structural drawings and technical specifications. The next logical step is integration with BIM platforms and direct export of recommendations into project documentation.