AI Papers Reader

Personalized digests of latest AI research

View on GitHub

New AI Approach Revolutionizes Web Design to Code Conversion

Wuhan, China – Researchers have unveiled LATCODER, a novel approach that significantly improves the accuracy of converting webpage designs into functional code, particularly in preserving the intricate layouts that often trip up current AI models. This breakthrough promises to streamline front-end web development by bridging the gap between visual design and implemented code more effectively.

The core of LATCODER, detailed in a recent paper, lies in its “Layout-as-Thought” (LAT) strategy. Unlike existing methods that attempt to generate code for an entire webpage at once, LATCODER breaks down the design into smaller, manageable “image blocks.” This approach mimics human cognitive processes, where complex problems are solved step-by-step.

Here’s how it works:

  1. Smart Division: LATCODER first employs an efficient algorithm to divide the webpage design into distinct image blocks. This division is guided by detecting horizontal and vertical lines within the design, ensuring that elements are grouped logically and respecting the structured nature of HTML and CSS. For example, a website’s navigation bar might be identified as one block, a product card as another, and a footer as a third.

  2. Block-by-Block Generation: Each identified image block is then fed into a large language model (LLM) with a “Chain-of-Thought” (CoT) prompt. This prompt encourages the LLM to analyze the block, generate initial code, and then refine it based on specific criteria like content accuracy, color matching, and overall layout consistency within that block. This focused approach allows LLMs to better capture the details of each individual component.

  3. Intelligent Assembly: Once code snippets for all blocks are generated, LATCODER intelligently assembles them. It offers two primary assembly strategies: absolute positioning (using the precise coordinates of each block from the original design) and an LLM-based method that leverages the model’s understanding of layout. A dynamic selection process then chooses the optimal assembly, often aided by a “verifier” that compares the generated webpage’s screenshot against the original design.

The paper highlights the effectiveness of LATCODER through extensive experiments. Using powerful LLMs like DeepSeek-VL2, Gemini, and GPT-40, LATCODER demonstrated substantial improvements. For instance, when using DeepSeek-VL2, the TreeBLEU score (a metric for structural similarity) increased by a remarkable 66.67%, and the Mean Absolute Error (MAE) for visual similarity decreased by 38% compared to direct prompting methods. Human preference evaluations also showed that users favored webpages generated by LATCODER over 60% of the time.

A key challenge addressed by LATCODER is the tendency of current LLMs to misinterpret or lose crucial layout information when converting complex designs. For example, an Instagram profile page might be presented as a series of stacked elements by a less advanced model, even when explicitly instructed to maintain horizontal alignment. LATCODER’s block-based approach, with its emphasis on precise positioning and visual consistency for each segment, effectively mitigates these issues.

To further test their approach, the researchers also introduced CC-HARD, a new benchmark dataset featuring even more complex webpage layouts. The positive results on this challenging dataset underscore LATCODER’s robust performance and its potential to advance the state-of-the-art in automated UI development. The study suggests that LATCODER offers a significant step forward in creating AI systems that can accurately translate visual web designs into high-fidelity code.