2025 · Journal article
Divide-and-Conquer: Generating UI Code from Screenshots
Recursively divides a webpage screenshot along visual separation lines, has a multimodal model generate HTML/CSS for each leaf segment, and assembles child code upward into the full page; evaluated with GPT-4o, Gemini and Claude models and a small developer study.
Scope and limitations
It reconstructs static visual layout as HTML/CSS, not interaction, JavaScript behavior or backend logic. Test pages are self-contained snapshots with images replaced by placeholders; the main evaluation samples 111 of 348 top-ranked sites, adds 80 harder GitHub Pages pages, and the usefulness study has four participants. Segmentation uses manually set thresholds, and assembly can still misarrange elements.
Why it fits this collection
A segment-wise method for rebuilding website layout from screenshots, complementing the benchmark-oriented website papers.
Publication and review details
- Venue
- Proceedings of the ACM on Software Engineering
- Status
- Journal article
- Publication / posting date
- June 19, 2025
- Date source
- ACM-deposited Crossref metadata gives 19 June 2025 for online and print publication; the article text labels the issue July 2025.
- First arXiv submission
- June 24, 2024
- arXiv
- 2406.16386
- Reviewed arXiv version
- v3
- DOI
- 10.1145/3729364
This is a PACMSE journal article, volume 2, FSE issue, pages 2099–2122 (article FSE094). Its published title replaces the arXiv title 'Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach'. Author names follow the article text, which prints 'Michael Lyu', as does Crossref; the arXiv record lists 'Michael R. Lyu'. The ACM Digital Library copy could not be retrieved during review; the reviewed text is the 24-page ACM-format arXiv v3. The paper-linked repository has no license file at the pinned commit; the full dataset is hosted separately on Hugging Face without a dataset card.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.