2018 · Conference paper

From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation

Chunyang Chen; Ting Su; Guozhu Meng; Zhenchang Xing; Yang Liu

UI2code translates a mobile UI image into an Android component hierarchy and layout skeleton using a neural encoder-decoder. The paper evaluates structural prediction and a small developer study of implementing layouts from generated skeletons.

Scope and limitations

The output covers component types and hierarchy from a single image; detailed styling and text require developer work. Structural token accuracy and an eight-participant study on five screenshots do not establish complete visual fidelity, navigation, app logic or backend recovery. The main split is screenshot-based; a separate evaluation on 1,208 images from 20 unseen apps provides limited generalization evidence.

Why it fits this collection

A native-app reconstruction foundation that explicitly generates and evaluates GUI structure from visual observations, beyond locating interface elements.

Publication and review details

Venue: ICSE 2018
Status: Conference paper
Publication / posting date: May 27, 2018
Date source: Publisher-deposited Crossref publication date for the ICSE proceedings article.
DOI: 10.1145/3180155.3180240

The author-hosted PDF prints 10.1145/3180155.3180222, which identifies another paper; the DOI here follows publisher-deposited metadata for this title, pages 665–676. The artifact README lists 29,887 screenshots, while the paper reports 185,277 image/skeleton pairs; they should not be treated as the same released corpus. The pinned legacy Torch/Lua repository has no license file; external data and model downloads were not tested.

Sources reviewed 2026-10-17

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Scope and limitations

Why it fits this collection

Publication and review details

Primary sources