AwesomeREA.org

2020 · Journal article

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Kevin Moran; Carlos Bernal-Cárdenas; Michael Curcio; Richard Bonett; Denys Poshyvanyk

ReDraw turns screenshots or design mock-ups into native Android GUI prototypes by detecting components, classifying widget types with a CNN, and assembling a plausible view hierarchy with a data-driven nearest-neighbor method.

Scope and limitations

The implementation covers 15 Android component classes and one screen at a time; multiple screens require manual combination, and hierarchy construction is tied to the mined screen size. Evaluation concerns appearance and view hierarchy, not recovery of original app logic or backends. Screenshots and reconstructed mock-ups served as proxies for real design artifacts; results were not independently reproduced here.

Why it fits this collection

Adds a data-driven foundation for recovering native widget types and plausible structure from pixels, complementing the existing REMAUI entry.

Publication and review details

Venue
IEEE Transactions on Software Engineering
Status
Journal article
Publication / posting date
February 2020
Date source
Publisher-deposited print issue date in Crossref (2020-02-01), represented at month precision.
First arXiv submission
February 7, 2018
arXiv
1802.02312
Reviewed arXiv version
v2
DOI
10.1109/TSE.2018.2844788

IEEE TSE 46(2), pages 196–221, February 2020. The author-hosted accepted manuscript and arXiv v2 are dated 2018; ESEC/FSE 2018 was a journal-first presentation, not the journal publication. Crossref creation on 2018-06-07 is a deposit date, not an independently verified online publication date. The author project page describes a rewritten implementation as forthcoming in Winter 2020; no original code repository was verified. The separately archived ReDraw dataset is dated 2019-01-02 (Zenodo 2530277).

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Primary sources

  1. https://api.crossref.org/works/10.1109/TSE.2018.2844788
  2. https://ieeexplore.ieee.org/document/8374985
  3. https://arxiv.org/abs/1802.02312
  4. https://arxiv.org/html/1802.02312v2
  5. https://www.cs.wm.edu/~denys/pubs/TSE%2718-ReDraw.pdf
  6. https://www.android-dev-tools.com/redraw
  7. https://2018.fseconference.org/details/fse-2018-Journal-First/3/Machine-Learning-Based-Prototyping-of-Graphical-User-Interfaces-for-Mobile-Apps
  8. https://zenodo.org/records/2530277
  9. https://zenodo.org/api/records/2530277