2024 · Conference paper
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
Studies screenshot-to-HTML reconstruction through instruction tuning and two evaluation benchmarks. Web2Code combines generated webpages, refined WebSight and pix2code image/code pairs, and WebSRC-derived question answering. Models produce HTML from a webpage image and instruction; rendered output is compared visually, while a separate benchmark measures screenshot understanding.
Scope and limitations
Evaluation uses 1,198 screenshots and 5,990 yes/no questions. WCGB grades rendered images with GPT-4V; its User Interface and Interactivity category measures element appearance, not executed click paths, persistent state or backend behavior. Synthetic and refined examples do not establish coverage of arbitrary real web applications, original-source equivalence or code quality. The paper reports contact-information issues in 2.7% of a 1,000-item privacy sample, so a blanket claim that the data is free of such information is unsupported.
Why it fits this collection
A foundational visual reconstruction dataset and evaluation method, useful for distinguishing screenshot fidelity from the behavioral guarantees of later website-reconstruction benchmarks.
Publication and review details
- Venue
- NeurIPS 2024 Datasets and Benchmarks
- Status
- Conference paper
- Publication / posting date
- 2024 (exact date not verified)
- Date source
- The official NeurIPS proceedings and publisher-deposited Crossref metadata identify publication year 2024; Crossref provides no finer publication date. The first arXiv submission date is recorded separately.
- First arXiv submission
- June 28, 2024
- arXiv
- 2406.20098
- Reviewed arXiv version
- v2
- DOI
- 10.52202/079017-3560
Published in NeurIPS volume 37, pages 112134–112157; full author names follow the proceedings PDF. The repository still cites the preprint; reviewed arXiv v2 is dated 2024-11-17. The LLaVA-based training subtree carries Apache-2.0. Dataset licensing is inconsistent: the GitHub README and dataset-card frontmatter say MIT, while the same card’s body says CC BY 4.0 with research-only/non-commercial wording. No blanket reuse permission is inferred for constituent data or model weights. The training metadata requires Python >=3.8 and pins PyTorch 2.0.1; evaluation needs browser rendering and an external vision-model API. No training, evaluation, model or dataset payload was run or downloaded.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.
Primary sources
- https://proceedings.nips.cc/paper_files/paper/2024/hash/cb66be286795d71f89367d596bf78ea7-Abstract-Datasets_and_Benchmarks_Track.html
- https://proceedings.neurips.cc/paper_files/paper/2024/file/cb66be286795d71f89367d596bf78ea7-Paper-Datasets_and_Benchmarks_Track.pdf
- https://arxiv.org/abs/2406.20098v2
- https://arxiv.org/html/2406.20098v2
- https://api.crossref.org/works/10.52202/079017-3560
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/README.md
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/web2code/README.md
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/web2code/pyproject.toml
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/web2code/LICENSE
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/code_generation/README.md
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/code_generation/gpt4_vision_evaluation.py
- https://github.com/MBZUAI-LLM/web2code/blob/48c356a97dd6199f29a9710fdd2b4b2689b5e361/webpage_understanding/README.md
- https://huggingface.co/datasets/MBZUAI/Web2Code/blob/21e587b4509b946e56732fe8c621b17bad631e6b/README.md