AwesomeREA.org

2026 · Conference paper

Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification

Zehai He; Wenyi Hong; Zhen Yang; Ziyang Pan; Mingdao Liu; Xiaotao Gu; Jie Tang

Benchmarks website development from visual prototypes and written requirements across 100 static-page, 66 interactive-frontend and 27 full-stack tasks, combining workflow-based functional checks with model-judged visual comparison.

Scope and limitations

The full-stack tier is specification-driven, not reconstruction solely by exploring a live website. Workflows test state management and CRUD outcomes, but finite agent judgments do not establish exact backend or durable-database equivalence. The current public evaluator maintains a browser session within each workflow and permits local business-API checks; this is not a universal persistence proof.

Why it fits this collection

A useful comparison for evaluating interactive and stateful website reconstruction, while separating written-specification development from behavior-only inference.

Publication and review details

Venue
ICML 2026
Status
Conference paper
Publication / posting date
September 29, 2026
Date source
Publication date stated for PMLR volume 306; the July 6–11 conference dates are not used as the publication date.
First arXiv submission
March 27, 2026
arXiv
2603.26648
Reviewed arXiv version
v3

The final PMLR paper has seven authors and pages 41393–41415; arXiv v3 is dated 2026-07-20. Current artifact evaluation differs from the paper configuration. Licensing signals conflict: README text says CC-BY-NC-SA-4.0, Hugging Face metadata says Apache-2.0, the project page says CC BY-SA 4.0, and setup.py lists MIT while the repository has no LICENSE file. No uniform artifact license is asserted.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Primary sources

  1. https://proceedings.mlr.press/v306/he26n.html
  2. https://proceedings.mlr.press/v306/
  3. https://raw.githubusercontent.com/mlresearch/v306/main/assets/he26n/he26n.pdf
  4. https://arxiv.org/abs/2603.26648
  5. https://arxiv.org/html/2603.26648v3
  6. https://vision2web-bench.github.io/
  7. https://github.com/zai-org/Vision2Web/blob/d80a4a3d0ea7326541ad87314294269a633789e4/README.md
  8. https://github.com/zai-org/Vision2Web/blob/d80a4a3d0ea7326541ad87314294269a633789e4/setup.py
  9. https://github.com/zai-org/Vision2Web/blob/d80a4a3d0ea7326541ad87314294269a633789e4/vision2web/evaluation/prompts.py
  10. https://github.com/zai-org/Vision2Web/blob/d80a4a3d0ea7326541ad87314294269a633789e4/vision2web/evaluation/functional_tester.py
  11. https://huggingface.co/datasets/zai-org/Vision2Web/blob/8f03299d92b9bd852e93852d0c21e8a4848ab661/README.md