Papers by Shangshu Qian
WAFFLE: Fine-tuning Multi-Modal Model for Automated Front-End Development (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promise in generating source code, but two major challenges persist in UI-to-HTML code generation: (1) effectively representing HTML’s hierarchical structure for LLMs; and (2) bridging the gap between the visual nature of UI designs and the text-based format of HTML code. |
| Approach: | They propose a structure-aware attention mechanism that uses a contrastive fine-tuning approach to align LLMs’ understanding of UI images and HTML code. |
| Outcome: | The proposed model outperforms existing methods on the WebSight-Test and Design2Code benchmarks. |