Papers by Mingwei Liu
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models fail to capture complex interplay between functionality and security. |
| Approach: | They propose a benchmark for secure code generation constructed from real-world, high-risk Java repositories. |
| Outcome: | The proposed benchmarks highlight the gap between functional and secure code generation in LLMs. |
ArkRepoBench: A Repository-Level Code Completion Benchmark for HarmonyOS Development (2026.findings-acl)
Copied to clipboard
Yanlin Wang, Bowen Zhang, Yanli Wang, Daya Guo, Terry Yue Zhuo, Jiachi Chen, Mingwei Liu, Xingong Zhang, Zibin Zheng
| Challenge: | Despite the maturity of LLM-based code assistance for mainstream languages, the capabilities of ArkTS are largely unexplored. |
| Approach: | They propose to benchmark repository-level code completion for ArkTS using 7,519 samples from 20 official HarmonyOS repositories. |
| Outcome: | The proposed benchmark covers multiple difficulty levels and categorizes completion instances into Single-File, Cross-Filled Independent, and Cross-Filed Dependent settings based on dependency analysis. |
From Conversation to Evaluation: Benchmarking LLMs on Development Knowledge via SimpleDevQA (2026.findings-acl)
Copied to clipboard
Jing Zhang, Lianghong Guo, Yanlin Wang, Terry Yue Zhuo, Yong Wang, Mingwei Liu, Jiachi Chen, Ensheng Shi, Yuchi Ma, Hongyu Zhang, Zibin Zheng
| Challenge: | Existing Dev Knowledge QA benchmarks are limited in development knowledge scope and often not built from real user queries. |
| Approach: | They conduct preliminary analysis of real user–LLM dialogues from WildChat to investigate the importance of Dev Knowledge QA in AI-assisted software development scenarios. |
| Outcome: | The proposed benchmark is based on real user–LLM dialogues from WildChat. |
Raw Pointer Rewriting with LLMs for Translating C to Safer Rust (2026.findings-acl)
Copied to clipboard
| Challenge: | C2Rust is a system programming language that enforces strict memory and type safety guarantees. |
| Approach: | They propose a raw pointer rewriting technique that lifts raw pointers in individual functions to appropriate Rust data structures. |
| Outcome: | The proposed technique eliminates 18.57% of local raw pointers and improves memory safety on 28 real-world C projects. |