Papers by Jinman Zhao
Fine-tuning Language Models for Joint Rewriting and Completion of Code with Potential Bugs (2024.findings-acl)
Copied to clipboard
| Challenge: | Previous work has demonstrated shortcomings of large language models of code (CodeLLMs) in completing drafty partial code with potential bugs. |
| Approach: | They propose to use large language models of code to fine-tune their models to rewrite and complete drafty partial code into functional full programs. |
| Outcome: | The proposed approach achieves superior pass rates over baselines and preserves the integrity of the original partial implementations. |
PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy (2025.acl-long)
Copied to clipboard
| Challenge: | Existing pipelines that combine document image restoration with semantic-aware post-OCR correction can improve text extraction from degraded images. |
| Approach: | They propose a two-stage pipeline that combines document image restoration with semantic-aware post-OCR correction to enhance both visual clarity and textual consistency. |
| Outcome: | The proposed pipeline reduces character error rates by 63.9-70.3% on 13,831 pages of real historical documents in English, French, and Spanish compared to OCR on raw images. |
PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embeddings assume fixed finite-size vocabularies, hindering their ability to provide useful word representations for out-of-vocaulary words. |
| Approach: | They propose a model that generalizes word embeddings without extra contextual information . they use the spellings of words to model subword segmentation and compute subword-based compositional word embeds. |
| Outcome: | The proposed model can generate meaningful subword segmentations without any source of explicit morphological knowledge. |
Generalizing Word Embeddings using Bag of Subwords (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings techniques have a fixed vocabulary, i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus. |
| Approach: | They propose a subword-level word vector generation model that views words as bags of character n-grams and provides good vectors for rare or unseen words. |
| Outcome: | The proposed model performs state-of-the-art in English word similarity task and in joint prediction of part-of speech tag and morphosyntactic attributes in 23 languages. |
Tiny Budgets, Big Gains: Parameter Placement Strategy in Parameter Super-Efficient Fine-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods such as LoRA and VeRA use memory-efficient methods to fine-tune large language models. |
| Approach: | They propose a method that uses only 1–5% of the standard LoRA parameters and achieves state-of-the-art performance across a wide range of tasks. |
| Outcome: | The proposed method achieves state-of-the-art performance across a wide range of tasks using only 1–5% of the standard LoRA parameters. |
Low-Rank Interconnected Adaptation across Layers (2025.findings-acl)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) method that learns weight updates W = AB for pretrained weights W through low-rank adapters A and B. |
| Approach: | They propose a low-rank interconnected adaptation across layers method that introduces an interconnected framework with locally shared A and globally shared B experts. |
| Outcome: | The proposed method improves expressiveness across domains and modalities and enables higher-rank W with equal or fewer parameters. |
CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models often lack coverage for subtle corner cases . a substantial amount of effort has been applied to address this challenge . |
| Approach: | They propose a framework that generates adversarial test cases that expose latent vulnerabilities in code submissions. |
| Outcome: | The proposed framework improves the True Negative Rate (TNR) of existing datasets and generates superior adversarial cases on liveCodeBench. |
Inside-Outside Algorithm for Probabilistic Product-Free Lambek Categorial Grammar (2025.coling-main)
Copied to clipboard
| Challenge: | Many studies have discovered hidden syntactic structures within language models without the guidance of explicit rules. |
| Approach: | They propose an inside-outside algorithm for Probabilistic Lambek Categorical Grammar. |
| Outcome: | The proposed algorithm is used in the estimation of probabilistic context-free grammars. |
Sequence-level Large Language Model Training with Contrastive Preference Optimization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new method to improve the performance of large language models requires a small computational cost. |
| Approach: | They propose a CPO procedure that can inject sequence-level information into the model at any training stage without expensive human labeled data. |
| Outcome: | The proposed objective surpasses the next token prediction in terms of win rate in instruction-following and text generation tasks. |
UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods such as LoRA and VeRA use a low-rank approximation method that reduces the number of trainable parameters without compromising performance. |
| Approach: | They propose a parameter-efficient fine-tuning approach that leverages a low-rank approximation method that reduces the number of trainable parameters without compromising performance. |
| Outcome: | The proposed approach outperforms existing methods on GLUE and E2E benchmarks and is effective in instruction-tuning large language models and image classification models. |
LLM-supertagger: Categorial Grammar Supertagging via Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that LLMs are underperforming in classification tasks due to their decoder-based nature. |
| Approach: | They propose a method that significantly boosts LLMs' performance in supertagging for both Combinatory Categorial Grammar (CCG) and Lambek Categorian Grammar (LCG). |
| Outcome: | The proposed method outperforms LSTM and encoder-based models and achieves state-of-the-art performance. |
A Generative Model for Lambek Categorial Sequents (2024.lrec-main)
Copied to clipboard
| Challenge: | generative models such as PLC+ generate grammatical sentences with a high probability of being grammatized. |
| Approach: | They propose a generative model, PLC+, for generating Lambek Categorial Grammar(LCG) sequents. |
| Outcome: | The proposed model generates Lambek Categorial Grammar(LCG) sequents and is more robust to probabilistic context-free grammars. |
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models (2025.acl-long)
Copied to clipboard
Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, WangYan WangYan, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, Bryan Hooi
| Challenge: | Large language models face significant challenges in handling long-context tasks because of their limited effective context window size during pretraining, which restricts their ability to generalize over extended sequences. |
| Approach: | They propose a training strategy for extending the context window of LLMs including impactful token analysis, position index transformation, and training optimization strategies. |
| Outcome: | Experiments on three types of LLMs show that LongRecipe can utilize long sequences while requiring only 30% of the target context window size. |
On the Representation Geometry of LoRA Model Merging (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for low-rank Adaptation (LoRA) fine-tuning focus on globally shared structure . combining SVD with CUR improves performance of LoRA model merging . |
| Approach: | They propose a training-free method that combines SVD and CUR decomposition to improve LoRA merging performance. |
| Outcome: | The proposed procedure improves on vision and language benchmarks. |
Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are limited to small sample and fail to demonstrate LLM susceptibility to context with potential human bias. |
| Approach: | They propose a benchmark for evaluating LLM investment decision-making when faced with uncertainty and possible human-biased opinions. |
| Outcome: | The proposed model can herd the explicit bias in context and even exceed human performance in predicting future stock return. |