Papers by Jianguo Li
SumSurvey: An Abstractive Dataset of Scientific Survey Papers for Long Document Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | a growing need for long document summarization datasets with 16k input is causing problems. |
| Approach: | They propose to use a dataset to analyze salient information in long document summarizations. |
| Outcome: | The proposed dataset outperforms existing models and LLMs in the distribution form of salient information and the distribution of salinal information is an indicator of quality. |
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models that use self-attention and position embedding have anomalous behavior that hinder long context window extrapolation. |
| Approach: | They propose a collinear constraint between Q and K to integrate RoPE and self-attention. |
| Outcome: | The proposed model integrates self-attention and position embedding into LLMs without fine-tuning. |
CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing multi-task learning approaches for large language models fall short due to computational intensive or lack of simultaneous task convergence. |
| Approach: | They propose a new multi-task learning approach that dynamically adjusts task weights during the training process, ensuring that the validation loss of all tasks progresses towards convergence at an even pace. |
| Outcome: | The proposed approach improves the performance of large language models by up to 13% compared to the second-best approaches. |
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study explores efficient ultra-long context modeling. |
| Approach: | They propose to use Hierarchical Sparse Attention to achieve efficient ultra-long context modeling. |
| Outcome: | The proposed model performs comparable to full-attention baselines on in-domain and out-of-domain tasks. |
E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Considerable efforts have been and are still being put into increasing the context length of Large Language Models (LLMs) |
| Approach: | They propose an approach that divides long contexts into chunks, compresses each into soft prompts using a pretrained text encoder, and aligns these representations with a decoder-only LLM via an adapter. |
| Outcome: | The proposed approach outperforms 8 state-of-the-art methods in effectiveness and efficiency for document summarization and question answering, and achieves the best performance on LongBench v2 among models of comparable size. |
ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text classification use only encoders or decoders that do not allow for the use of labels in unseen domains. |
| Approach: | They propose an autoencoder that encodes text into two disentangled spaces and decodes it to generate text with labels in the unseen domains. |
| Outcome: | The proposed model outperforms the existing methods in label-partially-unseen and label-fully-un-seeen scenarios and even outperfects the SOTA methods. |
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion large language models generate text through iterative denoising with bidirectional attention, enabling richer contextual dependencies. |
| Approach: | They propose a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens. |
| Outcome: | The proposed method achieves 5.48 times speedup with +0.48 accuracy on LLaDA-8B and is orthogonal to mainstream inference optimizations. |
GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Programming languages have rich semantics that are represented by graphs and not available from the surface form of source code. |
| Approach: | They propose to use graph neural networks and cross-modal alignment technologies to inject structural information of code into LLMs as an auxiliary task during finetuning. |
| Outcome: | The proposed framework improves on five code tasks with six different baseline LLMs, while incurring no cost at inference time. |
Multi-Modal Generative Adversarial Network for Short Product Title Generation in Mobile E-Commerce (N19-2)
Copied to clipboard
| Challenge: | Existing methods for short product title generation only consider textual information from long titles . MM-GAN incorporates image information and attribute tags from product, as well as textual info from original long titles. |
| Approach: | They propose a multi-modal generative adversarial network for short product title generation in E-commerce . they incorporate image information and attribute tags from product, as well as textual information from original long titles . |
| Outcome: | The proposed model outperforms state-of-the-art methods on a large-scale E-commerce dataset. |
JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models (2026.findings-acl)
Copied to clipboard
Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia
| Challenge: | Currently, evaluation criteria and methods used for jailbreak effectiveness are inconsistent. |
| Approach: | They propose a framework to measure jailbreak effectiveness using a model that filters out jailbreak noise while preserving the original malicious question. |
| Outcome: | The proposed framework outperforms existing evaluation methods on a challenging benchmark containing 330 human-labeled, non-rejected jailbreak instances. |
The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic Truthfulness (2026.acl-long)
Copied to clipboard
Yueheng Mao, Min Yu, Gengwang Li, Jianguo Jiang, Gang Li, Meng Zhang, Zhen Xu, Weiqing Huang, Ming Liu
| Challenge: | Large Language Models (LLMs) often generate overconfident yet factually incorrect hallucinations. |
| Approach: | They propose a black-box-based framework that captures stubborn hallucinations by integrating internal geometric dynamics with output probability distributions. |
| Outcome: | The proposed framework outperforms white-box methods and reduces computational overhead by over 90%. |