Papers by Chong Li
Copied to clipboard
| Challenge: | Existing neural audio codecs are not capable of handling multi-domain audio data . et al., 2023) integrate speech modality with text-based large language models . |
| Approach: | They propose a unified audio codec with a single codebook to support multi-domain audio data . they propose combining a mix-of-experts strategy and a partitioned domain-adaptive codebook method . |
| Outcome: | The proposed codec outperforms existing codecs on acoustic and semantic representation capabilities. |
Copied to clipboard
| Challenge: | Existing benchmarks focus more on end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization. |
| Approach: | They propose a benchmark specifically designed to explore the problem-solving principles by decomposing 6.5K visual math problems into 10.9K step-level questions for evaluation. |
| Outcome: | The proposed benchmark covers 6.5K visual math problems and 10.9K step-level questions spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts. |
Copied to clipboard
| Challenge: | Existing research has demonstrated that the ability of large language models (LLMs) to generate humorous sentences is limited to producing 25 unique jokes. |
| Approach: | They propose a multi-stage curriculum preference learning framework to optimize both pun structure preferences and humor preferences by a Chinese Pun dataset. |
| Outcome: | The proposed method significantly outperforms baseline models on Chinese and English benchmark datasets. |
Copied to clipboard
| Challenge: | Keyphrase extraction (KPE) extracts phrases in a document that provide a concise summary of the core content. |
| Approach: | They propose an unsupervised keyphrase extraction method that ranks candidates by similarity between embeddings of source document and masked document. |
| Outcome: | The proposed method outperforms state-of-the-art methods on six benchmarks . it achieves average 3.53 improvement over the existing method . |
Copied to clipboard
| Challenge: | Existing studies on neurons focus on emotion and rhetoric, neglecting their intrinsic connections. |
| Approach: | They propose a framework for fine-grained steering of emotion and rhetoric in large language models . they propose 'neuro-based' masking method that integrates multi-dimensional screening . |
| Outcome: | The proposed method achieves directed induction of non-target sentences and enhancement of emotion tasks via rhetoric neurons. |
Copied to clipboard
| Challenge: | Mainstream approaches to aligning large language models heavily rely on human preference data. |
| Approach: | They propose a framework that fine-tunes a policy model using pairwise feedback data automatically mined from its outputs. |
| Outcome: | The proposed framework outperforms the base model with an average win rate of 69.7% across seven conversational or instruction-following datasets. |
Copied to clipboard
| Challenge: | Existing studies on text simplification systems have focused on unsupervised methods due to the limited evaluation data in language and domain. |
| Approach: | They propose a Chinese text simplification dataset that provides a detailed analysis and an annotation process. |
| Outcome: | The proposed dataset evaluates the performance of unsupervised methods and advanced large language models. |
Copied to clipboard
| Challenge: | Existing stance detection datasets are limited to a limited set of specific targets . current models are limited in their ability to detect large numbers of unseen targets based on a large number of unidentified targets. |
| Approach: | They propose a speaker interaction and target-aware prototypical contrastive learning model that can detect public opinion towards specific targets using social media data. |
| Outcome: | The proposed model achieves state-of-the-art in zero-shot conversational stance detection with only an F1-macro score of 43.81%. |
Copied to clipboard
| Challenge: | Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features . |
| Approach: | They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction. |
| Outcome: | The proposed framework outperforms baseline methods on four datasets. |
Copied to clipboard
| Challenge: | Large language models respond well in high-resource languages but struggle in low-resourced languages. |
| Approach: | They propose a method to construct cross-lingual instruction following samples with instruction in English and response in low-resource languages. |
| Outcome: | The proposed method builds a large-scale cross-lingual instruction tuning dataset on 10 languages. |
Copied to clipboard
| Challenge: | Existing multimodal sentence representation learning methods focus on aligning images and text at a coarse level, resulting in cross-modal misalignment bias and intra-modal semantic divergence. |
| Approach: | They propose a dual-level alignment learning framework for multimodal sentence representation learning that promotes cross-modal and intra-modal alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on semantic textual similarity and transfer tasks on semantic similarity, ranking distillation and global intra-modal alignment learning. |
Copied to clipboard
| Challenge: | Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions. |
| Approach: | They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models. |
| Outcome: | The proposed benchmarks show that the models perform better in open-ended conversations. |
Copied to clipboard
| Challenge: | Document-level event extraction aims to extract structured information from unstructured text. |
| Approach: | They propose a cross-document event extraction pipeline that integrates event information from multiple documents and provides a comprehensive perspective on events. |
| Outcome: | The proposed pipeline achieves about 72% F1 in end-to-end cross-document event extraction, setting up a benchmark for future research. |
Copied to clipboard
| Challenge: | Existing research focuses on the analysis of contextual structure in dialogue and the interactions between different emotions. |
| Approach: | They propose a method that generates Proximal Emotion Mean Vectors (PEMVs) based on emotion feature queues to optimize the spatial representation of text features. |
| Outcome: | The proposed method achieves state-of-the-art performance on three widely used benchmark datasets. |
Copied to clipboard
| Challenge: | Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs. |
| Approach: | They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them. |
| Outcome: | The proposed architecture achieves comparable performance with GShard with 2B parameters and computation. |
Copied to clipboard
| Challenge: | Current research on in-image machine translation focuses on synthetic data with simple background, single font, fixed text position, and bilingual translation. |
| Approach: | They propose an end-to-end model to handle the challenge of practical conditions in PRIM . they annotate a real-world one-line text image with complex background, fonts, diverse text positions . |
| Outcome: | The proposed model improves translation quality and visual effect compared to other models. |
Copied to clipboard
| Challenge: | Existing models only output short phrases or sentences, raising doubts about their practical usability. |
| Approach: | They propose a dataset focused on document-level model editing that aims to correct errors and outdated knowledge in Large language models (LLMs) they propose to use document-based model editing to improve model capabilities in real-world scenarios. |
| Outcome: | The proposed model editing task improves model capabilities in real-world scenarios and reduces the cost of retraining. |
Copied to clipboard
| Challenge: | Large reasoning models (LRMs) generate explicit Chain-of-Thought rationales, but often suffer from "overthinking". |
| Approach: | They propose a unified training framework to synthesize concise reasoning chains by identifying tokens that reduce the model’s likelihood of the correct answer. |
| Outcome: | Experiments on deepSeek-R1-Distill-Qwen backbones show that MUTO yields better efficiency-accuracy Pareto frontier. |
Copied to clipboard
| Challenge: | RAAMove is a comprehensive multi-domain corpus dedicated to the annotation of move structures in Research Article (RA) abstracts. |
| Approach: | They propose a multi-domain corpus dedicated to the annotation of move structures in RA abstracts. |
| Outcome: | The proposed corpus is based on a human-annotated dataset and a BERT-based model to verify its effectiveness. |
Copied to clipboard
| Challenge: | Tokenization is a foundational step for Large Language Models (LLMs) but low compression rate of vanilla tokenizers decelerates training and inference process. |
| Approach: | They propose a method to replace the vocabulary of Large Language Models (LLMs) by learning a one-to-one mapping matrix for token IDs. |
| Outcome: | The proposed method significantly improves multilingual text compression rates and vocabulary initialization for Large Language Models. |
Copied to clipboard
| Challenge: | Existing approaches focus on predefined dimensions that overlook finer conceptual distinctions . a new framework is proposed to investigate the subdimensions underlying coarse-grained semantic dimensions . |
| Approach: | They propose a framework that decomposes word embeddings into multiple sub-embeddings . they propose to map these subdimensions to brain activation to assess their plausibility . |
| Outcome: | The proposed framework decomposes word embeddings from large language models into sub-embeddings, each encoding specific semantic information. |
Copied to clipboard
| Challenge: | Existing methods that adapt LVLMs to egocentric tasks overlook critical agent-environment interactions, limiting their ability to perform egoic reasoning. |
| Approach: | They propose a zero-shot paradigm to enhance egocentric reasoning by simulating human causal reasoning by formalizing ego-centric reasoning using a structural causal model. |
| Outcome: | The proposed method improves egocentric reasoning abilities on six tasks. |
Copied to clipboard
| Challenge: | Existing large language models focus on causal coherence, neglecting the complex story arcs and orchestration inherent in human narratives. |
| Approach: | They propose a high-dimensional framework for narrative orchestration that unifies human and model perspectives while jointly characterizing narrative function and structure in a common space. |
| Outcome: | The proposed framework unifies human and model perspectives while jointly characterizing narrative function and structure in a common space. |
Copied to clipboard
| Challenge: | Experimental results show that language group specialization on experts improves multilingual performance. |
| Approach: | They propose to dynamically group and scale up parameters of multilingual Large Language Models while boosting positive transfer among similar languages. |
| Outcome: | The proposed method reduces negative transfer between languages and boosts performance on 18 to 128 languages. |
Copied to clipboard
| Challenge: | Recent advances in multi-modal learning have enhanced MLLMs' ability to reason about visual content. |
| Approach: | They propose a framework that unifies multi-step multimodal reasoning with grounded visual understanding. |
| Outcome: | The proposed framework surpasses state-of-the-art methods by +6.5 gIoU and +9.2 cIou on ReasonSeg and achieves 49.7 mAP on SegInW under zero-shot settings. |
Copied to clipboard
| Challenge: | Existing memory systems for LLMs store isolated records and retrieve fragments . Existing systems store isolated data and fragments, limiting their ability to consolidate evolving experience and resolve conflicts. |
| Approach: | They propose an engram-inspired memory operating system that implements an 'engram'-inspired lifecycle for computational memory. |
| Outcome: | Experiments on LoCoMo, LongMemEval, and PersonaMeM-v2 show that EverMemeOS outperforms state-of-the-art methods on memory-augmented reasoning tasks. |
Copied to clipboard
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Copied to clipboard
| Challenge: | Text-to-SQL parsing aims to parse natural language questions into SQL queries . current attention-based approaches can only model alignments at the token level . |
| Approach: | They propose a method to leverage explicit lexico-logical alignments by identifying possible phrase-level alignments and injecting them as additional contexts into the parsing procedure. |
| Outcome: | The proposed approach improves performance by 3.4% on Squall. |
Copied to clipboard
| Challenge: | Experimental results show that a sequence-to-sequence learning framework with neural networks can be effective for Chinese Spelling Correction (CSC) |
| Approach: | They propose a sequence-to-sequence learning framework with neural networks that generates more valuable training instances and adds task-specific examples to enhance the model. |
| Outcome: | The proposed method improves generalization and robustness of multiple CSC models across three datasets. |
Copied to clipboard
| Challenge: | Existing methods for converting unstructured text into structured Knowledge Graphs (KGs) have limitations such as large amount of noise, inaccurate knowledge, and hallucination . |
| Approach: | They propose a GraphJudge framework to reduce noise in real-world documents . they propose Graphjudge to fine-tune a LLM as a graph judge to enhance quality . |
| Outcome: | The proposed framework eliminates noise in real-world documents and improves the quality of generated KGs. |
Copied to clipboard
| Challenge: | Despite the development of many subdirections, Cross-Document Cross-Lingual NLI remains largely unexplored. |
| Approach: | They propose a novel paradigm that extends traditional NLI capabilities to multi-document, multilingual scenarios by integrating RST-enhanced graph fusion with interpretability-aware prediction. |
| Outcome: | The proposed method improves on existing models and document-level NLI to multi-document, multilingual scenarios. |
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated impressive accomplishments in reasoning and planning, but their abilities in multi-agent collaborations remain unexplored. |
| Approach: | They propose to use explicit belief state representations to enhance task performance and the accuracy of ToM inferences for LLM-based agents. |
| Outcome: | The proposed model improves performance and accuracy of ToM inferences for LLM-based agents. |
Copied to clipboard
| Challenge: | Mainstream speaker diarization systems rely only on acoustic information, making it challenging in complex aural environments. |
| Approach: | They propose a multimodal approach that integrates audio, visual, and semantic cues to enhance speaker diarization. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on multi-party conversations . it integrates audio-visual-semantic cues into the clustering process for acoustic speaker embeddings . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in natural language processing tasks but are vulnerable to harmful content and being exploited for malicious purposes. |
| Approach: | They propose a framework to measure the risk coverage of alignment datasets across three dimensions: Lexical Diversity, Malicious Intent, and Jailbreak Tactics. |
| Outcome: | The proposed framework measures risk coverage across Lexical Diversity, Malicious Intent, and Jailbreak Tactics. |
Copied to clipboard
| Challenge: | Current approaches typically merge sentence-level parsing outputs for discourse input, resulting in fragmented graphs and degraded downstream performance. |
| Approach: | They propose a task for discourse-level text scene graph parsing that merges sentence-level outputs for discourse input and propose 'DiscoSG' a dataset of 400 expert-annotated and 8,430 synthesised multi-sentence caption-graph pairs is used to test the new task. |
| Outcome: | The proposed task improves SPICE by 30% over the baseline while achieving 86 faster inference than existing models. |
Copied to clipboard
| Challenge: | Structured Sentiment Analysis (SSA) is a problem of bi-lexical dependency graph parsing due to the internal structures of spans neglected. |
| Approach: | They propose to use latent spans as latent subtrees to model internal structures of spans and leverage TreeCRFs to extract the complete opinion tuple from a sentence. |
| Outcome: | The proposed method performs significantly better than all previous bi-lexical methods, achieving new state-of-the-art. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) extend their capabilities through function-calling (FC) however, obtaining and annotating real function-called data is challenging, and synthetic data from existing pipelines suffers from unreliable APIs, limited tool scalability, insufficient diversity, and weak quality control. |
| Approach: | They propose a pipeline for generating FC training data using reliable tools and a multi-agent framework that supports a dialogue generation system that produces conversations spanning diverse scenarios. |
| Outcome: | The proposed pipeline outperforms open-source models in in-domain FC performance and out-of-domain generalization while reaching FC capabilities comparable to some of the latest API-based models. |
Copied to clipboard
| Challenge: | Experimental results show that multi-head attention module evolves functional specialization after multi-task training. |
| Approach: | They propose a method to quantify the degree of functional specialization in multi-head attention . they propose 'multi-task training' method to increase functional specialisation and mitigate negative information transfer . |
| Outcome: | The proposed method increases functional specialization and mitigates negative information transfer in multi-task learning without adding any parameters. |
Copied to clipboard
| Challenge: | Existing studies show that multilingual generative models exhibit a strong language bias toward high-resource languages. |
| Approach: | They propose a cross-lingual alignment framework exploiting pairs of translation sentences to improve cross-linguistic abilities. |
| Outcome: | The proposed framework improves cross-lingual abilities and mitigates performance gap. |
Copied to clipboard
| Challenge: | Existing methods to develop dialogue agents for complex tasks require sparse reward signals. |
| Approach: | They propose a divide-and-conquer approach that exploits the hidden structure of a task . they use subgoals to divide a goal-oriented task into simpler subgoal sets . |
| Outcome: | The proposed approach performs competitively against state-of-the-art methods that require human-defined subgoals. |
Copied to clipboard
| Challenge: | Existing cross-document event coreference resolution models lack the ability to capture long-distance dependencies. |
| Approach: | They propose to construct document-level Rhetorical Structure Theory trees and cross-document Lexical Chains to model structural and semantic information of documents. |
| Outcome: | The proposed model outperforms baseline models on English and Chinese datasets by large margins. |
Copied to clipboard
| Challenge: | Deciphering oracle bone scripts using AI technology is not an overnight task due to the evolution of written language over millennia. |
| Approach: | They propose a framework that utilizes Large Multi-modal Models (LMMs) for interpreting Oracle Bone Script (OBS). |
| Outcome: | The proposed framework provides quantitative analyses and superior deciphering capability. |
Copied to clipboard
| Challenge: | Current approaches to decoding language from the human brain rely on unimodal representations, neglecting the brain’s inherently multimodal processing. |
| Approach: | They propose a framework that leverages Multimodal Large Language Models to align brain signals with a shared semantic space encompassing text, images, and audio. |
| Outcome: | The proposed framework achieves an 8.48% improvement on the most commonly used benchmark on fMRI datasets with textual, visual, and auditory stimuli. |
Copied to clipboard
| Challenge: | Recent studies have focused on identifying the sentiment polarity of aspects in product reviews. |
| Approach: | They propose to use supervised Contrastive Pre-Training to learn implicit sentiment . they propose to train large-scale sentiment-annotated corpora from in-domain language resources . |
| Outcome: | The proposed model achieves state-of-the-art performance on SemEval2014 benchmarks and comprehensively validates its effectiveness on learning implicit sentiment. |
Copied to clipboard
| Challenge: | Existing methods for multimodal sentiment analysis are often dynamically incomplete. |
| Approach: | They propose a new uncertainty-calibrated elastic alignment framework to address these issues by employing probabilistic imputation to capture cross-modal ambiguity and leverage the estimated uncertainty to drive elastic alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple benchmarks and consistently outperformed existing models. |
Copied to clipboard
| Challenge: | Existing methods for question answering system lack large-scale question matching corpora . lack of large-sized question matching results in problem solving . |
| Approach: | They propose a large-scale Chinese question matching corpus which is released to the public . they use a search engine to collect large-sized question pairs related to high-frequency words . |
| Outcome: | The proposed corpus is more general than paraphrase corpus as it focuses on intent matching rather than paraphrasing. |
Copied to clipboard
| Challenge: | Existing methods for deception detection lack sample-level dynamic annotations for emotions and personality. |
| Approach: | They propose a multi-model multi-prompt annotation scheme and a strict label quality evaluation standard for deception, emotion, and personality annotations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the MDPE and DDEP datasets. |
Copied to clipboard
| Challenge: | Aspect-based Sentiment Analysis (ABSA) data augmentation has attracted increasing attention in recent years due to data sparsity. |
| Approach: | They propose a framework to augment ABSA data using pseudo labels for target domain . they refine generated labeled data using a natural language inference filter . |
| Outcome: | The proposed framework outperforms 7 strong baselines on 4 kinds of ABSA tasks. |
Copied to clipboard
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue generation models struggle with dull and repetitive outputs, a problem commonly termed as text degeneration. |
| Approach: | They propose a framework that allows the model to "cheat" the objective by duplicating knowledge segments in a superficial pattern matching based on overlap. |
| Outcome: | The proposed framework can be applied to a WoW dataset and shows that it works across models and decoding strategies. |
Copied to clipboard
| Challenge: | Existing studies indicate that Large Language Models perform at a level comparable to humans with advantages of speed and cost-effectiveness in different fields. |
| Approach: | They propose to introduce four unexplored factors and a new dimension of question difficulty to provide a more comprehensive understanding of LLMs’ judgments across varying question intricacies. |
| Outcome: | The proposed dimensions of question difficulty and answer quantity provide valuable insights into optimizing LLMs’ performance as judges. |
Copied to clipboard
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Copied to clipboard
| Challenge: | Existing video LLMs excel at capturing the overall description of a video but lack the ability to demonstrate an understanding of temporal dynamics and localized content within the video. |
| Approach: | They propose a Time-Perception Enhanced Video Grounding via Boundary Perception and Temporal Reasoning to improve LLMs' understanding of video temporality. |
| Outcome: | The proposed method improves on three datasets: ActivityNet, Charades, and DiDeMo (up to 11.2% improvement on R@0.3). |
Copied to clipboard
| Challenge: | Existing alignment paradigms for creative writing use static reward signals and supervised data. |
| Approach: | They propose a constraint-aware reward model that synthesizes query-specific criteria to provide fine-grained preference judgments. |
| Outcome: | The proposed framework aligns models with human preferences across content quality and structural paradigms without supervised fine-tuning and ground-truth references. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly equipped with capabilities of real-time web search and integrated with protocols like the Model Context Protocol (MCP). |
| Approach: | They investigate the vulnerability of Large Language Models to hidden adversarial prompts . they evaluate two critical attack scenarios: malicious content relay and sensitive data leakage . |
| Outcome: | The proposed extension could introduce new security vulnerabilities. |