Papers by Xiang Jiang
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved considerable performance across various agentic planning tasks. |
| Approach: | They propose a data-centric approach that applies agents with knowledgeable self-awareness like humans to a heuristic situation judgement criterion to mark special tokens on their self-explored trajectories for collecting training data. |
| Outcome: | The proposed paradigm outperforms baseline models on various tasks with minimal external knowledge. |
Copied to clipboard
| Challenge: | Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, but academic research remains non-reproducible due to the lack of publicly available training data. |
| Approach: | They propose a system for long-form song generation with fine-grained style conditioning that includes a licensed synthetic dataset and a song generation model, Muse. |
| Outcome: | The proposed system achieves competitive performance on phoneme error rate, text–music style similarity, and audio aesthetic quality while enabling controllable segment-level generation across different musical structures. |
Copied to clipboard
| Challenge: | a small learning gap exists between large and small language models . long CoT data and large model responses are not beneficial for small models - a problem that may be due to the small student model's ability to handle distribution shifts. |
| Approach: | They propose a mix distillation strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. |
| Outcome: | The proposed strategy outperforms training on large and small models on short CoT and small model CoT. |
Copied to clipboard
| Challenge: | Extreme-scale language models have shown exceptional performance on a variety of language tasks, but the degree of control offered by these models through pure prompting is limited. |
| Approach: | They propose an inference-time policy adapter which tailors a large base model without fine-tuning it. |
| Outcome: | The proposed model outperforms baseline methods on five challenging text generation tasks and even over GPT-4. |
Copied to clipboard
| Challenge: | Existing text-to-SQL parsers lack the data to perform well with augmented synthetic data. |
| Approach: | They propose a framework that imposes strong typing constraints and incorporates key relationships from schema. |
| Outcome: | The proposed framework improves on the high-quality synthesized SQL and natural language question (NLQ) models have significant accuracy boosts and achieve new state-of-the-art performance on spider. |
Copied to clipboard
| Challenge: | Currently, most research focuses on the bidding algorithms used within auction mechanisms. |
| Approach: | They propose a personalized valuation framework that integrates Large Language Models to incorporate personalized semantic preference into users valuation process. |
| Outcome: | The proposed framework incorporates Large Language Models to incorporate personalized semantic preference into users valuation process. |
Copied to clipboard
| Challenge: | Existing ABSA methods only use one aspect or multiple aspects with the same sentiment polarity . recent studies show that neural network methods can be trained end-to-end and automatically learn important features. |
| Approach: | They propose a large-scale multi-aspect multi-sentiment dataset with two different aspects with different sentiment polarities. |
| Outcome: | The proposed model outperforms the state-of-the-art models on the large-scale dataset . it is based on a novel neural network approach that can be trained end-to-end . |
Copied to clipboard
| Challenge: | Xiaomingbot is a multilingual and multimodal software robot with four capabilities: news generation, news translation, news reading and avatar animation. |
| Approach: | They propose to build a multilingual and multimodal software robot with four inte- gal capabilities: news generation, news translation, news reading and avatar animation. |
| Outcome: | The proposed system generates Chinese news, then reads it in multiple languages and generates an animated avatar reading it. |
Copied to clipboard
| Challenge: | Recent diagnostic datasets on compositional generalization expose severe problems . state-of-the-art models trained on larger and more general datasets show better generalization ability . |
| Approach: | They conduct an empirical analysis by training Transformer models on a variety of training sets with different data factors including dataset scale, pattern complexity, example difficulty, etc. |
| Outcome: | The proposed model training on larger datasets improves on compositional generalization tasks. |
Copied to clipboard
| Challenge: | Recent studies have found that Task-oriented Dialogue systems can be more suitable for human users. |
| Approach: | They propose a framework to optimize ToD systems by leveraging Multiple User SimulaTors. |
| Outcome: | The proposed framework improves performance on multiWOZ with human evaluations and automatic evaluations. |
Copied to clipboard
| Challenge: | Experimental results show that the distilled language model outperforms its teacher model (ChatGPT) in most cases. |
| Approach: | They propose a Large Language Model (LLM) that leverages both distilled data from **ChatGPT** and real-world data from**doctors** in the supervised fine-tuning stage. |
| Outcome: | The proposed model outperforms the teacher model in most cases by using additional real-world data and RLMF to align the language model with the merits of both sources. |
Copied to clipboard
| Challenge: | Evaluating Retrieval-Augmented Generation (RAG) in large language models is challenging because benchmarks can quickly become stale. |
| Approach: | They propose a benchmark to evaluate Retrieval-Augmented Generation (RAG) in large language models (LLMs) using timelines and knowledge bases of fictional news events and entities to prevent LLMs from leveraging pretraining knowledge. |
| Outcome: | The proposed benchmark prevents LLMs from leveraging pretraining knowledge and ensures that no prior evidence exists in their training data. |
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination. |
| Approach: | They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns. |
| Outcome: | The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages. |
Copied to clipboard
| Challenge: | Recent studies show that LLM-based agents exhibit superior moral and emotional language performance compared to humans, raising expectations for their deployment in persuasive tasks. |
| Approach: | They propose a framework for generating persuasive multi-turn dialogues via agent self-play using user agents designed to simulate diverse persona-driven behaviors, a Dialog Agent executing task-oriented persuasion strategies and an Optimization Agent evaluating and refining dialogue outcomes. |
| Outcome: | The proposed framework significantly improved the persuasion capacity of small LLMs, increasing the organic traffic conversion rate by 22.4% (from 1.83% to 2.24%) . |
Copied to clipboard
| Challenge: | Cognitive task analysis (CTA) is a type of analysis used to elicit and represent the knowledge and thought processes of domain experts. |
| Approach: | They propose a weakly-supervised framework for automated CTA transcript parsing . they partition the parser process into a sequence labeling task and a text span-pair relation extraction task with distant supervision from human-curated protocol files. |
| Outcome: | The proposed framework reduces human labor and scales the task to a small scale. |
Copied to clipboard
| Challenge: | Existing datasets for supervised news summarization contain considerable amount of noise and expensive training data. |
| Approach: | They propose a large-scale and high-quality dataset for supervised abstractive news summarization containing 1.3 million training samples. |
| Outcome: | The proposed dataset is more factual and informative than established summarization datasets. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) do not perform satisfactorily in OOD and adversarial robustness evaluations. |
| Approach: | They propose to use linguistic rule induction to fine-tune large language models with linguistic rules to achieve better adversarial and OOD robustness. |
| Outcome: | The proposed model achieves comparable or better results with GPT-3.5 and GPT-4 on various adversarial and OOD robustness evaluations. |
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications. |
| Approach: | They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions. |
| Outcome: | The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions. |
Copied to clipboard
| Challenge: | Recent efforts to democratize ChatGPT have focused on leveraging real user and ChatGPP dialogues, but the most direct human needs are often ignored. |
| Approach: | They propose a method to simulate human behavior better by using real human-like questions extracted from real human conversations as a learning goal and a user simulator called ‘Socratic’. |
| Outcome: | The proposed model achieves SoTA performance among LLaMA-based 7B models in MT-Bench. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | a recent study shows that open-source large language models (LLMs) exhibit diverse strengths and weaknesses due to variations in their architectures and training data. |
| Approach: | They propose a framework that leverages the diverse strengths of open-source large language models. |
| Outcome: | The proposed framework outperforms individual LLMs and baseline methods across various metrics, establishing a substantial performance gap. |
Copied to clipboard
| Challenge: | Knowledge in natural language processing (NLP) is a rising trend especially after the advent of large scale pre-trained models. |
| Approach: | This tutorial introduces the key steps in integrating knowledge into natural language processing (NLP) it introduces knowledge grounding from text, knowledge representation and fusing. |
| Outcome: | This tutorial introduces the key steps in integrating knowledge into natural language processing including knowledge grounding from text, knowledge representation and fusing. |
Copied to clipboard
| Challenge: | Walk-based models have shown their advantages in knowledge graph reasoning but are limited by their representations and generalizability. |
| Approach: | They propose a walk-based model that leverages high-quality rules generated by symbolic-based methods to provide reward supervision for walk- based agents. |
| Outcome: | Experiments on benchmark datasets show that RuleGuider improves the performance of walk-based models without losing interpretability. |
Copied to clipboard
| Challenge: | Prior work on multimodal fashion tasks has been limited by the data in individual benchmarks or has leveraged generic vision-and-language pre-training but have not taken advantage of the characteristics of fashion data. |
| Approach: | They propose a fashion-specific pre-training framework based on weakly-supervised triplets constructed from fashion image-text pairs. |
| Outcome: | The proposed framework is based on weakly-supervised triplets constructed from fashion image-text pairs and is competitive on a diverse set of fashion tasks. |
Copied to clipboard
| Challenge: | Existing methods to extract sentimental triplets are infeasible and counterproductive . aspect Sentiment Triplets Extraction (ASTE) task is an emerging sub-task of Aspect-based Sentimence Analysis . |
| Approach: | They propose a retrieval-based approach to the Aspect Sentiment Triplet Extraction task . they retrieve semantic similar triplets from the training corpus and interpolate their label information . |
| Outcome: | The proposed approach establishes a new state-of-the-art on the Aspect Sentiment Triplet Extraction task. |
Copied to clipboard
| Challenge: | Existing literature on ambiguity and disambiguation with Large Language Models (LLMs) ambiguities are a fundamental challenge in human-AI interactions due to complexity and flexibility of human language. |
| Approach: | They propose to define key terms and concepts and categorize various disambiguation approaches enabled by LLMs and provide a comparative analysis of their advantages and disadvantages. |
| Outcome: | The proposed frameworks are compared against different disambiguation approaches and highlight their relevance for future research. |
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented text classification are successful in the few-shot scenario with limited retrieval space. |
| Approach: | They propose to use EM-L and R-L to provide task-specific guidance to retrieval metric . they also propose to incorporate retrieved memory alongside parameters for better generalization . |
| Outcome: | The proposed methods perform better on the few-shot scenario with limited retrieval space. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a great breakthrough in medicine, says a new study . existing studies on LLMs leverage subjective evaluation, but evaluation in medicine is professional . |
| Approach: | They propose a localized medical benchmark in Chinese rooted in native Chinese . they propose to use traditional Chinese medicine to evaluate large-scale LLMs . |
| Outcome: | a new benchmark is developed to evaluate large-scale LLMs in china . the proposed model is rooted in the native Chinese linguistic and cultural framework . |
Copied to clipboard
| Challenge: | Recent research on Text-to-SQL semantic parsing relies on parser or heuristic based approach to understand natural language query. |
| Approach: | They propose a general-purpose, modular neural semantic parsing framework that is based on token-level fine-grained query understanding. |
| Outcome: | The proposed framework outperforms the state-of-the-art model by 2.7% on a WikiTableQuestions test set. |
Copied to clipboard
| Challenge: | a new metric measures the percentage of questions that were answered incorrectly during fine-tuning . |
| Approach: | They propose a decoding strategy that draws outputs from multiple checkpoints along the training trajectory. |
| Outcome: | The proposed method improves reasoning performance and consistency across benchmarks. |
Copied to clipboard
| Challenge: | Emerging large reasoning models (LRMs) leverage long chain-of-thought (CoT) reasoning to enhance their reasoning capabilities. |
| Approach: | They conduct a systematic study of LRM safety using human annotations to assess their safety. |
| Outcome: | The proposed safety measures are compared to state-of-the-art models on strong and wildjailbreak datasets. |
Copied to clipboard
| Challenge: | Existing metrics for conditional image generation are opaque and lack explainability . evaluators of these metrics have limited ability to evaluate image synthesis tasks . |
| Approach: | They propose a Visual Instruction-guided Explainable metric for evaluating conditional image models. |
| Outcome: | The proposed model achieves a high Spearman correlation with human evaluations, but is weaker than GPT-4o and GPT-v in evaluating synthetic images. |
Copied to clipboard
| Challenge: | Empirically, we show SQ-Transformer achieves stronger compositional generalization than the vanilla Transformer on low-complexity datasets. |
| Approach: | They propose a Transformer that explicitly encourages systematicity in the embeddings and attention layers even with low-complexity data. |
| Outcome: | Empirically, the proposed model achieves stronger compositional generalization than the vanilla Transformer on low-complexity datasets. |
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |
Copied to clipboard
| Challenge: | Grapheme-to-phoneme conversion datasets suffer from the long-tail problem . context learning for polyphonic characters often stems from a single dimension . |
| Approach: | They propose a model for long-tailed polyphone disambiguation in Mandarin that decouples representation and classification learnings. |
| Outcome: | The proposed model can decouple representation and classification learnings . it achieves transition learning of context from local to global . |
Copied to clipboard
| Challenge: | Existing large language models exhibit unidirectional behavior when processing bidirectional relationships . authors propose a solution to alleviate the reversal curse in Diffusion LLMs . |
| Approach: | They propose a model that addresses the "reversal curse" of bidirectional behavior in large language models . they propose 'entity-aware training' and balanced data construction to alleviate asymmetry and missing relations . |
| Outcome: | The proposed model alleviates the "reversal curse" in Diffusion LLMs . the proposed model employs whole-entity masking to mitigate entity fragmentation . |
Copied to clipboard
| Challenge: | Using large language models, we can understand knowledge mechanisms in LLMs for learning, storage, utilization, and evolution. |
| Approach: | They propose to analyze knowledge mechanisms in Large Language Models (LLMs) they examine utilization, evolution, and the potential dark knowledge (hypothesis) they hope to help understand knowledge in LLMs and provide insights for future research . |
| Outcome: | The proposed model can be used to analyze the evolution of parametric knowledge in LLMs. |
Copied to clipboard
| Challenge: | Existing methods for Sequence Labeling require high-quality annotations, but imperfect annotations are relatively easy to obtain from crowdsourcing (noisy labels) Existing approaches to learn a model without knowing the underlying ground truth label sequences in the target domain are expensive and time-consuming. |
| Approach: | They propose a framework Consensus Network that can be trained on annotations from multiple sources. |
| Outcome: | The proposed framework improves on learning with crowd annotations and unsupervised cross-domain model adaptation in two practical settings. |
Copied to clipboard
| Challenge: | Recent datasets expose the lack of systematic generalization ability in standard sequence-to-sequence models. |
| Approach: | They propose two techniques to address the lack of systematic generalization ability in standard sequence-to-sequence models by mutual exclusivity training and prim2primX data augmentation. |
| Outcome: | The proposed methods improve on two widely-used compositionality datasets. |
Copied to clipboard
| Challenge: | Existing pruning methods for Large Language Models rely on unstructured pruning or require special hardware to accelerate computation. |
| Approach: | They propose a retraining-free structured pruning method called SoBP . they evaluate the effectiveness of SoBP across 14 models from 3 LLM families . |
| Outcome: | The proposed method outperforms current state-of-the-art pruning methods on 8 datasets. |
Copied to clipboard
| Challenge: | Existing datasets are limited due to the difficulty of collecting precise molecule-description pairs. Existing approaches to enhance large language models include a data augmentation framework and a new dataset called SAPubChem-41. |
| Approach: | They propose a framework that interweaves model fine-tuning and data augmentation to overcome the scarcity of high-quality labeled data. |
| Outcome: | The proposed framework interweaves model fine-tuning and data augmentation to overcome the scarcity of high-quality labeled data. |
Copied to clipboard
| Challenge: | Existing model editing methods focus on single-round editing and often face significant challenges in sequential model editing. |
| Approach: | They propose a model editing method that optimizes the target layer’s hidden states using the model’s original weights to prevent model failure. |
| Outcome: | The proposed method outperforms existing model editing methods and is available on the open-source platform 4open.science. |