Papers by Tianyu Jiang

42 papers
Learning Prototypical Functions for Physical Artifacts (2021.acl-long)

Copied to clipboard

Challenge: a new task is designed to learn the prototypical uses of human-made physical objects . human beings are creative, and they create things for a reason . humans often infer that the object will be used in the most prototypical way unless told otherwise .
Approach: They propose a task to learn the prototypical uses for human-made physical objects . they use frames from FrameNet to represent a set of common functions for objects based on their prototypical function .
Outcome: The proposed task uses masked patterns to model prototypical uses for objects . the proposed model predicts the prototypical functions of objects and can be used to make models .
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models (2023.emnlp-demo)

Copied to clipboard

Challenge: MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements.
Approach: a new system is built to integrate music-related tools and an autonomous workflow . the system is based on large language models (LLMs) that can be used to organize and decompose requests .
Outcome: the proposed system integrates numerous music-related tools and an autonomous workflow to address user requirements.
DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlog (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing work on question-answer extraction fails to integrate incomplete utterances from dialog context for composite QA retrieval.
Approach: They propose a task where questions and corresponding answers might be separated across different utterances.
Outcome: The proposed methods perform well on 5 customer service datasets and set a benchmark for N-to-N DialogQAE with utterance and session level evaluation metrics.
Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels (2023.acl-long)

Copied to clipboard

Challenge: Several recent papers claim to have achieved human parity at sentence-level machine translation.
Approach: They propose to use a dataset with rich discourse annotations to evaluate MT performance . they find that MT outputs differ fundamentally from human translations in terms of latent discourse structures.
Outcome: The proposed dataset builds upon the large-scale parallel corpus BWB . it covers 15,095 entity mentions in both languages and compares them to human translations .
Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models exhibit strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC) when instructions are revealed progressively in multi-turn settings, models get "Lost in Conversation"
Approach: They propose a framework that encourages models to generate correct answers and judge solvability in multi-turn conversations.
Outcome: The proposed framework improves models' ability to balance problem-solving with abstention . it reduces premature answering behaviors that cause lost-in-conversation (LiC)
LaMP-Val: Large Language Models Empower Personalized Valuation in Auction (2025.findings-emnlp)

Copied to clipboard

Challenge: Currently, most research focuses on the bidding algorithms used within auction mechanisms.
Approach: They propose a personalized valuation framework that integrates Large Language Models to incorporate personalized semantic preference into users valuation process.
Outcome: The proposed framework incorporates Large Language Models to incorporate personalized semantic preference into users valuation process.
Autoregressive Structured Prediction with Language Models (2022.findings-emnlp)

Copied to clipboard

Challenge: Recent years have seen a paradigm shift in NLP towards using pretrained language models for a wide range of tasks.
Approach: They propose to model structures as sequences of actions in autoregressive manner with PLMs . their approach allows in-structure dependencies to be learned without any loss .
Outcome: The proposed approach achieves state-of-the-art on all structured prediction tasks.
Rhetorical Questions in LLM Representations: A Linear Probing Study (2026.acl-long)

Copied to clipboard

Challenge: Rhetorical questions are asked not to seek information, but to persuade or signal stance . how large language models internally represent rhetorical questions remains unclear .
Approach: They analyze rhetorical questions in LLM representations using linear probes on two social-media datasets with different discourse contexts.
Outcome: The results show that rhetorical signals emerge early and are most stably captured by last-token representations.
Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment (2022.acl-long)

Copied to clipboard

Challenge: Existing methods to predict missing facts in knowledge graphs are limited in language alignment . SS-AGA uses seed alignment as an edge type to fuses all KGs as a whole graph .
Approach: They propose a self-supervised adaptive graph alignment method that fuses all KGs as a whole graph by regarding alignment as 'a new edge type' they propose SS-AGA method that uses relation-aware attention weights to capture potential alignment pairs in a new paradigm.
Outcome: The proposed method can predict missing facts in a knowledge graph (KG) but language alignment is scarce and new alignment identification is noisy.
Exploiting Definitions for Frame Identification (2021.eacl-main)

Copied to clipboard

Challenge: a frame-semantic parsing task is to determine which frame best captures the meaning of a word or phrase in a sentence.
Approach: They propose a frame identification model that generates representations for frames and lexical units (senses) they evaluate the model on three data sets and show it consistently achieves better performance than previous systems.
Outcome: The proposed model consistently outperforms previous systems on three data sets.
Identifying Physical Object Use in Sentences (2022.emnlp-main)

Copied to clipboard

Challenge: Prior research has focused on learning the prototypical functions of physical objects . but many sentences refer to objects even when they are not used .
Approach: They propose a task that determines whether a physical object mentioned in a sentence was used or likely will be used.
Outcome: The proposed model exploits data augmentation methods and FrameNet to fine-tune a pre-trainedmodel.
M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation (2026.acl-long)

Copied to clipboard

Challenge: prevailing methods for machine translation are often hindered by misleading reward signals.
Approach: They propose a framework that aligns large language models to human preferences . they propose 'M2PO' to correct the bias towards partial errors .
Outcome: The proposed framework outperforms open-source models and achieves parity with proprietary models.
GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Large language models excel at factual recall, arithmetic reasoning, multi-turn dialogue . their capacity as askers, formulating strategic, adaptive, and information-seeking questions, remains less explored .
Approach: They propose a protocol for evaluating large language models as strategic question-askers . they propose entropy-based methods that filter candidates via ConceptNet and Bayesian method that tracks belief updates over semantic concepts .
Outcome: The proposed method is model-agnostic and supports post hoc analysis.
Exploring Concreteness Through a Figurative Lens (2026.acl-long)

Copied to clipboard

Challenge: Static concreteness ratings are widely used in NLP, yet a word’s concreteness can shift with context, especially in figurative language such as metaphor, where common concrete nouns can take abstract interpretations.
Approach: They conduct a layer-wise and geometric analysis of LLM hidden representations across four model families to examine how models distinguish literal vs. figurative usage.
Outcome: The results show that LLMs separate literal and figurative usage in early layers and that mid-to-late layers compress concreteness into a one-dimensional direction consistent across models.
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)

Copied to clipboard

Challenge: Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages.
Approach: They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English.
Outcome: The proposed model outperforms open-source and Tibetan-focused models on diverse tasks.
CHEER-Ekman: Fine-grained Embodied Emotion Classification (2025.acl-short)

Copied to clipboard

Challenge: Emotions manifest through physical experiences and bodily reactions, yet identifying such embodied emotions in text remains understudied.
Approach: They propose to extend existing binary embodied emotion dataset with Ekman’s six basic emotion categories.
Outcome: The proposed dataset outperforms existing methods with large language models.
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data (2021.acl-long)

Copied to clipboard

Challenge: Existing work focuses on learning deep NER models with weak supervision without any human annotation.
Approach: They propose a framework that can suppress the noise of the weak labels and fine-tune over the strongly labeled data.
Outcome: The proposed framework outperforms existing methods on Named Entity Recognition tasks with weak supervision and weakly labeled data.
Towards Robust Evaluation of Visual Activity Recognition: Resolving Verb Ambiguity with Sense Clustering (2026.findings-eacl)

Copied to clipboard

Challenge: Existing evaluation methods for visual activity recognition systems fail to capture ambiguities in verb semantics and image interpretation.
Approach: They propose a framework that constructs verb sense clusters to evaluate visual activity recognition systems.
Outcome: The proposed framework provides a more robust evaluation of visual activity recognition systems.
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Copied to clipboard

Challenge: Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach: They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome: The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.
THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption (2022.findings-acl)

Copied to clipboard

Challenge: enabling pre-trained models inference on ciphertext data is difficult due to the complex computations in transformer blocks.
Approach: They propose an approximation approach for transformers which enables inference on ciphertext data.
Outcome: The proposed approach can infer pre-trained models on encrypted data with negligible performance drop but enjoy theory-guaranteed privacy-preserving advantage.
Do Emotions Influence Moral Judgment in Large Language Models? (2026.findings-acl)

Copied to clipboard

Challenge: Recent systems enforce explicit ethical constraints, but moral judgment rarely involves such clear-cut prohibitions.
Approach: They develop an emotion-induction pipeline that infuses emotion into moral situations and evaluate shifts in moral acceptability across datasets and LLMs.
Outcome: The proposed pipeline can infuses emotion into moral situations and evaluate moral acceptability shifts across datasets and LLMs.
Learning Prototypical Goal Activities for Locations (P18-1)

Copied to clipboard

Challenge: a goal-act is an activity that represents a common reason people go to a location . recognizing goals is essential for narrative text understanding and story comprehension .
Approach: They use a text corpus and semi-supervised learning to learn goal-acts for specific locations . they extract activities and locations that co-occur in goal-oriented syntactic patterns .
Outcome: The proposed method outperforms baseline methods when judged against goal-acts identified by human annotators.
Anatomy of a Feeling: Narrating Embodied Emotions via Large Vision-Language Models (2025.findings-emnlp)

Copied to clipboard

Challenge: ELENA is a framework for embodied emotion analysis using large vision language models . ELEna uses attention maps and a persistent bias towards the facial region .
Approach: They propose a framework that utilizes large vision language models to generate ELENA . they propose to use attention maps to describe emotional reactions from body parts .
Outcome: The proposed framework outperforms baseline models without fine-tuning . it uses large vision language models to generate embodied emotion narratives .
ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL (2026.acl-long)

Copied to clipboard

Challenge: Current Text-to-SQL reasoning models lack integrated execution feedback during generation.
Approach: They propose a text-to-SQL framework that interacts with the SQL execution engine during decoding and dynamically adjusts reasoning based on execution feedback.
Outcome: The proposed framework achieves 89.1% accuracy on Spider and 65.3% on BIRD at the 7B scale.
Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs (2026.eacl-long)

Copied to clipboard

Challenge: Existing studies treat prompts as flat text, overlooking their internal structure, and different components within a prompt contribute unequally to robustness.
Approach: They propose a framework that decomposes prompts into functional components and a method that selectively modifies components to expose component-wise vulnerabilities.
Outcome: The proposed framework exposes component-wise vulnerabilities while ensuring linguistic plausibility through perplexity-based filtering.
Exploiting Commonsense Knowledge about Objects for Visual Activity Recognition (2023.findings-acl)

Copied to clipboard

Challenge: Existing tasks that aim to identify the objects in an image are object detection and image classification, but recent work has focused on more comprehensive image under- standing tasks.
Approach: They propose to incorporate commonsense knowledge about physical objects into a transformer-based model that is trained to predict the actionverb for visual activity recognition.
Outcome: The proposed model incorporates prototypical function knowledge about physical objects to predict the actionverb for visual activity recognition.
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation (2022.naacl-main)

Copied to clipboard

Challenge: Standard evaluation metrics, e.g., BLEU, TER and METEOR, focus on the quality of translations at the sentence level and do not consider discourse-level features.
Approach: They propose to use a metric to take discourse coherence into consideration by categorizing discourse-related spans and calculating the similarity-based F1 measure of categorized spans.
Outcome: The proposed metric possesses better selectivity and interpretability at the document-level, and is more sensitive to document- level nuances.
A Structured Span Selector (2022.naacl-main)

Copied to clipboard

Challenge: a typical approach to natural language processing tasks involves selecting text spans and making decisions about them.
Approach: They propose a grammar-based structured span selection model which learns to make use of partial span annotations.
Outcome: The proposed model improves on two popular span prediction tasks.
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Copied to clipboard

Challenge: Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach: They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome: The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.
IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters (2026.acl-industry)

Copied to clipboard

Challenge: Existing efforts focus on activation within ongoing dialogues, while overlooking a key real-world bottleneck.
Approach: They propose a conversation starter generation system that generates personalized starters to guide users into conversation without explicit user intent.
Outcome: The proposed system improves user active days by +1.84 and click-through rate by +94.25 and has been deployed in production.
Affective Event Classification with Discourse-enhanced Self-training (2020.emnlp-main)

Copied to clipboard

Challenge: Prior work on recognizing affective events focused on producing lexical resources of verbs or event phrases with corresponding affective polarity values.
Approach: They propose a BERT-based model for affective event classification and a discourse-enhanced self-training method that iteratively improves the classifier with unlabeled data.
Outcome: The proposed model outperforms existing models with unlabeled data and improves recall and precision.
Do LLMs Encode Frame Semantics? Evidence from Frame Identification (2025.emnlp-main)

Copied to clipboard

Challenge: Using the FrameNet lexical resource, we evaluate large language models under prompt-based inference and observe that they can perform frame identification effectively even without explicit supervision.
Approach: They evaluate large language models under prompt-based inference and observe that they encode latent knowledge of frame semantics.
Outcome: The proposed model can generate coherent frame definitions while generalizing well to out-of-domain benchmarks.
Evaluating the Impact of Verbal Multiword Expressions on Machine Translation (2026.acl-long)

Copied to clipboard

Challenge: Verbal multiword expressions (VMWEs) are difficult for machine translation because their meanings are often not recoverable from their component words.
Approach: They analyze the impact of verbal idioms, verb-particle constructions, and light verb constructions on machine translation quality from English to multiple languages.
Outcome: The proposed system improves translation quality by focusing on verb idioms, verb-particle constructions and light verb constructions.
LSEG: A Fine-tuning Free Method for NL2FOL via Logic-Structure and Entropy Guided Inference Controlling (2026.findings-acl)

Copied to clipboard

Challenge: Large language models struggle with natural language to first order logic (NL2FOL) translation due to logical hallucination.
Approach: They propose a fine-tuning free framework to correct hidden state deviation by leveraging logical stability across logic preserving perturbations of the input.
Outcome: The proposed framework improves logical consistency during inference and improves accuracy over baselines.
My Heart Skipped a Beat! Recognizing Expressions of Embodied Emotion in Natural Language (2024.naacl-long)

Copied to clipboard

Challenge: a new task is needed to recognize physical manifestations of emotions in natural language . physical manifestation of emotions affects not only our mental state but also our physical state .
Approach: They propose a task to recognize expressions of embodied emotion in natural language . they use body part mentions with human annotations to extract emotional manner expressions .
Outcome: The proposed model can train without gold data and improve performance with gold data.
ConMeC: A Dataset for Metonymy Resolution with Common Nouns (2025.naacl-long)

Copied to clipboard

Challenge: Prior work on metonymy resolution has focused on named entities, but common nouns are also a frequent problem.
Approach: They propose a dataset that combines a metonymy dataset and a chain-of-thought based prompting method for detecting metonyms using large language models.
Outcome: The proposed method can detect metonymy using large language models while still struggling with nuanced semantic understanding.
Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation (2026.acl-long)

Copied to clipboard

Challenge: Current systems often fall short of this goal in settings where translation hinges on culturally grounded entities such as books, films, places, songs and idioms.
Approach: They propose a framework that anchors supervision on a verifiable, entity-level reward signal and incorporates lightweight structural gates to stabilize optimization.
Outcome: The proposed framework improves on XC-Translate and shows that it can learn a robust reasoning process rather than imitating reference translations.
SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment (2026.findings-acl)

Copied to clipboard

Challenge: Low-resource language tokens are often routed to different experts than those activated by high-resourced inputs, which hinders their efficacy in multilingual contexts.
Approach: They propose a framework to transfer specialized capabilities from high-resource languages as anchors to low-resourced languages by using a symmetric Jensen-Shannon constraint.
Outcome: The proposed framework outperforms standard instruction tuning on 5 low-resource languages and 3 benchmarks.
Bridging the Temporal Gap in Multimodal LLMs: Deeply Stacking Temporal Tokens for Audio-Visual Speech Recognition (2026.findings-acl)

Copied to clipboard

Challenge: Existing audio-visual speech recognition systems suffer from a temporal gap . visual speech patterns captured from lip movements provide complementary information that remains inherently robust to acoustic noise.
Approach: They propose a framework that deeply stacks temporal tokens across both encoding and decoding stages to bridge this temporal gap.
Outcome: The proposed framework outperforms existing supervised, self-supervised, and LLM-based methods by 6.1% on LRS2 and 7.8% on LLS3.
MetFuse: Figurative Fusion between Metonymy and Metaphor (2026.acl-long)

Copied to clipboard

Challenge: Metonymy and metaphor are two fundamental linguistic phenomena in figurative language that involve concept mapping.
Approach: They propose a framework that transforms a literal sentence into three figurative variants . they propose 'metonymic, metaphoric, and hybrid' datasets that can be used to map metonymy and metaphor .
Outcome: The proposed framework transforms a literal sentence into three figurative variants . hybrid examples yield the largest gains on metonymy tasks, the study shows .
A Computational Approach to Visual Metonymy (2026.eacl-long)

Copied to clipboard

Challenge: Visual metonymy is a form of indirect representation in which an image evokes a concept not by depicting it directly, but by presenting visually associated cues that invite the viewer to infer the intended meaning.
Approach: They propose a pipeline grounded in semiotic theory that leverages large language models and text-to-image models to generate metonymic visual representations.
Outcome: The proposed pipeline exploits large language models and text-to-image models to generate metonymic visual representations.
Neuron-Level Sequential Editing for Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Existing model editing methods focus on single-round editing and often face significant challenges in sequential model editing.
Approach: They propose a model editing method that optimizes the target layer’s hidden states using the model’s original weights to prevent model failure.
Outcome: The proposed method outperforms existing model editing methods and is available on the open-source platform 4open.science.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations