Papers by Kenny Zhu
Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained masked language models inherit a considerable amount of relational knowledge from the source corpora. |
| Approach: | They propose to specialize pretrained masked language models into relational models from the perspective of network pruning. |
| Outcome: | The proposed model can represent grounded commonsense relations at non-trivial sparsity while being generalizable . the proposed model improves on a wealth of NLP tasks, but we know little about how much knowledge it imparts . |
Multi-turn Response Selection using Dialogue Dependency Relations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-turn response selection ignore the dependencies between the turns. |
| Approach: | They propose a dialogue extraction algorithm to transform a dialog history into threads based on their dependency relations. |
| Outcome: | The proposed model outperforms the state-of-the-art models on DSTC7 and DSTF8* with competitive results on UbuntuV2 . |
Leaner and Faster: Two-Stage Model Compression for Lightweight Text-Image Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing text-image approaches use pre-trained vision-language representations for text retrieval . however, these models pose non-trivial memory requirements and substantial indexing time . |
| Approach: | They propose a framework to compress large pre-trained dual-encoders for lightweight text-image retrieval. |
| Outcome: | The proposed model performs better on Flickr30K and MSCOCO benchmarks than the original full model on mobile devices. |
Opinion Summarization by Weak-Supervision from Mix-structured Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization of multiple reviews lack reference summaries . OAs and ISs are often mismatched between review input and summary . |
| Approach: | They propose a method to generate mixed-structured synthetic training data for opinion summarization. |
| Outcome: | The proposed method outperforms existing methods on Yelp, Amazon and RottenTomatos datasets. |
Transcribing Vocal Communications of Domestic Shiba lnu Dogs (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study has focused on how animals communicate, but the study has been limited . previous studies have focused on a simple classification problem, requiring a model to get a label . |
| Approach: | They extract Shiba Inu dogs' vocal communications from YouTube videos and translate them into phonetic scripts using a systematic process. |
| Outcome: | The proposed framework produces the first-of-its-kind Shiba Inu vocal communication dataset . it will be useful for future research in zoology and linguistics. |
Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual sequence labeling require bilingual resources and require linguistic knowledge. |
| Approach: | They propose a multilingual language model with deep semantic Alignment to generate language-independent representations for cross-lingual sequence labeling. |
| Outcome: | The proposed model achieves state-of-the-art NER and POS performance across European languages and on distant language pairs such as English and Chinese. |
Semantic Space Grounded Weighted Decoding for Multi-Attribute Controllable Dialogue Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Controlling chatbot utterance generation with multiple attributes is a useful but under-studied problem. |
| Approach: | They propose a framework that possesses strong controllability with a weighted decoding paradigm and improves generation quality with an attribute semantics space. |
| Outcome: | The proposed framework achieves high control accuracy with simultaneous control of 3 aspects while producing interesting and sensible responses even in an out-of-distribution robustness test. |
Mining Cross-Cultural Differences and Similarities in Social Media (P18-1)
Copied to clipboard
| Challenge: | a new paper examines the problem of computing cross-cultural differences and similarities in natural language understanding . cross-culture differences are important for cross-lingual research, especially in social media . |
| Approach: | They propose a framework for computing cross-cultural differences and similarities from social media . they propose to use a social media platform to find similar terms for slang across languages . |
| Outcome: | The proposed framework outperforms baseline methods on two novel tasks. |
Detection of Multiple Mental Disorders from Social Media with Two-Stream Psychiatric Experts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing mental disease detection methods are not backed by domain knowledge and thus fail to produce interpretable results. |
| Approach: | They propose a framework that can learn the shared clues of all diseases while also capturing the specificity of each single disease. |
| Outcome: | Experiments on the detection of 7 diseases show that the proposed model can boost detection performance by more than 10%, especially in relatively rare classes. |
Post-Training Dialogue Summarization using Pseudo-Paraphrasing (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to dialogue summarization use dialogue-specific features that require additional knowledge to recognize or make the models harder to tune. |
| Approach: | They propose to post-train pretrained language models to rephrase from dialogue to narratives and fine-tune them as usual. |
| Outcome: | The proposed approach outperforms existing models by summary quality and implementation costs. |
Incomplete Utterance Rewriting by A Two-Phase Locate-and-Fill Regime (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models with incomplete utterances have too large search space, resulting in poor quality of rewriting results. |
| Approach: | They propose a 2-phase rewriting framework which predicts empty slots in the utterance that need to be completed and generates the part to be filled into each position. |
| Outcome: | The proposed framework achieves state-of-the-art results on several public rewriting datasets. |
Phonetic and Lexical Discovery of Canine Vocalization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to study animal language systems rely on human prior knowledge on limited data. |
| Approach: | They propose a self-supervised approach that enables the accurate classification of phones and an adaptive grammar induction method that identifies phone sequence patterns that suggest a preliminary vocabulary within dog vocalizations. |
| Outcome: | The proposed approach breaks the barrier existing approaches relying on human prior knowledge on limited data. |
Knowledge Base Question Answering via Encoding of Complex Query Graphs (D18-1)
Copied to clipboard
| Challenge: | Existing KBQA methods focus on simpler questions and do not work well on complex questions . a knowledge-based question answering approach is able to answer complex questions using a standard knowledge base . |
| Approach: | They propose to encode query structure into a uniform vector representation of a question and its semantic components into . |
| Outcome: | The proposed approach outperforms existing methods on complex questions while staying competitive on simple questions. |
Automatic Reconstruction of Ancient Chinese Pronunciations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | A human language is comprised of a pronunciation system and a writing system, both evolving and changing over time. |
| Approach: | They reformulate existing phonetic rules into a dataset of 70,943 entries for 17,001 Chinese characters and use it to perform a temporal prediction task. |
| Outcome: | The transformer-based model significantly advances the digitization and computational reconstruction of ancient Chinese phonology, providing a more complete and temporally contextualized resource for computational linguistics and historical research. |
ExtRA: Extracting Prominent Review Aspects from Customer Feedback (D18-1)
Copied to clipboard
| Challenge: | Existing methods for analyzing and summarizing customer reviews are based on a number of prominent review aspects. |
| Approach: | They propose a framework for extracting the most prominent aspects of a given product type from textual reviews. |
| Outcome: | The proposed framework extracts K most prominent aspect terms which do not overlap semantically without supervision. |
Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work evaluates faithfulness using models trained on related tasks or in-domain synthetic data. |
| Approach: | They propose to do zero-shot faithfulness evaluation with a foundation language model. |
| Outcome: | The proposed model outperforms ChatGPT on faithfulness and inconsistency detection with 24x fewer parameters and is competitive with existing models. |
Pruning Pre-trained Language Models with Principled Importance and Self-regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models often contain a vast amount of parameters, posing nontrivial requirements for storage and computation. |
| Approach: | They propose a pruning method where model prediction is regularized by the latest checkpoint with increasing sparsity throughout pruning. |
| Outcome: | The proposed approach is effective at sparsity levels, and can be applied to natural language understanding, question answering, and data-to-text generation tasks. |
Statistically Profiling Biases in Natural Language Reasoning Datasets and Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to evaluate NLP models' weaknesses are limited by “hypothesis-only” tests and CheckLists. |
| Approach: | They propose a lightweight general statistical profiling framework that automatically identifies potential biases in multiple-choice NLU datasets without requiring additional test cases. |
| Outcome: | The proposed framework assesses the extent to which models exploit these biases through black-box testing, confirming prior findings and revealing new insights. |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
Adaptive Multi-Task Transfer Learning for Chinese Word Segmentation in Medical Text (C18-1)
Copied to clipboard
| Challenge: | Chinese word segmentation (CWS) tools face a performance drop when dealing with domain text . domain-specific CWS requires extremely high annotation cost due to ambiguity caused by domain terms and writing style . |
| Approach: | They propose to exploit domain-invariant knowledge from high resource to low resource domains to build Chinese word segmentation models. |
| Outcome: | The proposed model achieves higher accuracy than single-task CWS and other transfer learning baselines . the model is based on domain-invariant knowledge from high resource to low resource domains based in the biomedical domain . |
ChatMatch: Evaluating Chatbots by Autonomous Chat Tournaments (2022.acl-long)
Copied to clipboard
| Challenge: | Existing automated evaluation systems of chatbots rely on static chat scripts as ground truth, which is hard to obtain. |
| Approach: | They propose an interactive chatbot evaluation framework that allows chatbots to compete with each other like in a sports tournament. |
| Outcome: | The proposed framework can rank chatbots independently from their model architectures and domains . existing evaluation systems rely on static chat scripts as ground truth . |
Reducing Sensitivity on Speaker Names for Text Generation from Dialogues (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models are sensitive to nuances, resulting in unfairness in real-world applications. |
| Approach: | They propose to quantitatively measure a model's sensitivity on speaker names and comprehensively evaluate a number of known methods for reducing speaker name sensitivity. |
| Outcome: | The proposed approach reduces speaker name sensitivity and improves quality of generation. |
Automatic Extraction of Commonsense LocatedNear Knowledge (P18-2)
Copied to clipboard
| Challenge: | LocatedNear relation is a kind of commonsense knowledge describing two physical objects that are typically found near each other in real life. |
| Approach: | They propose to automatically extract LocatedNear relation from corpus by a sentence-level relation classifier and aggregating scores of entity pairs from a large corpus. |
| Outcome: | The proposed method can be used to extract the commonsense LOCATEDNEAR relation from a large corpus. |
Transferable and Efficient: Unifying Dynamic Multi-Domain Product Categorization (2023.acl-industry)
Copied to clipboard
| Challenge: | e-commerce platforms are encountering increasingly complex product categorization scenarios . multiple business domains correspond to different category taxonomies, with different depths and distinct literal expressions of category names. |
| Approach: | They propose a taxonomy-agnostic framework that calculates semantic relatedness between product titles and category names in the vector space. |
| Outcome: | The proposed framework outperforms strong baselineson three dynamic multi-domain product categorization tasks. |
Context Compression for Auto-regressive Transformers with Sentinel Tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing Transformer-based LLMs have limited performance due to complexity of attention module . key-value cache is the major memory footprint and inference latency problem . |
| Approach: | They propose a plug-and-play approach that incrementally compresses token activation into compact ones . they also profile the benefit of context compression on improving the system throughout . |
| Outcome: | The proposed approach reduces memory footprint and inference latency by compressing tokens into compact ones. |
Length Control in Abstractive Summarization by Pretraining Information Selection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing length-controllable summarization models generate summaries as long as training data . current methods only control lengths at decoding stage, but adapt to desired lengths . |
| Approach: | They propose a length-aware attention mechanism to adapt the encoding of the source based on the desired length. |
| Outcome: | The proposed method produces high-quality summaries with desired lengths and even those short lengths never seen in the training data. |
Mapping Long-term Causalities in Psychiatric Symptomatology and Life Events from Social Media (2024.naacl-long)
Copied to clipboard
Siyuan Chen, Meilin Wang, Minghao Lv, Zhiling Zhang, Juqianqian Juqianqian, Dejiyangla Dejiyangla, Yujia Peng, Kenny Zhu, Mengyue Wu
| Challenge: | Existing studies focus on the semantic content of social media posts, overlooking the evolving nature of mental disorders and symptoms. |
| Approach: | They extract causality between psychiatric symptoms and life events from social media posts and extract temporal attributes to improve diagnosis and treatment planning. |
| Outcome: | The extracted causality features improve diagnostic and treatment planning and improve performance in tasks such as depression and diagnosis point detection. |
Symptom Identification for Interpretable Detection of Multiple Mental Disorders on Social Media (2022.emnlp-main)
Copied to clipboard
| Challenge: | Mental disease detection (MDD) from social media has suffered from poor generalizability and interpretability due to lack of symptom modeling. |
| Approach: | They propose to annotate a social media corpus of symptom classes related to 7 mental disorders using a knowledge graph and a new annotation framework to facilitate further research. |
| Outcome: | The proposed model outperforms strong pure-text baselines and provides convincing MDD explanations with case studies. |
Controlling Length in Abstractive Summarization Using a Convolutional Neural Network (D18-1)
Copied to clipboard
| Challenge: | Convolutional neural networks (CNNs) can't generate summaries of desired lengths due to space or length constraints. |
| Approach: | They propose an approach to constrain the summary length by extending a convolutional sequence to sequence model. |
| Outcome: | The proposed model outperforms baseline models in terms of ROUGE score, length variations and semantic similarity. |
Reference-free Summarization Evaluation via Semantic Correlation and Compression Ratio (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for summarization use human annotations as reference. |
| Approach: | They propose a new automatic reference-free evaluation metric that compares semantic distribution between source document and summary by pretrained language models and considers summary compression ratio. |
| Outcome: | The proposed metric is more consistent with human evaluation in terms of coherence, consistency, relevance and fluency. |