Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Parameter-efficient (PE) methods for adapting pre-trained language models to downstream tasks are still lacking in many cases. |
| Approach: | They propose a general PE priming framework to enhance few-shot adaptation and generalization ability of PE methods. |
| Outcome: | The proposed framework reveals that the best priming strategy facilitates adaptation to target tasks. |
Rephrasing Invokes Better Generations for Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning and input pre-processing are under-studied . e.g., ReLLM replaces low-frequency words with their high-frequency counterparts . |
| Approach: | They propose a method that automatically paraphrases input content for better output generation. |
| Outcome: | The proposed method is user-friendly and requires no additional training. |
Exploring Compositional Generalization of Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | a recent study has found that large language models can generalize compositional instructions from simple instructions to complex ones. |
| Approach: | They study the generalization ability of large language models with respect to compositional instructions . they first construct a dataset with the help of ChatGPT guided by the self-instruct technique . |
| Outcome: | The proposed model can generalize from simple instructions to more intricate ones, the authors show . their results show that training LLMs on higher-order compositional instructions improves performance on lower-order ones, but not on higher order ones. |
Explainable CED: A Dataset for Explainable Critical Error Detection in Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing studies of critical error detection lack content addressing the causes of catastrophic errors. |
| Approach: | They propose a dataset that introduces the attributes of error explanation and correction regarding critical errors. |
| Outcome: | The proposed dataset reduces time costs and mitigates human annotation bias. |
SMARTR: A Framework for Early Detection using Survival Analysis of Longitudinal Texts (2024.naacl-srw)
Copied to clipboard
| Challenge: | a paper aims to detect expensive insurance claims early using textual information from claims notes. |
| Approach: | They propose a model that leverages survival analysis concepts from claims notes to enhance a posteriori classification and early detection. |
| Outcome: | The proposed model improves classification and early detection without reducing performance . it is based on a privately held corpus of claim files from a Canadian insurer . |
Fast Exact Retrieval for Nearest-neighbor Lookup (FERN) (2024.naacl-srw)
Copied to clipboard
| Challenge: | Exact nearest neighbor search is computationally intensive and complex . Attention has shifted towards Approximate Nearest-Neighbor (ANN) retrieval techniques . |
| Approach: | They propose an algorithm for logarithmic Fast Exact Retrieval for Nearest-neighbor lookup which achieves O(dlog N) look-up with 100% recall on 10 million d=128 uniformly generated vectors. |
| Outcome: | The proposed algorithm achieves O(dlog N) look-up with 100% recall on 10 million d=128 uniformly generated vectors. |
Start Simple: Progressive Difficulty Multitask Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | a novel neural network training strategy that trains neural networks using subtasks of progressive difficulty is proposed . this strategy could help us understand how neural networks learn, authors say . |
| Approach: | They propose a multitask learning strategy that employs progressive difficulty subtasks to train neural networks. |
| Outcome: | The proposed strategy can improve model performance across a range of NLP tasks and data sets. |
LUCID: LLM-Generated Utterances for Complex and Interesting Dialogues (2024.naacl-srw)
Copied to clipboard
Joe Stacey, Jianpeng Cheng, John Torr, Tristan Guigue, Joris Driesen, Alexandru Coca, Mark Gaynor, Anders Johannsen
| Challenge: | Existing datasets with limited domain coverage and few challenging conversational phenomena are often unlabelled . Existing data is limited in quality and lacks a robust evaluation process . |
| Approach: | They propose a high quality data generation system that generates high quality dialogues using 4,277 conversations across 100 intents. |
| Outcome: | The proposed system produces high quality dialogue data with high quality labels. |
Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages (2024.naacl-srw)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a useful component in NLP applications. |
| Approach: | They propose to use annotated named entity corpora to classify a given entity into a category within a textual document. |
| Outcome: | The proposed model achieves an F1 score of 0.80 on an unseen dataset for Indian languages. |
Knowledge-centered conversational agents with a drive to learn (2024.naacl-srw)
Copied to clipboard
| Challenge: | Unlike traditional task-oriented dialogue agents, knowledgeable agents can autonomously determine what they know and do not know, what is the epistemic status of what they do not understand, and what they need to learn. |
| Approach: | They propose an adaptive conversational agent that assesses the quality of its knowledge and is driven to become more knowledgeable. |
| Outcome: | The proposed agent can learn effective policies to acquire the knowledge needed by assessing the efficiency of these capabilities during interaction. |
Exploring Inherent Biases in LLMs within Korean Social Context: A Comparative Analysis of ChatGPT and GPT-4 (2024.naacl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been criticized for perpetuating stereotypes against diverse groups based on race, sexual orientation, and other attributes. |
| Approach: | They devised a set of prompts that reflect major societal issues in Korea and assign varied personas to both ChatGPT and GPT-4 to assess the toxicity of the generated sentences. |
| Outcome: | The proposed model produces twice the level of toxic content as ChatGPT and GPT-4 under certain conditions. |
To Clarify or not to Clarify: A Comparative Analysis of Clarification Classification with Fine-Tuning, Prompt Tuning, and Prompt Engineering (2024.naacl-srw)
Copied to clipboard
| Challenge: | Xu et al., 2019) show that pre-trained language model fine-tuning and prompt tuning are better than manual prompt engineering for clarification identification. |
| Approach: | They propose to use pre-trained language model fine-tuning, prompt tuning and manual prompt engineering to model clarification identification. |
| Outcome: | The proposed model outperforms pre-trained language model fine-tuning, prompt tuning and manual prompt engineering on the task of clarification identification. |
Detecting Response Generation Not Requiring Factual Judgment (2024.naacl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) have undergone considerable development and can solve various natural language processing tasks. |
| Approach: | They aimed to achieve both attractiveness and factuality in a dialogue response by crowdsourcing a dataset and performing classification tasks on several models. |
| Outcome: | The proposed model with the highest classification accuracy could yield about 88% accurate classification results. |
Unknown Script: Impact of Script on Cross-Lingual Transfer (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for high-resource languages are not available for all languages, and the vast majority of the world's languages are excluded from these models. |
| Approach: | They propose to use pre-trained models to analyze the effect of the target language and its script on cross-lingual transfer. |
| Outcome: | The proposed model is based on six models pre-trained on NER and POS tasks in the original script and romanized version. |
Improving Repository-level Code Search with Text Conversion (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to search for related files based on similarity between code snippets are not effective for repository-level code generation. |
| Approach: | They propose to take similarities between code snippets and the texts converted from them into LLMs to search for related files and perform generation. |
| Outcome: | The proposed method improves the accuracy of code search on the repository level. |
Improving Multi-lingual Alignment Through Soft Contrastive Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to train multi-lingual sentence embeddings ruins the mono-lingual space. |
| Approach: | They propose a method to align multi-lingual embeddings based on similarity of sentences measured by a pre-trained mono-lingual teacher model. |
| Outcome: | The proposed method outperforms existing multi-lingual embeddings including LaBSE on five languages and on a translation pair for Tatoeba dataset. |
Few-Shot Event Argument Extraction Based on a Meta-Learning Approach (2024.naacl-srw)
Copied to clipboard
| Challenge: | Recent studies on few-shot event extraction focus on event trigger detection and argument extraction in meta-learning contexts. |
| Approach: | They propose to use prototypical networks to perform few-shot event argument extraction . they propose to inject syntactic knowledge into the model to enhance relation embeddings . |
| Outcome: | The proposed approach achieves strong performance on ACE 2005 in several few-shot configurations. |
Investigating Web Corpus Filtering Methods for Language Model Development in Japanese (2024.naacl-srw)
Copied to clipboard
| Challenge: | a high quality web corpus is essential for large language models to be developed . strong filtering methods can lead to lesser performance in downstream tasks . |
| Approach: | They build classifiers and language models that can process large amounts of corpora quickly enough for pretraining LLMs. |
| Outcome: | The proposed method is the most accurate and leads to lesser performance in downstream tasks. |
Referring Expressions in Human-Robot Common Ground: A Thesis Proposal (2024.naacl-srw)
Copied to clipboard
| Challenge: | PhD examines the processes through which common ground shapes the pragmatic use of referring expressions in human-robot interaction. |
| Approach: | They investigate the processes through which common ground shapes the pragmatic use of referring expressions in human-robot interaction. |
| Outcome: | The proposed model can model linguistic processes and resolve referring expressions using pragmatic inference. |
Source Code is a Graph, Not a Sequence: A Cross-Lingual Perspective on Code Clone Detection (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for code clone detection are fast and scalable, but they may fail to detect clones that have different syntax or structure. |
| Approach: | They propose to use graph-based methods for code clone detection instead of sequence-based models. |
| Outcome: | The proposed graph-based methods outperforms sequence-based models on cross-lingual code clones. |
Distilling Text Style Transfer With Self-Explanation From LLMs (2024.naacl-srw)
Copied to clipboard
| Challenge: | Text Style Transfer (TST) aims to alter the style of text while preserving its core content. |
| Approach: | They propose a framework that leverages large language models alongside chain-of-thought prompting to facilitate TST. |
| Outcome: | The proposed framework surpasses supervised fine-tuning and knowledge distillation methods in low-resource settings. |
Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) language models have a performance gap due to the large decoding space and difficulty in capturing dependency between target words accurately. |
| Approach: | They propose to use reinforcement learning to enhance the performance of edit-based NAR models by using stepwise reward maximization and episodic reward maximisation. |
| Outcome: | The proposed model outperforms autoregressive models in the evaluation of an edit-based model. |
Evaluation Dataset for Japanese Medical Text Simplification (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing studies on medical text simplification in English have not been well explored in Japanese because of the lack of a parallel corpus of this domain. |
| Approach: | They propose a lexically constrained reranking method that allows to avoid technical terms to be output. |
| Outcome: | The proposed method improves on the weblogs of Japanese patients and reduces the need for a training corpus. |
Multi-Source Text Classification for Multilingual Sentence Encoder with Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual sentence encoders suffer from performance degradation for non-English languages. |
| Approach: | They propose a method of machine translating a source sentence into English and then inputting it together with the source sentence in a multi-source manner. |
| Outcome: | The proposed method improves the performance of pre-trained multilingual sentence encoders in Japanese on sentiment analysis and topic classification tasks. |
A Reproducibility Study on Quantifying Language Similarity: The Impact of Missing Values in the URIEL Knowledge Base (2024.naacl-srw)
Copied to clipboard
| Challenge: | URIEL aggregates linguistic information for 4,005 languages and computes distances based on this information. |
| Approach: | They propose to use a typological knowledge base to quantify language similarity to investigate URIEL's ambiguity in calculating language distances and handling missing values. |
| Outcome: | The URIEL knowledge base does not provide information about typological features for 31% of the languages it represents, undermining the reliability of the database, especially on low-resource languages. |
Coding Open-Ended Responses using Pseudo Response Generation by Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing pipelines for survey research using open-ended responses require time and cost-consuming manual tasks. |
| Approach: | They propose an LLM-based method to automate parts of the grounded theory approach . they generate and annotate pseudo open-ended responses and use them as training data . |
| Outcome: | The proposed method is highly efficient andcost-saving compared to human-based methods. |
Cross-Task Generalization Abilities of Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | a thesis proposal advocates for the crucial role of cross-task generalization in NLP systems. |
| Approach: | They propose to benchmark cross-task generalization abilities with diverse NLP tasks . they also propose to develop model architectures for improving cross- task generalization . |
| Outcome: | This paper compares cross-task generalization abilities with diverse NLP tasks . it also analyzes and predicts the generalization landscape of current state-of-the-art large language models . |
Commentary Generation from Data Records of Multiplayer Strategy Esports Game (2024.naacl-srw)
Copied to clipboard
| Challenge: | Esports play logs are expensive for human experts to provide individual games with play-by-play commentaries. |
| Approach: | They first build large-scale esports data-to-text datasets that pair structured data and commentaries from a popular eSports game, League of Legends. |
| Outcome: | The proposed model can generate game commentaries from esports’ data records while examining the impact of the pre-trained language models. |
Facilitating Opinion Diversity through Hybrid NLP Approaches (2024.naacl-srw)
Copied to clipboard
| Challenge: | Modern democracies face a critical issue of declining citizen participation in decision-making. |
| Approach: | They propose a three-layered hierarchy for representing perspectives obtained by a mixture of human intelligence and large language models. |
| Outcome: | The proposed approach can extract high-level insights into arguments employed by citizens or motivations underlying their decisions. |
HybridBERT - Making BERT Pretraining More Efficient Through Hybrid Mixture of Attention Mechanisms (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pretrained transformer-based language models have produced state-of-the-art performance in most natural language understanding tasks. |
| Approach: | They propose two hybrid architectures that combine self-attention and additive attention mechanisms with sub-layer normalization to achieve double the pretraining accuracy of a vanilla-BERT baseline. |
| Outcome: | The proposed architectures outperform BERT-base on two downstream tasks while accelerating inference. |