Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)

Prompt-based Zero-shot Text Classification with Conceptual Knowledge (2023.acl-srw)

Challenge:	In Sinitic Historical Phonology, notable tasks that could benefit from machine learning include the comparison of dialects and reconstruction of proto-languages systems.
Approach:	They propose to use a knowledge graph to obtain multi-dialectal representations of Sinitic syllables by using unsupervised clustering techniques and the BoxE technique from knowledge base learning.
Outcome:	The proposed representations capture phonemic contrast from the input dialects and can be used to infer Middle Chinese labels.

Copied to clipboard

Yuqi Wang, Wei Wang, Qi Chen, Kaizhu Huang, Anh Nguyen, Suparna De

How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese (2023.acl-srw)

Challenge:	Existing approaches to pre-training language models rely on verbalizers to translate the predicted vocabulary to task-specific labels.
Approach:	They propose a framework that incorporates conceptual knowledge for text classification in the extreme zero-shot setting.
Outcome:	The proposed framework outperforms prompt-based approaches on four widely-used datasets for sentiment analysis and topic detection on the same experimental settings.

Copied to clipboard

Takuro Fujii, Koki Shibata, Atsuki Yamaguchi, Terufumi Morishita, Yasuhiro Sogawa

Syntactic Text Processing

Challenge:	Existing studies on scriptio continua languages lack comprehensiveness of tokenizers . authors use Byte-Pair-Encoding or Unigram instead of WordPiece for subword tokenizer .
Approach:	They investigate the effect of tokenizers on the downstream performance of pretrained language models in scriptio continua languages where no explicit spaces exist between words.
Outcome:	The proposed tokenizers perform better on a wide range of tasks compared with other tokenizer methods . the results show that each task has an optimal morphological analyzer .

Semantic-Aware Dynamic Retrospective-Prospective Reasoning for Event-Level Video Question Answering (2023.acl-srw)

Copied to clipboard

Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster

Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models (2023.acl-srw)

Challenge:	Event-Level Video Question Answering (EVQA) requires complex reasoning across video events to obtain the visual information needed to provide optimal answers.
Approach:	They propose a semantic-aware dynamic retrospective-prospective reasoning approach for video-based question answering that explicitly uses the Semantic Role Labeling (SRL) structure of the question in the dynamic reasoning process.
Outcome:	The proposed model outperforms existing models on a trafficQA benchmark dataset.

Copied to clipboard

Tomoki Sugimoto, Yasumasa Onoe, Hitomi Yanaka

Constructing Multilingual Code Search Dataset Using Neural Machine Translation (2023.acl-srw)

Challenge:	Existing datasets and analyses focusing on English do not address the need for resources in other languages.
Approach:	They propose a Japanese NLI benchmark focused on temporal inference . they use a set of temporal patterns to generate diverse examples .
Outcome:	The proposed model can perform fine-grained analysis in Japanese and English.

Copied to clipboard

Ryo Sekizawa, Nan Duan, Shuai Lu, Hitomi Yanaka

Multimodal Neural Machine Translation Using Synthetic Images Transformed by Latent Diffusion Model (2023.acl-srw)

Challenge:	Existing datasets for code search are monolingual, but their query data are only in English.
Approach:	They construct a multilingual code search dataset in four natural and four programming languages using a neural machine translation model and apply back-translation data filtering to it.
Outcome:	The proposed model pre-trained with all natural and programming language data performs best under almost all settings.

Copied to clipboard

Ryoya Yuasa, Akihiro Tamura, Tomoyuki Kajiwara, Takashi Ninomiya, Tsuneo Kato

Enhancing Ancient Chinese Understanding with Derived Noisy Syntax Trees (2023.acl-srw)

Challenge:	Existing methods to translate source language sentences using images are not optimal for machine translation.
Approach:	They propose a new multimodal neural machine translation model using synthetic images transformed by a latent diffusion model.
Outcome:	The proposed model improves translation performance on English-German translation tasks using the Multi30k dataset.

Copied to clipboard

Ping Wang, Shitou Zhang, Zuchao Li, Jingrui Hou

Responsible & Trustworthy NLP

Challenge:	Syntactic information is not used in modern Chinese understanding tasks due to the lack of syntactical annotation.
Approach:	They propose a confidence-based syntax encoding network to alleviate the side effects of unsupervised syntax derivation and the incompatibility between ancient and modern Chinese.
Outcome:	The proposed component alleviates side effects from unsupervised syntax derivation and incompatibility between ancient and modern Chinese.

The Turing Quest: Can Transformers Make Good NPCs? (2023.acl-srw)

Copied to clipboard

Qi Chen Gao, Ali Emami

Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section (2023.acl-srw)

Challenge:	Pre-trained Transformer-based language models have demonstrated impressive conversational abilities, but their use in real-world applications remains unexplored.
Approach:	They propose a pipeline for automatic construction of NPC scripts using Transformer-based believable scripts for a variety of game genres and specifications.
Outcome:	The proposed pipeline generates scripts that fool judges in a variety of game genres and contexts, and can be easily compared to human-written scripts.

Copied to clipboard

Hongyi Zheng, Yixin Zhu, Lavender Jiang, Kyunghyun Cho, Eric Oermann

Intriguing Effect of the Correlation Prior on ICD-9 Code Assignment (2023.acl-srw)

Challenge:	Clinical notes have a long time span over multiple long documents.
Approach:	They propose a framework to analyze clinical notes with high predictive power . they propose to combine different types of notes to improve performance .
Outcome:	The proposed framework could be used to extract information from clinical notes . it shows that the sample size can be optimized for large contexts .

Copied to clipboard

Zihao Yang, Chenkang Zhang, Muru Wu, Xujin Liu, Lavender Jiang, Kyunghyun Cho, Eric Oermann

Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks (2023.acl-srw)

Challenge:	The Ninth Revision of the International Classification of Diseases (ICD-9) is a standardized coding system used worldwide to classify and code diseases, injuries, and other health conditions.
Approach:	They evaluate the usefulness of correlation bias and suggest it could improve ICD-9 code assignment in some cases.
Outcome:	The proposed model improves on classes that are more imbalanced and less correlated with other codes, but the effect on individual class can be negative or positive.

Copied to clipboard

Mateusz Baran, Joanna Baran, Mateusz Wójcik, Maciej Zięba, Adam Gonczarek

Information Extraction & Text Mining

Challenge:	Current approaches to OOD detection in NLP are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts.
Approach:	They evaluated eight methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications.
Outcome:	The proposed methods are easily integrable into existing NLP systems and require no additional OOD data or model modifications.

Can LMs Store and Retrieve 1-to-N Relational Knowledge? (2023.acl-srw)

Copied to clipboard

Haruki Nagasawa, Benjamin Heinzerling, Kazuma Kokuta, Kentaro Inui

Theoretical Linguistics Rivals Embeddings in Language Clustering for Multilingual Named Entity Recognition (2023.acl-srw)

Challenge:	Pretraining language models on large amounts of text has made it difficult to store and retrieve world knowledge.
Approach:	They propose to view pretrained language models as knowledge bases by examining their ability to store and retrieve world knowledge.
Outcome:	The proposed model can store and retrieve world knowledge with high accuracy, but it is not clear how accurately it can handle 1-to-N relational knowledge.

Copied to clipboard

Sakura Imai, Daisuke Kawahara, Naho Orita, Hiromune Oda

Information Extraction & Text Mining

Challenge:	Existing studies have used descriptive typological features and a coarse language family classification as baselines for language clustering.
Approach:	They propose two types of language groupings based on morpho-syntactic features in a nominal domain and one based upon a head parameter.
Outcome:	The proposed methods outperform state-of-the-art embedding-based models in multilingual named entity recognition (NER) . their results suggest that theoretical linguistics plays a significant role in multi-lingual learning tasks.

Native Language Prediction from Gaze: a Reproducibility Study (2023.acl-srw)

Copied to clipboard

Lina Skerath, Paulina Toborek, Anita Zielińska, Maria Barrett, Rob Van Der Goot

Responsible & Trustworthy NLP

Challenge:	Existing studies have shown that the linguistic properties of a speaker’s native language affect the cognitive processing of other languages.
Approach:	They found that the correlation between eye movements and native language similarity may be more complex than the original study found.
Outcome:	The proposed model shows that the correlation between eye movements and native language similarity may be more complex than the original study.

MedTem2.0: Prompt-based Temporal Classification of Treatment Events from Discharge Summaries (2023.acl-srw)

Copied to clipboard

Yang Cui, Lifeng Han, Goran Nenadic

Sudden Semantic Shifts in Swedish NATO discourse (2023.acl-srw)

Challenge:	Clinical texts contain important temporal information, such as medication start and end dates, appointment dates, and diagnosis dates.
Approach:	They propose to use prompt-based learning and fine-tuning to classify temporal relations between treatments and hospitalisation periods in discharge summaries.
Outcome:	The proposed method identifies whether a treatment was administered between the time of admission and discharge from the hospital.

Copied to clipboard

Brian Bonafilia, Bastiaan Bruinsma, Denitsa Saynova, Moa Johansson

Building a Buzzer-quiz Answering System (2023.acl-srw)

Challenge:	Using word embeddings, we study sudden semantic shifts that occur when a sudden event radically changes public opinion on a topic.
Approach:	They use word embeddings to study how Twitter associations evolve . they find domain knowledge and data selection are of prime importance when using word embeds to understand semantic shifts.
Outcome:	The proposed method validates associations on Twitter with NATO in real-world events but is difficult to distinguish between noise and real-time signals.

Copied to clipboard

Naoya Sugiura, Kosuke Yamada, Ryohei Sasano, Koichi Takeda, Katsuhiko Toyama

Natural Language Interfaces

Challenge:	A buzzer quiz is a genre of quiz in which multiple players simultaneously listen to a quiz being read aloud and respond it by buzzing in as soon as they can predict the answer.
Approach:	They propose two types of buzzer-quiz answering systems: a system that directly generates an answer from part of a question by using an autoregressive language model and a second system that reconstructs the entire question by applying an autoreregressively language model.
Outcome:	The proposed system estimates the accuracy of the answers by using the internal scores of each model.

Probing for Hyperbole in Pre-Trained Language Models (2023.acl-srw)

Copied to clipboard

Nina Schneidermann, Daniel Hershcovich, Bolette Pedersen

Towards Efficient Dialogue Processing in the Emergency Response Domain (2023.acl-srw)

Challenge:	Hyperbole is a common figure of speech that involves the use of exaggerated language for emphasis or effect.
Approach:	They conduct edge and minimal description length probing experiments on three pre-trained language models to explore the extent to which hyperbolic information is encoded . they also annotate 63 hyperbole sentences from the HYPO dataset according to an operational taxonomy to conduct an error analysis to explore encoding of different hyperboli categories.
Outcome:	The results show that hyperbole is encoded in a limited extent in pre-trained models and mostly in the final layers.

Copied to clipboard

Tatiana Anikina

Dialogue Systems & Conversational Agents

Challenge:	Adapters perform dialogue act classification and domain-specific slot tagging in the emergency response domain.
Approach:	They propose to build a system that performs dialogue act classification and domain-specific slot tagging while being efficient, flexible and robust.
Outcome:	The proposed model performs well in the emergency response domain while being efficient, flexible and robust.

I already said that! Degenerating redundant questions in open-domain dialogue systems. (2023.acl-srw)

Copied to clipboard

Long Mai, Julie Carson-berndsen

Natural Language Interfaces

Challenge:	Neural text generation models have been successful in short open-domain conversations, but their performance degrades significantly in the long term.
Approach:	They propose a method to generate training data without crowdsourcing . they adapt negative training, decoding, and classification methods to mitigate redundancy problem .
Outcome:	The proposed method reduces the rate of redundant questions from 27.2% to 8.7% while improving the quality of the original model.

Is a Knowledge-based Response Engaging?: An Analysis on Knowledge-Grounded Dialogue with Information Source Annotation (2023.acl-srw)

Copied to clipboard

Takashi Kodama, Hirokazu Kiyomaru, Yin Jou Huang, Taro Okahisa, Sadao Kurohashi

Dialogue Systems & Conversational Agents

Challenge:	Currently, most knowledge-grounded dialogue models focus on reflecting given external knowledge.
Approach:	They analyze human behavior by annotating utterances in an existing knowledge-grounded dialogue corpus and find that speaker-derived information improves dialogue engagingness.
Outcome:	The proposed model cannot include speaker-derived information as often as humans do.

Choosing What to Mask: More Informed Masking for Multimodal Machine Translation (2023.acl-srw)

Copied to clipboard

Julia Sato, Helena Caseli, Lucia Specia

Multilinguality

Challenge:	Pre-trained language models have achieved remarkable results on several NLP tasks.
Approach:	They propose three new masking strategies for cross-lingual visual pre-training that focus on learning different linguistic patterns.
Outcome:	The proposed methods outperform the baseline model and achieve state-of-the-art accuracy on the Portuguese-English MMT task.

Combining Tradition with Modernness: Exploring Event Representations in Vision-and-Language Models for Visual Goal-Step Inference (2023.acl-srw)

Copied to clipboard

Chong Shen, Carina Silberer

Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)

Challenge:	Existing methods for representing procedural knowledge are limited to capturing the most crucial information, namely actions and the participants, to learn stereotypical event sequences.
Approach:	They propose a task that uses images to identify steps towards achieving a goal in the multimodal domain.
Outcome:	The proposed task uses images that represent steps towards achieving a textually expressed goal in the multimodal domain.

Copied to clipboard

Stephanie Schoch, Ritwick Mishra, Yangfeng Ji

Distractor Generation for Fill-in-the-Blank Exercises by Question Type (2023.acl-srw)

Challenge:	Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models.
Approach:	They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model.
Outcome:	The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset.

Copied to clipboard

Nana Yoshimi, Tomoyuki Kajiwara, Satoru Uchida, Yuki Arase, Takashi Ninomiya

Challenge:	Existing studies have generated words that are semantically similar to the correct words as distractors for fill-in-the-blank questions.
Approach:	They propose a method to automatically generate distractors for fill-in-the-blank questions in entrance examinations for Japanese universities.
Outcome:	The proposed method is effective on 500 actual questions on English fill-in-the-blank questions in Japanese universities.

Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity (2023.acl-srw)

Copied to clipboard

Gabriel Simmons

LECO: Improving Early Exiting via Learned Exits and Comparison-based Exiting Mechanism (2023.acl-srw)

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities in generating fluent text, as well as tendencies to reproduce undesirable social biases.
Approach:	They propose that LLMs reproduce moral biases associated with political groups in the United States, an instance of a broader capability termed moral mimicry.
Outcome:	The LLMs generated by the models reproduce moral biases associated with political groups in the United States, and this is an instance of a broader capability termed moral mimicry.

Copied to clipboard

Jingfan Zhang, Ming Tan, Pengyu Dai, Wei Zhu

Authorship Attribution of Late 19th Century Novels using GAN-BERT (2023.acl-srw)

Challenge:	Recent work on dynamic early exiting has neglected the intermediate exits’ architectural designs.
Approach:	They propose a framework for learning exits and COmparison-based early exiting to improve PTMs’ early exit performance.
Outcome:	The proposed framework achieves the SOTA performance on multi-exit BERT training and dynamic early exiting on pre-trained models.

Copied to clipboard

Kanishka Silva, Burcu Can, Frédéric Blain, Raheem Sarwar, Laura Ugolini, Ruslan Mitkov

How-to Guides for Specific Audiences: A Corpus and Initial Findings (2023.acl-srw)

Challenge:	Conventional techniques and neural networks are the two main authorship attribution methods.
Approach:	They used a dataset of late 19th century novels in English to fine-tune a transformer-based authorship attribution model using transfer learning.
Outcome:	The proposed model outperforms the existing model with 0.88 accuracy and F1 scores.

Copied to clipboard

Nicola Fanton, Agnieszka Falenska, Michael Roth

Responsible & Trustworthy NLP

Challenge:	wikiHow guides for specific target groups reflect disparate social norms and subtle stereotypes, a new study shows . wikihow guides are subject to subtle biases, and we aim to raise awareness of these inequalities in future work.
Approach:	They investigate the extent to which how-to guides from wikiHow differ in practice depending on intended audience.
Outcome:	The findings show that how-to guides from wikiHow differ in practice depending on the intended audience.

“When Words Fail, Emojis Prevail”: A Novel Architecture for Generating Sarcastic Sentences With Emoji Using Valence Reversal and Semantic Incongruity (2023.acl-srw)

Copied to clipboard

Faria Binte Kader, Nafisa Hossain Nujat, Tasmia Binte Sogir, Mohsinul Kabir, Hasan Mahmud, Md Kamrul Hasan