Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022

LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue (2022.findings-aacl)

Challenge:	Mauritian Creole is a French-based creole and a lingua franca of the Republic of Mauritius.
Approach:	They describe a dataset for benchmarking machine translation quality of Mauritian Creole.
Outcome:	The proposed dataset compares KreolMorisienMT with existing models and human evaluation reveals the systems’ high translation quality.

Copied to clipboard

Anthony Sicilia, Malihe Alikhani

Dialogue Systems & Conversational Agents

Challenge:	Generating coherent, human-like text for dialogue remains a challenge . lack of careful design of rewards can lead to mode-collapse in dialogue .
Approach:	They propose a theoretical framework for learning to generate text in dialogue . they propose to use data-shift to develop theoretical guarantees for learners .
Outcome:	The proposed framework improves both task-success and human-likeness of the generated text.

Copied to clipboard

Yacine Gaci, Boualem Benatallah, Fabio Casati, Khalid Benabdeslem

Semantic Similarity

Challenge:	Existing methods of matching user queries with subjective tags rely on cosine similarity or semantic similarity models fail to recognize conceptual connections between tags.
Approach:	They propose a conceptual similarity pipeline to leverage conceptual awareness when assessing similarity between tags.
Outcome:	The proposed pipeline generates high-quality datasets and evaluates the model on a downstream application.

TaskMix: Data Augmentation for Meta-Learning of Spoken Intent Understanding (2022.findings-aacl)

Copied to clipboard

Surya Kant Sahu

Low-Resource NLP

Challenge:	Meta-Learning requires a large number of training tasks to learn representations that transfer well to unseen tasks.
Approach:	They propose a method which synthesizes new tasks by linearly interpolating existing tasks.
Outcome:	The proposed method outperforms baselines and does not degrade performance even when it is high.

Understanding the Use of Quantifiers in Mandarin (2022.findings-aacl)

Copied to clipboard

Guanyi Chen, Kees van Deemter

Challenge:	a corpus of short texts in Mandarin is analyzed to examine the "coolness" hypothesis . quantified expressions are used to describe short texts, but are not as informative as English .
Approach:	They propose a corpus of Mandarin in which quantified expressions figure prominently.
Outcome:	The proposed corpus of short texts in Mandarin is compared with an English corpus.

Does Representational Fairness Imply Empirical Fairness? (2022.findings-aacl)

Copied to clipboard

Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, Lea Frermann

SEHY: A Simple yet Effective Hybrid Model for Summarization of Long Scientific Documents (2022.findings-aacl)

Challenge:	Neural methods have been trained on datasets which embody cultural and societal stereotypes, captured in spurious correlations between target labels and protected attributes.
Approach:	They propose a debiasing method that encourages a latent space that separates instances based on target label, while mixing instances that share protected attributes.
Outcome:	The proposed method shows that representational fairness does not imply empirical fairness across methods.

Copied to clipboard

Zhihua Jiang, Junzhan Yang, Dongning Rao

Text Generation

Challenge:	Abstractive approaches to extract salient sentences from long documents are not effective due to their size.
Approach:	They propose a simple yet effective approach that exploits the discourse information of a document to select salient sections instead of sentences for summary generation.
Outcome:	The proposed approach avoids full-text understanding and retains salient information given the length limit.

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation (2022.findings-aacl)

Copied to clipboard

Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu, Yingzhan Lin, Zheng-Yu Niu

Dialogue Systems & Conversational Agents

Challenge:	Experimental results show PLATO-XL achieves state-of-the-art results across multiple conversational tasks.
Approach:	They propose to train PLATO-XL models with up to 11 billion parameters, trained on Chinese and English social media conversations.
Outcome:	The proposed model achieves state-of-the-art on multiple conversational tasks, verifying its potential as a foundation model of conversational AI.

A Hybrid Architecture for Labelling Bilingual Māori-English Tweets (2022.findings-aacl)

Copied to clipboard

David Trye, Vithya Yogarajan, Jemma König, Te Taka Keegan, David Bainbridge, Mark Apperley

Meta-Learning Adaptive Knowledge Distillation for Efficient Biomedical Natural Language Processing (2022.findings-aacl)

Challenge:	a hybrid approach to annotate mixed Mori-English text is developed . existing language detection tools struggle with interlingual homographs .
Approach:	They propose a hybrid approach that couples Mori-language orthography with machine learning to annotate mixed Mi-English text.
Outcome:	The proposed approach outperforms existing methods on out-of-domain data.

Copied to clipboard

Abiola Obamuyide, Blair Johnston

The Effects of Surprisal across Languages: Results from Native and Non-native Reading (2022.findings-aacl)

Challenge:	Existing knowledge distillation methods have been proposed to reduce the size of large models for biomedical natural language processing tasks.
Approach:	They propose a meta-learning approach which adaptively learns parameters that enable optimal rate of knowledge exchange between teacher and student models from the distillation data during knowledge distillation.
Outcome:	The proposed method improves the performance of knowledge distillation methods on two biomedical natural language processing tasks.

Copied to clipboard

Andrea de Varda, Marco Marelli

Cross-Lingual Transfer

Challenge:	Context-dependent predictive processes have been proposed as a core component of the human cognitive system.
Approach:	They extract surprisal estimates from mBERT and assess their predictive power on the MECO corpus, a cross-linguistic dataset of eye movement behavior in reading.
Outcome:	The proposed model is based on a cross-linguistic dataset of eye movement behavior in reading.

Assessing How Users Display Self-Disclosure and Authenticity in Conversation with Human-Like Agents: A Case Study of Luda Lee (2022.findings-aacl)

Copied to clipboard

Won Ik Cho, Soomin Kim, Eujeong Choi, Younghoon Jeong

Dialogue Systems & Conversational Agents

Challenge:	Existing studies on how people interact with conversational agents have not investigated the interaction authenticity of human-like agents.
Approach:	They construct a taxonomy to discern the users’ self-disclosure in the dialogue and the communication authenticity displayed in the user posting.
Outcome:	The proposed taxonomy can be used for future research and industrial development.

Block Diagram-to-Text: Understanding Block Diagram Images by Generating Natural Language Descriptors (2022.findings-aacl)

Copied to clipboard

Shreyanshu Bhushan, Minho Lee

Multimodality

Challenge:	Block diagram summarization is a task that can be used to generate text from block diagrams.
Approach:	They propose a framework that converts block diagram images into text by extracting contextual meaning from the images in the form of triplets.
Outcome:	The proposed framework outperforms existing methods and techniques on a dataset of handwritten block diagrams.

Multi-Domain Dialogue State Tracking By Neural-Retrieval Augmentation (2022.findings-aacl)

Copied to clipboard

Lohith Ravuru, Seonghan Ryu, Hyungtak Choi, Haehun Yang, Hyeonmok Ko

Natural Language Interfaces

Challenge:	Existing approaches for DST are conditioned on previous dialogue states, but the dependency on previous dialogs makes it difficult to prevent error propagation to subsequent turns.
Approach:	They propose to create a Neural Index based on dialogue context by analyzing user dialogue and previous turn state and generating a retrieval-guided generation approach.
Outcome:	The proposed framework retrieves dialogue context from the index built using unstructured dialogue state and structured user/system utterances.

TaKG: A New Dataset for Paragraph-level Table-to-Text Generation Enhanced with Knowledge Graphs (2022.findings-aacl)

Copied to clipboard

Qianqian Qi, Zhenyun Deng, Yonghua Zhu, Lia Jisoo Lee, Michael Witbrock, Jiamou Liu

Structured Data in NLP

Challenge:	Existing table-to-text generation benchmarks have some limitations, such as E2E and ToTTo focusing on singlesentence generation tasks.
Approach:	They propose a new table-to-text generation dataset called TaKG that uses a set of knowledge graphs to enhance table input.
Outcome:	The proposed model outperforms existing models for short-text generation tasks and shows reliable performance on long-text generated across a variety of metrics.

Revisiting Checkpoint Averaging for Neural Machine Translation (2022.findings-aacl)

Copied to clipboard

Yingbo Gao, Christian Herold, Zijian Yang, Hermann Ney

Machine Translation

Challenge:	Checkpoint averaging is a simple and effective method to boost the performance of converged neural machine translation models.
Approach:	They propose to use checkpoint averaging to increase model performance . they also propose to calculate weighted average instead of simple mean .
Outcome:	The proposed method is widely adopted in neural machine translation research.

Modeling Referential Gaze in Task-oriented Settings of Varying Referential Complexity (2022.findings-aacl)

Copied to clipboard

Özge Alacam, Eugen Ruppert, Sina Zarrieß, Ganeshan Malhotra, Chris Biemann, Sina Zarrieß

Psycholinguistics

Challenge:	Referential gaze is a fundamental phenomenon for psycholinguistics and human-human communication.
Approach:	They propose a multimodal NLP task to predict when the gaze is referential . they train a sequential attention-based LSTM model and a transformer encoder architecture to model referential gaze and transfer gaze features to unseen situated settings .
Outcome:	The proposed model can be applied to situations with different referential complexities . the proposed model is based on an attention-based LSTM model and a multivariate transformer encoder architecture .

Automating Interlingual Homograph Recognition with Parallel Sentences (2022.findings-aacl)

Copied to clipboard

Yi Han, Ryohei Sasano, Koichi Takeda

Cross-Lingual Transfer

Challenge:	Existing methods for interlingual homograph recognition require linguistic knowledge and massive annotation work.
Approach:	They propose an automatic interlingual homograph recognition method based on cross-lingual word embedding similarity and co-occurrence of form-identical words in parallel sentences.
Outcome:	The proposed method can make accurate predictions across languages.

CoRAL: a Context-aware Croatian Abusive Language Dataset (2022.findings-aacl)

Copied to clipboard

Ravi Shekhar, Mladen Karan, Matthew Purver

A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation (2022.findings-aacl)

Challenge:	Semi-automated comment moderation systems can greatly aid human moderators by either automatically classifying the examples or allowing the moderator to prioritize which comments to consider first.
Approach:	They propose to use a language and culturally aware Croatian Abusive dataset to analyze inappropriate comments in a context-based manner.
Outcome:	The proposed dataset shows that current models degrade when comments are not explicit and further degrades when language skill and context knowledge are required to interpret the comment.

Copied to clipboard

Rose Hirigoyen, Amal Zouaq, Samuel Reyd

Text Generation

Challenge:	Current architectures are unable to integrate knowledge base schema and handle questions unseen during training rendering them unusable outside the scope of topics covered in the training set.
Approach:	They propose to integrate a copy mechanism for neural SPARQL query generation by adding a knowledge base layer and a dynamic knowledge base vocabulary to two Seq2Seq architectures.
Outcome:	The proposed model outperforms existing models on state-of-the-art datasets and shows a significant increase in performance.

A Multilingual Multiway Evaluation Data Set for Structured Document Translation of Asian Languages (2022.findings-aacl)

Copied to clipboard

Bianka Buschbeck, Raj Dabre, Miriam Exel, Matthias Huck, Patrick Huy, Raphael Rubino, Hideki Tanaka

On Measures of Biases and Harms in NLP (2022.findings-aacl)

Challenge:	a lack of evaluation data sets for structured content limits progress in machine translation . a common use case of machine translation is the translation of structured or formatted documents .
Approach:	They propose a multilingual multiway evaluation data set for machine translation of structured documents of Asian languages Japanese, Korean and Chinese.
Outcome:	The proposed data set is well suited for multilingual evaluation and contains richer annotation tag sets than existing data sets.

Copied to clipboard

Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang

Logographic Information Aids Learning Better Representations for Natural Language Inference (2022.findings-aacl)

Challenge:	Recent studies show that natural language processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.
Approach:	They propose a framework for harms and questions to help practitioners understand biases . they propose measurable measures to detect and mitigate biased groups .
Outcome:	The proposed framework provides a framework for harms and questions for practitioners to answer to guide the development of bias measures.

Copied to clipboard

Zijian Jin, Duygu Ataman

Cross-domain Analysis on Japanese Legal Pretrained Language Models (2022.findings-aacl)

Challenge:	Statistical language models often ignore information related to logographic features of written text . many languages, including Chinese and Vietnamese, use logographic writing systems .
Approach:	They propose to provide language models with logographic information to learn better semantic representations.
Outcome:	The proposed model improves semantic representations in six languages with logographic writing systems.

Copied to clipboard

Keisuke Miyazaki, Hiroaki Yamada, Takenobu Tokunaga

Multilingual CheckList: Generation and Evaluation (2022.findings-aacl)

Challenge:	Existing studies do not care the performance of domain-adapted PLMs for a generic domain.
Approach:	They propose to use pretraining strategies to build pretrained language models specialised in the legal domain to improve their performance.
Outcome:	The pretrained language models can learn domain-specific and general word meanings simultaneously and can distinguish them.

Copied to clipboard

Karthikeyan K, Shaily Bhatt, Pankaj Singh, Somak Aditya, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury

Part Represents Whole: Improving the Evaluation of Machine Translation System Using Entropy Enhanced Metrics (2022.findings-aacl)

Challenge:	Multilingual evaluation benchmarks usually contain limited high-resource languages and do not test models for specific linguistic capabilities.
Approach:	They propose an algorithm for automatically extracting target language CheckList templates from machine translated instances of a source language templates.
Outcome:	The proposed algorithm compares with CheckLists created with human verification in Hindi and 9 other languages.

Copied to clipboard

Yilun Liu, Shimin Tao, Chang Su, Min Zhang, Yanqing Zhao, Hao Yang

Machine Translation

Challenge:	Existing machine translation metrics have poor correlations with human assessments . entropy-based evaluations are often limited to a limited number of samples .
Approach:	They propose a fast and unsupervised approach to enhance machine translation metrics using entropy by introducing sentence-level difficulty.
Outcome:	The proposed method outperforms existing metrics on five sub-tracks in the WMT19 Metrics shared tasks.

Memformer: A Memory-Augmented Transformer for Sequence Modeling (2022.findings-aacl)

Copied to clipboard

Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu, Alborz Geramifard, Zhou Yu

Open-Domain Conversational Question Answering with Historical Answers (2022.findings-aacl)

Challenge:	Experimental results show that Memformer uses 8.1x less memory space and 3.2x faster on inference.
Approach:	They propose an efficient neural network that utilizes an external dynamic memory to encode and retrieve past information.
Outcome:	The proposed model achieves comparable performance against baselines with 8.1x less memory space and 3.2x faster on inference.

Copied to clipboard

Hung-Chieh Fang, Kuo-Han Hung, Chen-Wei Huang, Yun-Nung Chen

Natural Language Interfaces

Challenge:	Existing approaches to conversational question answering are limited due to the large number of candidate documents.
Approach:	They propose a model that leverages historical answers to boost retrieval performance . they propose to use open-domain conversational question answering to solve these problems .
Outcome:	The proposed model outperforms baseline models in extractive and generative reader settings on OR-QuAC dataset.

Robustness Evaluation of Text Classification Models Using Mathematical Optimization and Its Application to Adversarial Training (2022.findings-aacl)

Copied to clipboard

Hikaru Tomonari, Masaaki Nishino, Akihiro Yamamoto

Robustness in NLP

Challenge:	Neural networks are vulnerable to adversarial examples due to slightly perturbed input data.
Approach:	They propose a method that evaluates the robustness of text classification models by an optimization problem that identifies a minimum synonym swap that changes the classification result.
Outcome:	The proposed method achieves high scores in human evaluations of grammatical correctness and semantic similarity for an IMDb dataset and implements adversarial training with the IMD and SST2 datasets.

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models (2022.findings-aacl)

Copied to clipboard

Yizhi Li, Ge Zhang, Bohao Yang, Chenghua Lin, Anton Ragni, Shi Wang, Jie Fu

Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models (2022.findings-aacl)

Challenge:	Existing methods do not examine social groups categorised by geographical information, leaving the region-related biases in pre-trained LMs unexplored.
Approach:	They propose a hierarchical regional bias evaluation method to quantify regional bias in pre-trained language models.
Outcome:	The proposed method evaluates regional bias with regard to comprehensive topics and measures potential regional bias that can be propagated to downstream tasks.

Copied to clipboard

Syrielle Montariol, Arij Riabi, Djamé Seddah

Chop and Change: Anaphora Resolution in Instructional Cooking Videos (2022.findings-aacl)

Challenge:	Zero-shot cross-lingual transfer learning has been shown to be challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as hate speech detection.
Approach:	They propose to train on multilingual auxiliary tasks to improve zero-shot transfer of hate speech detection models across languages by bringing a cross-lingual knowledge proxy to the task.
Outcome:	The proposed methods improve zero-shot transfer of hate speech detection models across languages and domains using multilingual auxiliary tasks fine-tuned.

Copied to clipboard

Cennet Oguz, Ivana Kruijff-Korbayova, Emmanuel Vincent, Pascal Denis, Josef van Genabith

Information Extraction & Text Mining

Challenge:	temporally evolving entities present challenges for anaphora resolution tasks . recipes provide rich source for referring expressions of transformed entities .
Approach:	They propose to use annotations to annotate recipes for anaphora resolution task . they propose to employ temporal features to improve anamorphic resolution .
Outcome:	The proposed annotation scheme improves the performance of the anaphora resolution task.

“#DisabledOnIndianTwitter” : A Dataset towards Understanding the Expression of People with Disabilities on Indian Twitter (2022.findings-aacl)

Copied to clipboard

Ishani Mondal, Sukhnidh Kaur, Kalika Bali, Aditya Vashistha, Manohar Swaminathan

Topic-aware Multimodal Summarization (2022.findings-aacl)

Challenge:	a majority of disabled Indians exist at the margins of society with little to no access to social media . as access to ICTs and high-speed internet grows, Indian Twitter's user base is expanding to include disability influencers, activists, and everyday disabled users.
Approach:	They propose a hierarchical annotation taxonomy to classify tweets into various themes including discrimination, advocacy, and self-identification.
Outcome:	The proposed taxonomy classifies 2,384 tweets into various themes including discrimination, advocacy, and self-identification.

Copied to clipboard

Sourajit Mukherjee, Anubhav Jangra, Sriparna Saha, Adam Jatowt

Information Extraction & Text Mining

Challenge:	Existing work on multimodal summarization does not consider the topic of the content.
Approach:	They propose a topic-aware MS system which performs two tasks simultaneously: differentiating images into "on-topic" and "off-topic".
Outcome:	The proposed system outperforms the state-of-the-art approach by 1.7 % in ROUGE-L metric.

ArgGen: Prompting Text Generation Models for Document-Level Event-Argument Aggregation (2022.findings-aacl)

Copied to clipboard

Debanjana Kar, Sudeshna Sarkar, Pawan Goyal

Hierarchical Processing of Visual and Language Information in the Brain (2022.findings-aacl)

Challenge:	Existing discourse-level information extraction tasks are extractive in nature, but extracting information from larger bodies of discourse-like documents requires more natural language understanding and reasoning capabilities.
Approach:	They propose a conditional text generation approach which generates consolidated event-arguments at a document-level with minimal loss of information.
Outcome:	The proposed approach generates document-level argument spans in a low-resource and zero-shot setting and can be leveraged in other related multilingual text generation tasks.

Copied to clipboard

Haruka Kawasaki, Satoshi Nishida, Ichiro Kobayashi

Visual Data in NLP

Challenge:	In recent years, many studies have been conducted to elucidate the mechanism of information representation in the brain under stimuli evoked by various modalities.
Approach:	They constructed encoding models that predict brain activity based on features extracted from hidden layers of VGG16 for visual information and BERT for language information.
Outcome:	The proposed model predicts the brain activity of visual and language information in the cortex and shows that it is getting closer to that of BERT as VGG16 moves to higher layers, while the representational contents differ significantly between the two modalities.

Differential Bias: On the Perceptibility of Stance Imbalance in Argumentation (2022.findings-aacl)

Copied to clipboard

Alonso Palomino, Khalid Al Khatib, Martin Potthast, Benno Stein

BeamR: Beam Reweighing with Attribute Discriminators for Controllable Text Generation (2022.findings-aacl)

Challenge:	a theoretical model of bias classification is not feasible because of complexity of interpreting language phenomena.
Approach:	They propose to analyze whether a text is biased based on an algorithmic analysis . they propose to use a model to determine whether x is more biased than y .
Outcome:	a crowdsourcing study shows that differences in stance bias are perceptible when (light) support is provided through training or visual aids.

Copied to clipboard

David Landsman, Jerry Zikun Chen, Hussain Zaidi

R&R: Metric-guided Adversarial Sentence Generation (2022.findings-aacl)

Challenge:	Recent advances in natural language processing have led to the availability of large pre-trained language models with rich generative capabilities.
Approach:	They propose a method to combine generative LMs with attribute discriminators to control different attributes of text generation.
Outcome:	The proposed method performs better than existing state-of-the-art approaches in sentiment steering and machine translation formality tasks.

Copied to clipboard

Lei Xu, Alfredo Cuesta-Infante, Laure Berti-Equille, Kalyan Veeramachaneni

Information Extraction & Text Mining

Challenge:	Existing methods prioritize misclassification by maximizing each perturbation’s effectiveness at misleading a text classifier.
Approach:	They propose a rewrite and rollback framework for adversarial attack that optimizes a critique score which combines fluency, similarity, and misclassification metrics.
Outcome:	The proposed framework outperforms current state-of-the-art in attack success rate by +16.2%, +12.8%, and +14.0% on the classifiers respectively.

A Simple yet Effective Learnable Positional Encoding Method for Improving Document Transformer Model (2022.findings-aacl)

Copied to clipboard

Guoxin Wang, Yijuan Lu, Lei Cui, Tengchao Lv, Dinei Florencio, Cha Zhang