Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop
Investigation on Data Adaptation Techniques for Neural Named Entity Recognition (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition use only a limited number of samples . data augmentation and selftraining are popular methods to generate additional synthetic data . |
| Approach: | They investigate the impact of data augmentation and data augmented on named entity recognition tasks. |
| Outcome: | The proposed methods improve the performance of three named entity recognition tasks. |
Stage-wise Fine-tuning for Graph-to-Text Generation (2021.acl-srw)
Copied to clipboard
| Challenge: | Graph-to-text generation has benefited from pre-trained language models (PLMs) but they fail to fully utilize the structure information of the input graph. |
| Approach: | They propose a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tracks model on Wikipedia before adapting to graph- to-text generation. |
| Outcome: | The proposed model improves the performance of the English WebNLG 2017 dataset by using tree-level embeddings to capture the inter-dependency structures of the input graph. |
Transformer-Based Direct Hidden Markov Model for Machine Translation (2021.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have found that word alignments produced by the multi-head cross-attention weights are poor. |
| Approach: | They propose to introduce the hidden Markov model to the transformer architecture and introduce alignment components while keeping the system monolithic. |
| Outcome: | The proposed model outperforms the baseline model but is slower in training and decoding. |
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing literature on relation classification models shows no consensus on optimal architecture . |
| Approach: | They propose a search space for BERT based relation classification models and employ an ENAS method to find better architectures. |
| Outcome: | The proposed method can find better architectures than baseline models on eight benchmark RC tasks. |
How Low is Too Low? A Computational Perspective on Extremely Low-Resource Languages (2021.acl-srw)
Copied to clipboard
| Challenge: | Sumerian is one of the world’s oldest written languages attested from at least the beginning of the 3rd millennium BC. |
| Approach: | They propose to use interpretLR to train attention-based deep learning models in a low-resource language, Sumerian cuneiform, which includes part-of-speech tagging, named entity recognition, and machine translation. |
| Outcome: | The proposed pipeline outperforms the large language model RoBERTa for POS Tagging and NER. |
On the Relationship between Zipf’s Law of Abbreviation and Interfering Noise in Emergent Languages (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have shown that emergent languages do not obey ZLA when neural agents play a signaling game. |
| Approach: | They propose to add an explicit penalty on word lengths to a signaling game to simulate a ZLA-like tendency when interfering noises are added to the agents' environment. |
| Outcome: | The proposed model shows that the noise on a speaker is one of the factors for ZLA, while noise on the listener and a channel is not. |
Long Document Summarization in a Low Resource Setting using Pretrained Language Models (2021.acl-srw)
Copied to clipboard
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, Andrew McCallum
| Challenge: | Existing abstractive summarization methods only achieve 17.9 ROUGE-L in low-resource settings. |
| Approach: | They propose to use a modern abstractive summarization algorithm to extract salient sentences from long documents to improve their performance. |
| Outcome: | The proposed method beats several competitive salience detection baselines and the identified salient sentences agree with independent human labeling by domain experts. |
Attending Self-Attention: A Case Study of Visually Grounded Supervision in Vision-and-Language Transformers (2021.acl-srw)
Copied to clipboard
| Challenge: | a growing body of research has been focused on what attention heads learn during the pre-training of visual grounded language models. |
| Approach: | They propose to use visual grounding to supervise attention directly to learn visual ground. |
| Outcome: | The proposed method improves the performance of a state-of-the-art visual grounded language model on vision-and-language tasks. |
Video-guided Machine Translation with Spatial Hierarchical Attention Network (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing studies use pretrained motion detection models as verb sense ambiguity representations to solve the verb sense problem. |
| Approach: | They propose to use video contents as auxiliary information to address the word sense ambiguity problem in machine translation. |
| Outcome: | Experiments on the VATEX dataset show that the proposed system achieves 35.86 BLEU-4 score, which is 0.51 score higher than the single model of the SOTA method. |
Stylistic approaches to predicting Reddit popularity in diglossia (2021.acl-srw)
Copied to clipboard
| Challenge: | Past research has shown that style is a strong predictor of community response, but what about a diglossia? |
| Approach: | They propose to use punctuation, stopwords and part-of-speech tags to predict the popularity of a Reddit post in a diglossia in Singapore where the basilect co-exists with an acrolect . |
| Outcome: | The proposed approach combines natural language processing (NLP) techniques with punctuation, stopwords and part-of-speech tags to predict popular posts in a diglossia. |
“I’ve Seen Things You People Wouldn’t Believe”: Hallucinating Entities in GuessWhat?! (2021.acl-srw)
Copied to clipboard
| Challenge: | a problem with natural language generation systems is the generation of tokens that are unrelated to the source input. |
| Approach: | They propose two new models to play the GuessWhat?! referential game . they propose to adapt the best visual processing models available to mitigate this issue . |
| Outcome: | The proposed models generate few hallucinations compared to other models available in the literature. |
How do different factors Impact the Inter-language Similarity? A Case Study on Indian languages (2021.acl-srw)
Copied to clipboard
| Challenge: | a large amount of natural language data is added daily on the Internet. |
| Approach: | They performed an extensive case study on language similarity in Indian subcontinent . they focused on factors such as size and type of corpus, similarity algorithms, subword segmentation . |
| Outcome: | The proposed approach measures how similar the two languages are on the basis of their lexical, morphological and syntactic features. |
COVID-19 and Misinformation: A Large-Scale Lexical Analysis on Twitter (2021.acl-srw)
Copied to clipboard
| Challenge: | Social media is used by individuals and organisations as a platform to spread misinformation. |
| Approach: | They compile a large corpus of tweets related to coronavirus and perform an analysis to discover patterns with respect to vocabulary usage. |
| Outcome: | The proposed model based on lexical features is effective in identifying misinformation-related tweets with accuracy over 80%. |
Situation-Based Multiparticipant Chat Summarization: a Concept, an Exploration-Annotation Tool and an Example Collection (2021.acl-srw)
Copied to clipboard
| Challenge: | Currently, text chat does not offer navigation or full-featured search, although the high volumes of messages demand it. |
| Approach: | They propose a data annotation tool for situation-based summarization that can be used to extract messages from chat logs. |
| Outcome: | The proposed tool is the first to be developed for situation-based summarization. |
Modeling Text using the Continuous Space Topic Model with Pre-Trained Word Embeddings (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing topic models that extract latent topics from text are based on latent topic and do not use intermediate variables such as latent subjects. |
| Approach: | They propose a model that extends the continuous space topic model (CSTM) they pre-train word embeddings which capture the semantics of words and plug them into the CSTM . |
| Outcome: | The proposed model performs better than the baseline model in terms of perplexity and convergence speed. |
Semantics of the Unwritten: The Effect of End of Paragraph and Sequence Tokens on Text Generation with GPT2 (2021.acl-srw)
Copied to clipboard
| Challenge: | Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
| Approach: | They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage. |
| Outcome: | The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings (2021.acl-srw)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual word embedding methods learn a linear transformation matrix that maps two monolingual embeddable spaces that are separately trained with monolingual corpora. |
| Approach: | They propose a method that maps two monolingual embedding spaces that are separately trained with monolingual corpora using a pseudo-parallel corpus. |
| Outcome: | The proposed method outperforms other methods given the same amount of data and shows that using a pseudo-parallel corpus makes the source and target corpora (partially) parallel . |
Joint Detection and Coreference Resolution of Entities and Events with Document-level Context Aggregation (2021.acl-srw)
Copied to clipboard
| Challenge: | Recent work on extracting information from sentences or paragraphs has a difficulty analyzing longer contexts. |
| Approach: | They propose a jointly trained model that can be used for various information extraction tasks at the document level. |
| Outcome: | The proposed model improves entity and event typing and typing on documents from the ACE05-E+ dataset. |
“Hold on honey, men at work”: A semi-supervised approach to detecting sexism in sitcoms (2021.acl-srw)
Copied to clipboard
| Challenge: | sexist dialogue in sitcoms is an important part of society's development, according to Sink and Mastro (2017). |
| Approach: | They propose a semi-supervised text classification model that automatically detects instances of sexism in popular sitcom dialogues. |
| Outcome: | The proposed model outperforms deep learning-based systems in detecting sexist dialogues over time and shows that sexism decreases over the years. |
Observing the Learning Curve of NMT Systems With Regard to Linguistic Phenomena (2021.acl-srw)
Copied to clipboard
| Challenge: | Using a semi-automatic process, we observe the linguistic performance of various neural machine translation models. |
| Approach: | They observe the linguistic performance of a neural machine translation model on several steps on the training process. |
| Outcome: | The proposed system performs well on training of English-to-German models. |
Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing data augmentation methods for reading comprehension lack robustness to challenge sets whose distribution is different from that of training sets. |
| Approach: | They propose a question-answer pair generation method that generates multiple diverse QA pairs from a paragraph to mitigate this problem. |
| Outcome: | The proposed model improves the accuracy of 12 challenge sets and the in-distribution accuracy. |
Tools Impact on the Quality of Annotations for Chat Untangling (2021.acl-srw)
Copied to clipboard
| Challenge: | Creating annotated datasets is time-consuming and expensive . a tool that minimizes task load and facilitates the annotation process is lacking in studies on how it influences annotation quality. |
| Approach: | They propose to use SLATE and Parlay to improve annotation quality for a task of chat-untangling. |
| Outcome: | The proposed tool improves the user experience for the task of chatuntangling. |
How Many Layers and Why? An Analysis of the Model Depth in Transformers (2021.acl-srw)
Copied to clipboard
| Challenge: | In deep transformers, weights are tied across layers, resulting in multiple layers. |
| Approach: | They propose a variant of Albert that adapts the number of layers for each token of the input. |
| Outcome: | The proposed model implements the key specificity of Albert and iterates on token representations over time. |
Edit Distance Based Curriculum Learning for Paraphrase Generation (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing studies show that curriculum learning improves translation quality on machine translation . paraphrase generation allows a certain level of semantic divergence between source and target . |
| Approach: | They propose to apply curriculum learning to paraphrase generation for the first time . they propose to use edit distance to improve paraphrase quality . |
| Outcome: | The proposed method improves paraphrase generation quality, compared with previous methods . it uses edit distance, which is not possible for previous methods, the authors say . |
Changing the Basis of Contextual Representations with Explicit Semantics (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing transformer-based contextual representations are opaque as their latent dimensions are not directly interpretable. |
| Approach: | They propose an algorithm where the output representation expresses human-interpretable information of each dimension. |
| Outcome: | The proposed transformations are able to predict supersense category of a word by looking for its transformed coordinate with the largest coefficient. |
Personal Bias in Prediction of Emotions Elicited by Textual Opinions (2021.acl-srw)
Copied to clipboard
| Challenge: | Various models for emotion recognition have been used in different studies. |
| Approach: | They propose to use an annotated corpus to estimate personal emotional bias to estimate individual responses to texts . they propose to employ a new BERT-based transformer architecture to predict emotions from an individual human perspective. |
| Outcome: | The proposed method improves the quality of personalized reasoning and may boost the quality and reliability of content recommendation systems. |
MVP-BERT: Multi-Vocab Pre-training for Chinese BERT (2021.acl-srw)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) improve performance of Chinese NLP tasks, but the vocabulary (vocab) for these Chinese PLMs remains to be the one provided by Google Chinese BERT (CITATION), which is based on Chinese characters (chars). |
| Approach: | They propose two versions of multi-vocab pre-training (MVP) to improve Chinese PLMs' expressiveness. |
| Outcome: | The proposed model outperforms the recent AMBERT (CITATION) model on span-level tasks and is more robust against adversarial attacks. |
CMTA: COVID-19 Misinformation Multilingual Analysis on Twitter (2021.acl-srw)
Copied to clipboard
| Challenge: | myths, sensationalism, rumours and misinformation, generated intentionally or unintentionally, spread rapidly through social networks during the COVID-19 pandemic . evaluation of tweets for recognizing misinformation can create beneficial understanding to review the top quality and also the readability of online information concerning the COV-19. |
| Approach: | They propose a multilingual COVID-19 related tweet analysis method that uses a deep learning model for multilingual tweet misinformation detection and classification. |
| Outcome: | The proposed method outperforms monolingual models in the misinformation detection task and shows that it can be used to improve the quality and readability of online information. |
Predicting pragmatic discourse features in the language of adults with autism spectrum disorder (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing tools to quantify atypicality in discourse and pragmatics are difficult to precisely identify and quantify. |
| Approach: | They present a corpus of transcribed natural conversations produced in an experimental setting and annotate them for three pragmatic features on a three-point scale. |
| Outcome: | The proposed model yields higher accuracy than previous approaches for deriving these features, with F1 exceeding 0.82 for all three pragmatic features. |
SumPubMed: Summarization Dataset of PubMed Scientific Articles (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing summarization models that can extract the top few lines of news articles fail to summarize long documents. |
| Approach: | They constructed a scientific summarization dataset from MEDLINE articles from the PubMed archive to address this problem. |
| Outcome: | The proposed model outperforms existing models on news article summarization datasets and shows that it is more efficient to extract the top few lines. |
A Case Study of Analysis of Construals in Language on Social Media Surrounding a Crisis Event (2021.acl-srw)
Copied to clipboard
| Challenge: | construal level theory (CLT) uses concreteness as covariate to analyze language around political import events. |
| Approach: | They propose to include psycholinguistic measures of concreteness as covariates in topic models to analyze the language around an event of political import. |
| Outcome: | The proposed model incorporates measures of concreteness as covariates to inform the analysis of language around the 2017 rally. |
Cross-lingual Evidence Improves Monolingual Fake News Detection (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing methods focused on one language and do not use multilingual information. |
| Approach: | They propose a new technique based on cross-lingual evidence that can be used for fake news detection . they compared their proposed technique with strong baselines on two datasets of general-topic news . |
| Outcome: | The proposed technique improves existing methods and can be used on real and fake news datasets. |
Neural Machine Translation with Synchronous Latent Phrase Structure (2021.acl-srw)
Copied to clipboard
| Challenge: | grammatical information annotation requires high human resources and is not trivial due to language mismatches and out-of-vocabulary problem. |
| Approach: | They propose to incorporate grammatical information without supervising annotation by induced latent phrase structure and synchronized phrase structures in encoder and decoder to enhance explainability. |
| Outcome: | The proposed method produces better performance and explainability in translation and alignment tasks without extra resources. |
Zero Pronouns Identification based on Span prediction (2021.acl-srw)
Copied to clipboard
| Challenge: | Pro-drop languages allow omissions of essential phrases or arguments . the presence of zero-pronouns affects downstream tasks of NLP . |
| Approach: | They propose a query-based method to identify zero-pronoun arguments . they use Japanese and Chinese datasets to evaluate the method . |
| Outcome: | The proposed method surpasses the sequence labeling baseline on Japanese and Chinese datasets. |
On the differences between BERT and MT encoder spaces and how to address them in translation tasks (2021.acl-srw)
Copied to clipboard
| Challenge: | Various studies show that pretrained language models cannot replace encoders in neural machine translation despite their success in other tasks. |
| Approach: | They propose a supervised transformation from one into the other to improve the applicability of BERT in neural machine translation. |
| Outcome: | The proposed transformations show that they cannot replace encoders in MT despite their success in other tasks. |
Synchronous Syntactic Attention for Transformer Neural Machine Translation (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing syntaxbased NMT models use monolingual syntactic information on either side or both. |
| Approach: | They propose a mechanism that synchronizes source-side and target-side syntactic self-attentions by minimizing the difference between target- and target side self- attentions mapped by the encoder-decoder attention matrix. |
| Outcome: | The proposed method improves translation performance on WMT14 En-De, WMT16 En-Ro, and ASPEC Ja-En (up to +0.38 points in BLEU). |