SEMEVAL

Transparent, Efficient, and Robust Word Embedding Access with WOMBAT (C18-2)

Challenge:	Existing work on humor recognition does not examine the actual joke mechanism . a recent study focused on humor-specific stylistic features, but few have tried to establish a connection between them and humor theories.
Approach:	They propose to model the set-up and punchline as part developing semantic uncertainty and disrupt audience expectations.
Outcome:	The proposed features can tell jokes from non-jokes, compared with baselines.

Copied to clipboard

Mark-Christoph Müller, Michael Strube

Progressive Self-Training with Discriminator for Aspect Term Extraction (2021.emnlp-main)

Challenge:	WOMBAT is a Python tool that supports accessing word embeddings from code.
Approach:	WOMBAT is a Python tool which supports NLP practitioners in accessing word embeddings from code.
Outcome:	WOMBAT is a Python tool which supports accessing word embeddings from code.

Copied to clipboard

Qianlong Wang, Zhiyuan Wen, Qin Zhao, Min Yang, Ruifeng Xu

Aspect-Category-Opinion-Sentiment Quadruple Extraction with Implicit Aspects and Opinions (2021.acl-long)

Challenge:	Existing approaches to extract aspect terms from review sentences are limited due to lack of annotated data.
Approach:	They propose to refine conventional self-training to progressive self-teaching to reduce noise . they use a discriminator to filter the noisy pseudo-labels.
Outcome:	The proposed model outperforms baseline models and achieves state-of-the-art performance on four SemEval datasets.

Copied to clipboard

Hongjie Cai, Rui Xia, Jianfei Yu

All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes (2020.aacl-main)

Challenge:	Existing studies in aspect-based sentiment analysis ignore aspects and opinions in product reviews.
Approach:	They propose a task to extract aspect-category-opinion-sentiment quadruples from review sentences . they construct two new datasets that contain annotations of implicit aspects and opinions .
Outcome:	The proposed task provides full support for aspect-based sentiment analysis with implicit aspects and opinions.

Copied to clipboard

Dushyant Singh Chauhan, Dhanush S R, Asif Ekbal, Pushpak Bhattacharyya

Knowledge-Enriched Two-Layered Attention Network for Sentiment Analysis (N18-2)

Challenge:	Empirical results show the efficacy of our proposed multi-task framework over existing state-of-the-art systems.
Approach:	They propose a multi-task, multi-modal deep learning framework to solve multiple tasks simultaneously.
Outcome:	The proposed framework performs better than existing state-of-the-art systems on a complicated form of information, i.e., memes.

Copied to clipboard

Abhishek Kumar, Daisuke Kawahara, Sadao Kurohashi

ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (P18-1)

Challenge:	Existing sentiment analysis systems are prone to word shortening, exaggeration, lack of grammar and appropriate punctuation.
Approach:	They propose a two-layered attention network based on Bidirectional Long Short-Term Memory for sentiment analysis using the Knowledge Graph Embedding generated using the WordNet.
Outcome:	The proposed model outperforms the state-of-the-art system on the benchmark dataset of SemEval 2017 Task 5 by 1.7 and 3.7 points respectively.

Copied to clipboard

John Wieting, Kevin Gimpel

Text Generation

Challenge:	Using neural machine translation, we generate more than 50 million sentential paraphrase pairs from a large parallel corpus.
Approach:	They use a dataset of more than 50 million English-English sentential paraphrase pairs to generate them automatically using neural machine translation.
Outcome:	The proposed dataset outperforms all supervised systems on every SemEval semantic textual similarity competition and shows how it can be used for paraphrase generation.

Injecting Relational Structural Representation in Neural Networks for Question Similarity (P18-2)

Copied to clipboard

Antonio Uva, Daniele Bonadiman, Alessandro Moschitti

Representation Learning

Challenge:	Recent years have seen exponential growth and use of web forums, where users can exchange and find information just asking questions in natural language.
Approach:	They propose to use Tree Kernels to learn a model on relatively few pairs of questions as gold standard (GS) predicting labels on a very large corpus of question pairs is also a useful approach, they propose .
Outcome:	The proposed model can learn more accurate models after fine tuning on GS.

Transfer Capsule Network for Aspect Level Sentiment Classification (P19-1)

Copied to clipboard

Zhuang Chen, Tieyun Qian

Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains (C18-1)

Challenge:	Lack of aspect-level labeled data is a major obstacle in sentiment classification due to high cost . document-level labels like reviews are easily accessible from online websites .
Approach:	They propose a transfer capsule network model for transferring document-level knowledge to aspect-level sentiment classification by encapsulating sentence-level semantic representations into semantic capsules.
Outcome:	The proposed model can transfer document-level knowledge to aspect-level sentiment classification.

Copied to clipboard

Jeremy Barnes, Roman Klinger, Sabine Schulte im Walde

A Multi-task Learning Framework for Opinion Triplet Extraction (2020.findings-emnlp)

Challenge:	Existing domain adaptation methods for sentiment analysis are sensitive to domain differences, resulting in classifiers that perform poorly on new domains.
Approach:	They propose a domain adaptation problem as an embedding projection task using two mono-domain embeddable spaces and a bi-domain space to project across domains and predict sentiment.
Outcome:	The proposed model performs better on domains similar to state-of-the-art methods while requiring longer training times.

Copied to clipboard

Chen Zhang, Qiuchi Li, Dawei Song, Benyou Wang

Domain Mismatch Doesn’t Always Prevent Cross-lingual Transfer Learning (2022.lrec-1)

Challenge:	Existing approaches to Aspect-based sentiment analysis (ABSA) use aspect terms and their corresponding sentiment polarities as a reference, but they lack opinion terms as .
Approach:	They propose a multi-task learning framework to extract aspect terms and opinion terms and parse their sentiment dependencies with a biaffine scorer.
Outcome:	The proposed framework outperforms baseline and state-of-the-art approaches on four SemEval benchmarks.

Copied to clipboard

Daniel Edmiston, Phillip Keung, Noah A. Smith

Multilinguality

Challenge:	Recent studies have reported that domain mismatch prevents cross-lingual transfer . UBLI and UNMT do not work well when underlying monolingual corpora come from different domains .
Approach:	They show that a simple initialization regimen can overcome domain mismatch in cross-lingual transfer . they pre-train word embeddings on concatenated domain-mismatched corpora and use them as initializations .
Outcome:	The initialization regimen can overcome the domain mismatch effect in cross-lingual transfer learning . the initializations were used for MUSE UBLI, UN Parallel UNMT, and the SemEval 2017 task .

Dependency Position Encoding for Relation Extraction (2022.findings-naacl)

Copied to clipboard

Qiushi Guo, Xin Wang, Dehong Gao

Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction (2023.findings-emnlp)

Challenge:	Existing methods to extract relation extraction from sentence are limited in focusing on leveraging dependency information.
Approach:	They propose dependency position encoding (DPE) that incorporates dependency connections and dependency types into the self-attention mechanism to distinguish the importance of different word dependencies.
Outcome:	The proposed method significantly outperforms the previous methods on SemEval 2010 Task 8, KBP37, and TACRED.

Copied to clipboard

Jiaxiang Chen, Yu Hong, Qingting Xu, Jianmin Yao

PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction (2020.coling-main)

Challenge:	Aspect Term Extraction (ATE) is a task of automatically extracting aspect terms from sentences.
Approach:	They propose to automatically rewrite sentences from virtual experts with different roles . they leverage ChatGPT to determine virtual experts in the considered domains .
Outcome:	The proposed method can be used to expand the predictions obtained on the original sentences without retraining or fine-tuning the baseline extractors.

Copied to clipboard

Yichun Yin, Chenguang Wang, Ming Zhang

Enhancing Aspect Term Extraction with Soft Prototypes (2020.emnlp-main)

Challenge:	Existing word embeddings that capture the contextual information only produce moderate results in aspect term extraction.
Approach:	They propose a positional dependency-based word embedding which takes both dependency context and positional context into account for aspect term extraction.
Outcome:	The proposed method outperforms other embedding methods in aspect term extraction.

Copied to clipboard

Zhuang Chen, Tieyun Qian

FastSense: An Efficient Word Sense Disambiguation Classifier (L18-1)

Challenge:	Existing studies focus on designing neural sequence taggers to extract linguistic features from token level.
Approach:	They propose to correlating aspects with each other through soft prototypes . they propose to combine ATE with almost all sequence taggers to extract aspect terms .
Outcome:	The proposed model boosts the performance of three typical ATE methods on four SemEval datasets.

Copied to clipboard

Tolga Uslu, Alexander Mehler, Daniel Baumartz, Wahed Hemati

Cross-Level Semantic Similarity for Serbian Newswire Texts (2022.lrec-1)

Challenge:	Word Sense Disambiguation (WSD) is a task that is often overlooked by NLP pipelines because of its complexity and complexity.
Approach:	They propose a neural network-based tool for word sense disambiguation called fastSense.
Outcome:	The proposed tool can process huge amounts of data quickly and surpasses state-of-the-art tools in terms of F-measure.

Copied to clipboard

Vuk Batanović, Maja Miličević Petrović

Evaluating Evaluation Measures for Ordinal Classification and Ordinal Quantification (2021.acl-long)

Challenge:	Semantic similarity is a measure of the level of semantic overlap between texts of different lengths.
Approach:	They present a cross-level semantic similarity (CLSS) dataset in Serbian and compare it to its English counterpart, SemEval CLSS. They also use pre-trained language models to fine-tune the dataset.
Outcome:	The proposed dataset is compared to its preexisting counterpart in English, SemEval CLSS. The results are presented and state-of-the-art pre-trained language models are evaluated on the CLSS task in Serbian.

Copied to clipboard

Tetsuya Sakai

Text Classification

Challenge:	Ordinal Classification (OC) tasks require ordinal classes, not nominal ones, to be evaluated.
Approach:	They use data from the SemEval and NTCIR communities to clarify evaluation measures for Ordinal Classification and Ordinal Quantification tasks.
Outcome:	The evaluation measures for Ordinal Classification (OC) and Ordinal Quantification (OQ) tasks are ordinal, not nominal.

Can Word Sense Distribution Detect Semantic Changes of Words? (2023.findings-emnlp)

Copied to clipboard

Xiaohang Tang, Yi Zhou, Taichi Aida, Procheta Sen, Danushka Bollegala

Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies (P19-1)

Challenge:	Existing methods to detect semantic variations of words are not accurate for time-sensitive predictions.
Approach:	They propose to use pretrained static sense embeddings to annotate a word's occurrence with a sense id to compare its distributions.
Outcome:	The proposed method compares word sense distributions across two corpora to predict meaning change . the results show that pretrained LLMs can detect changes in words over time .

Copied to clipboard

Shuhei Kurita, Anders Søgaard

Language Models

Challenge:	Existing dependency parsing algorithms do not support directed acyclic graphs . a a systole-based dependency parses sentences using binary semantic relations that are not trees .
Approach:	They propose an iterative predicate selection algorithm for semantic dependency parsing . they train the algorithm using multi-task learning and task-specific policy gradient training .
Outcome:	The proposed model achieves a new state of the art on the SemEval 2015 task 18 dataset .

Aspect Based Sentiment Analysis with Gated Convolutional Networks (P18-1)

Copied to clipboard

Wei Xue, Tao Li

Multi-task Pairwise Neural Ranking for Hashtag Segmentation (P19-1)

Challenge:	Aspect-based sentiment analysis can provide more detailed information than general sentiment analysis.
Approach:	They propose a model based on convolutional neural networks and gating mechanisms which can selectively output the sentiment features according to the given aspect or entity.
Outcome:	The proposed model can selectively output sentiment features according to the given aspect or entity.

Copied to clipboard

Mounica Maddela, Wei Xu, Daniel Preoţiuc-Pietro

Low-Resource NLP

Challenge:	Hashtags are used to add metadata to textual utterances, but their semantic content is difficult to infer as they often contain multiple tokens joined together.
Approach:	They propose to use a dataset of 12,594 hashtags to infer hashtag semantics . they propose to frame the problem as a pairwise ranking problem between candidate segmentations .
Outcome:	The proposed methods show 24.6% error reduction in hashtag segmentation accuracy compared to the current state-of-the-art method.

Matching the Blanks: Distributional Similarity for Relation Learning (P19-1)

Copied to clipboard

Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, Tom Kwiatkowski

Unsupervised Paraphrasing of Multiword Expressions (2023.findings-acl)

Challenge:	Efforts to build general purpose relation extractors that can model arbitrary relations are limited in their ability to generalize.
Approach:	They propose to build task-agnostic relation representations solely from entity-linked text to extend Harris’ distributional hypothesis to relations.
Outcome:	The proposed representations outperform previous methods on SemEval 2010 Task 8, KBP37, and TACRED even without using any of the task’s training data.

Copied to clipboard

Takashi Wada, Yuji Matsumoto, Timothy Baldwin, Jey Han Lau

To be Closer: Learning to Link up Aspects with Opinions (2021.emnlp-main)

Challenge:	Existing methods for paraphrasing multiword expressions in context are unsupervised . multiwords are notoriously difficult to model because the meaning of the whole can diverge substantially from that of the component words.
Approach:	They propose an unsupervised approach to paraphrasing multiword expressions in context using monolingual corpus data and pre-trained language models.
Outcome:	The proposed method outperforms all unsupervised systems and rivals supervised systems on the SemEval 2022 idiomatic text similarity task.

Copied to clipboard

Yuxiang Zhou, Lejian Liao, Yang Gao, Zhanming Jie, Wei Lu

Modeling Multi-Granularity Hierarchical Features for Relation Extraction (2022.naacl-main)

Challenge:	Dependency parsers are not designed for capturing interaction between opinion words and aspect words.
Approach:	They propose to learn an aspect-centric tree structure to shorten distance between aspects and opinion words.
Outcome:	The proposed model outperforms baselines on five aspect-based sentiment datasets.

Copied to clipboard

Xinnian Liang, Shuangzhi Wu, Mu Li, Zhoujun Li

Multi-grained Attention Network for Aspect-Level Sentiment Classification (D18-1)

Challenge:	Existing work on relation extraction focuses on constructing explicit structured features using knowledge graph and dependency tree.
Approach:	They propose a method to extract multi-granularity features based solely on the original input sentences.
Outcome:	The proposed method outperforms state-of-the-art models that even use external knowledge on three public benchmarks: SemEval 2010 Task 8, Tacred, and Tacred Revisited.

Copied to clipboard

Feifan Fan, Yansong Feng, Dongyan Zhao

It is Simple Sometimes: A Study On Improving Aspect-Based Sentiment Analysis Performance (2024.findings-acl)

Challenge:	Existing approaches to aspect sentiment classification use coarse-grained attention mechanisms . a novel approach captures word-level interaction between aspect and context .
Approach:	They propose a novel multi-grained attention network model for aspect level sentiment classification . they use a fine-grounded attention mechanism to capture word-level interaction between aspect and context .
Outcome:	The proposed model outperforms the state-of-the-art methods on three datasets . it shows that aspect-level interactions can bring extra useful information and improve performance .

Copied to clipboard

Laura Cabello, Uchenna Akujuobi

Text Emotion Distribution Learning from Small Sample: A Meta-Learning Approach (D19-1)

Challenge:	Existing studies have focused on developing ad hoc models of varying complexity for ABSA subtasks.
Approach:	They propose an instruction-based model with task descriptions followed by in-context examples on ABSA subtasks.
Outcome:	The proposed method outperforms state-of-the-art methods on most domains and achieves competitive results on biomedical domain datasets.

Copied to clipboard

Zhenjie Zhao, Xiaojuan Ma

Complex and Precise Movie and Book Annotations in French Language for Aspect Based Sentiment Analysis (L18-1)

Challenge:	Existing methods for text emotion distribution learning require a large amount of training data, which is difficult to obtain due to inconsistent perception of fine-grained emotion intensity.
Approach:	They propose a meta-learning approach to learn text emotion distributions from a small sample using tensor decomposition to capture contextual semantic similarity.
Outcome:	The proposed method outperforms state-of-the-art methods on a widely used EDL dataset.

Copied to clipboard

Stefania Pecore, Jeanne Villaneau

SenseBERT: Driving Some Sense into BERT (2020.acl-main)

Challenge:	Aspect Based Sentiment Analysis (ABSA) aims at collecting detailed opinion information according to products and their features.
Approach:	They propose to use linguistics tools to enhance text classification with aspect-based sentiment analysis.
Outcome:	The proposed method is based on two French online reviews datasets.

Copied to clipboard

Yoav Levine, Barak Lenz, Or Dagan, Ori Ram, Dan Padnos, Or Sharir, Shai Shalev-Shwartz, Amnon Shashua, Yoav Shoham

DeepREF: A Framework for Optimized Deep Learning-based Relation Classification (2022.lrec-1)

Challenge:	Existing approaches for self-supervision operate at word form level, which serves as a surrogate for the underlying semantic content.
Approach:	They propose a method to employ weak-supervision directly at the word sense level, without the use of human annotation.
Outcome:	The proposed model achieves significantly improved lexical understanding without human annotation on the ‘Word in Context’ task.

Copied to clipboard

Igor Nascimento, Rinaldo Lima, Adrian-Gabriel Chifu, Bernard Espinasse, Sébastien Fournier

Word Sense Induction with Neural biLM and Symmetric Patterns (D18-1)

Challenge:	Existing frameworks for relation extraction (RE) are limited due to lack of implementation details.
Approach:	They propose to use deep learning to develop relation extraction systems using deep learning models.
Outcome:	The proposed framework is inspired by the OpenNRE and REflex existing frameworks.

Copied to clipboard

Asaf Amrami, Yoav Goldberg

A Novel Aspect-Guided Deep Transition Model for Aspect Based Sentiment Analysis (D19-1)

Challenge:	Existing methods for word sense induction use a language model to predict probable substitutes for target words.
Approach:	They propose to use a language model to predict probable substitutes for target words . they replace the ngram-based language model with a recurrent model to generate strong substitute vectors .
Outcome:	The proposed method surpasses the current state-of-the-art on the SemEval 2013 task by a large margin.

Copied to clipboard

Yunlong Liang, Fandong Meng, Jinchao Zhang, Jinan Xu, Yufeng Chen, Jie Zhou

Structural Bias for Aspect Sentiment Triplet Extraction (2022.coling-1)

Challenge:	Existing models use aspect-independent encoders for sentence representation generation.
Approach:	They propose an aspect-guided deep transition model which guides the sentence encoding from scratch with a specially-designed deep transition architecture.
Outcome:	The proposed model outperforms existing models on multiple datasets on aspect-category sentiment analysis and aspectterm sentiment analysis without additional features.

Copied to clipboard

Chen Zhang, Lei Ren, Fang Ma, Jingang Wang, Wei Wu, Dawei Song

Transition-based Semantic Dependency Parsing with Pointer Networks (2020.acl-main)

Challenge:	Existing structural bias adapters for aspect sentiment triplet extraction are under-confident . a large-scale dataset for ASTE shows the adapter is effective and efficient to a larger scale.
Approach:	They propose to use a structural adapter to integrate structural bias into pretrained language models . they propose to add a relative position structure in place of the syntactic dependency structure .
Outcome:	The proposed adapter achieves state-of-the-art performance over strong baselines, but with a light parameter demand and low latency.

Copied to clipboard

Daniel Fernández-González, Carlos Gómez-Rodríguez

Syntactic Text Processing

Challenge:	Existing dependency parsers cannot be directly applied, so they need to be adaptable to deal with the absence of singlehead and connectedness constraints.
Approach:	They propose a transition system that produces labelled directed acyclic graphs and performs semantic dependency parsing with Pointer Networks.
Outcome:	The proposed system outperforms graph-based models and outperformed existing models on a harder NLP problem.

Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss (2024.findings-acl)

Copied to clipboard

Wei He, Marco Idiart, Carolina Scarton, Aline Villavicencio

Representation Learning

Challenge:	Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in natural language processing (NLP).
Approach:	They propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning by using adaptive contrastive learning and resampling miners.
Outcome:	The proposed model outperforms previous models significantly on a SemEval challenge and outperformed previous alternatives in many metrics.

Combination of Contextualized and Non-Contextualized Layers for Lexical Substitution in French (2022.lrec-1)

Copied to clipboard

Kévin Espasa, Emmanuel Morin, Olivier Hamon

Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals (2022.emnlp-main)

Challenge:	Lexical substitution task requires to substitute a target word by candidates in a given context.
Approach:	They propose a method to find synonyms for a target word and rank them based on the context of the sentence.
Outcome:	The proposed method increases the BERT based system on the OOT measure but decreases on the BEST measure in the SemDis 2014 benchmark.

Copied to clipboard

Maarten De Raedt, Fréderic Godin, Chris Develder, Thomas Demeester

Robustness in NLP

Challenge:	a recent study found that finetuned language models rely on spurious patterns in training data . this limitation limits their performance on out-of-distribution (OOD) test data.
Approach:	They propose a method that only requires annotation of a small fraction of training data . they add 1% manual counterfactuals to training data and generate extra counterfacts in vector space .
Outcome:	The proposed approach improves sentiment classification using IMDb data and other sets for OOD tests.

More DWUGs: Extending and Evaluating Word Usage Graph Datasets in Multiple Languages (2024.emnlp-main)

Copied to clipboard

Dominik Schlechtweg, Pierluigi Cassotti, Bill Noble, David Alfter, Sabine Schulte Im Walde, Nina Tahmasebi

Multimodality

Challenge:	Word Usage Graphs (WUGs) represent word sense clusters from simple pairwise word use judgments.
Approach:	They propose to use a weighted graph to represent human semantic proximity judgments for pairs of word uses to infer word sense clusters from simple pairwise word use judgments.
Outcome:	The proposed approach can be applied in a Word Sense Induction (WSI) setting or for Word sense disambiguation (WSD) it is the first and to date largest manually annotated, diachronic WUG dataset.

Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis (2023.findings-acl)

Copied to clipboard

Joachim Wagner, Jennifer Foster

Aspect-Based Sentiment Analysis as Fine-Grained Opinion Mining (2020.lrec-1)

Challenge:	In a set of masking experiments, we examine the extent to which the tokens identified as salient by LIME and a gradient-based method are being used by the classifier.
Approach:	They use a BERT BASE model to mask the sentiments of an English dataset and find that both methods produce faithful rationales.
Outcome:	The proposed classifier outperforms both the gradient-based and black-box saliency methods on the SemEval 2016 english dataset.

Copied to clipboard

Gerardo Ocampo Diaz, Xuanming Zhang, Vincent Ng