Papers with transformer-based
Transferability of Syntax-Aware Graph Neural Networks in Zero-Shot Cross-Lingual Semantic Role Labeling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies in cross-lingual semantic role labeling (SRL) lack a comprehensive analysis of their network selection. |
| Approach: | They compare the transferability of graph neural network-based models with universal dependency trees to English and 23 target languages. |
| Outcome: | The proposed models perform better in resource-poor languages than in resource rich ones. |
KNU-HYUNDAI’s NMT system for Scientific Paper and Patent Tasks onWAT 2019 (D19-52)
Copied to clipboard
Cheoneum Park, Young-Jun Jung, Kihoon Kim, Geonyeong Kim, Jae-Won Jeon, Seongmin Lee, Junseok Kim, Changki Lee
| Challenge: | We submitted our transformer-based neural machine translation system to the translation tasks of the 6th workshop on Asian Translation (WAT 2019). |
| Approach: | They propose a transformer-based neural machine translation system for Chinese-Japanese, English-Japanese, and Korean->Japanoise translation tasks. |
| Outcome: | The proposed system performed well on the two translation tasks and was ranked first in terms of the BLEU scores in all the JPC2 subtasks. |
Entity-level Sentiment Analysis in Contact Center Telephone Conversations (2022.emnlp-industry)
Copied to clipboard
Xue-yong Fu, Cheng Chen, Md Tahmid Rahman Laskar, Shayna Gardiner, Pooja Hiranandani, Shashi Bhushan Tn
| Challenge: | Entity-level sentiment analysis is useful in a business context to understand user emotions towards certain entities. |
| Approach: | They propose to use a model that predicts the sentiment about entities mentioned in a given text to build an entity-level sentiment analysis system that analyzes English telephone conversation transcripts. |
| Outcome: | The proposed system analyzes English telephone conversation transcripts to provide business insight. |
ARTIST: A Transformer-based Chinese Text-to-Image Synthesizer Digesting Linguistic and World Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-Image Synthesis (TIS) is a popular task to convert natural language texts into realistic images. |
| Approach: | They propose a transformer-based Chinese text-to-image synthesizer for high-resolution image generation that incorporates linguistic and relational knowledge facts into the model to ensure better performance without the usage of ultra-large models. |
| Outcome: | The proposed model outperforms existing models in Chinese with linguistic and relational knowledge facts. |
Latent Part-of-Speech Sequences for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning target side syntactic structure are greedy and only allow them to explore a limited portion of the latent space. |
| Approach: | They propose a new latent variable model, LaSyn, that captures the co-dependence between syntax and semantics while allowing for effective inference over the latent space. |
| Outcome: | The proposed model captures the co-dependence between syntax and semantics while allowing for efficient inference over the latent space. |
Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data (2021.acl-short)
Copied to clipboard
| Challenge: | a generic transformer-based model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Approach: | They investigate whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Outcome: | The proposed model achieves 81.03% exact match accuracy on Django and 32.57 BLEU score on CoNaLa. |
White-box Testing of NLP models with Mask Neuron Coverage (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent research has shown that black-box testing is not applicable to NLP models. |
| Approach: | They propose a set of white-box testing methods that are customized for transformer-based NLP models and adapt them to a black-box test suite. |
| Outcome: | The proposed methods can reduce testing suites by 60% while retaining failing tests, thereby concentrating faultdetection power of the test suite. |
Pipeline Signed Japanese Translation Focusing on a Post-positional Particle Complement and Conjugation in a Low-resource Setting (2021.findings-acl)
Copied to clipboard
| Challenge: | a pipeline translation method is proposed to take advantage of the similarities and differences between sign language and spoken language. |
| Approach: | They propose a pipeline translation method that takes advantage of similarities between spoken and spoken Japanese . they map glosses to spoken language words and train them using a monolingual Japanese corpus . |
| Outcome: | The proposed method performs robustly on the low-resource corpus and is +4.4/+4.9 points above baseline. |
NerKor+Cars-OntoNotes++ (2022.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we present an upgraded version of the Hungarian NYTK-NerKor named entity corpus . it contains twice as many annotated spans and 7 times as many distinct entity types as the original version. |
| Approach: | They present an upgraded version of the Hungarian NYTK-NerKor named entity corpus with an extended OntoNotes 5 annotation scheme. |
| Outcome: | The enhanced version of the corpus contains twice as many annotated spans and 7 times more distinct entity types than the original version. |
LipKey: A Large-Scale News Dataset for Absent Keyphrases Generation and Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has addressed each element individually, but this study focuses on LipKey, the largest news corpus with human-written abstractive summaries, absent keyphrases, and titles. |
| Approach: | They propose a novel news dataset that consists of highly absent keyphrases . they combine lips keyphrase and TF-IDF to obtain abstractive summaries . |
| Outcome: | The proposed dataset is the largest news corpus with human-written abstractive summaries, absent keyphrases, and titles. |
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Contemporary leading-edge systems for abstractive (long) text summarization employ Transformer encoderdecoder architectures that only consider the nuclearity annotation . |
| Approach: | They propose to incorporate Rhetorical Structure Theory into a novel summarization model that incorporates both the types and uncertainty of rhetorical relations. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic metrics and human evaluation. |
MockingBERT: A Method for Retroactively Adding Resilience to NLP Models (2022.coling-1)
Copied to clipboard
| Challenge: | Existing remediations have compromised accuracy or required full model re-training with each new class of attacks. |
| Approach: | They propose a method of retroactively adding resilience to misspellings to transformer-based NLP models and propose generating adversarial misspells using an approximate method. |
| Outcome: | The proposed method significantly reduces the cost needed to evaluate a model’s resilience to adversarial attacks. |
SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature review models have addressed literature review generation, but lack of large-scale datasets has been a stumbling block. |
| Approach: | They propose to use a large-scale dataset to evaluate automatic literature review generation models. |
| Outcome: | The proposed model can generate summaries comparable to human-written reviews while lacking detailed information. |
Analysis of LLM as a grammatical feature tagger for African American English (2025.findings-naacl)
Copied to clipboard
| Challenge: | African American English (AAE) presents unique challenges in natural language processing (NLP). |
| Approach: | They evaluate the ability of different NLP systems to recognize distinctive AAE grammatical features by using sentence-level binary classification tasks using both zero-shot and fewshot strategies. |
| Outcome: | The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. |
RNSum: A Large-Scale Dataset for Automatic Release Note Generation via Commit Logs Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | a release note is a technical document that describes the latest changes to a software product. |
| Approach: | They propose to extract and then abstract release notes from GitHub repositories using a transformer-based network like BART. |
| Outcome: | The proposed methods generate less noisy release notes at higher coverage than baselines. |
DEplain: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification (2023.acl-long)
Copied to clipboard
| Challenge: | Current text simplification research mostly focuses on English and on sentencelevel simplification. |
| Approach: | They propose to use a dataset of parallel, professionally written and manually aligned simplifications in plain German "plain DE" and "Einfache Sprache" they build a web harvester and experiment with automatic alignment methods to facilitate integration of non-aligned and to be-published parallel documents. |
| Outcome: | The proposed dataset of parallel, professionally written and manually aligned simplifications in plain German is extended to 750 document pairs and 3.5k sentence pairs. |
Seeing Is Believing! towards Knowledge-Infused Multi-modal Medical Dialogue Generation (2024.lrec-main)
Copied to clipboard
Abhisek Tiwari, Shreyangshu Bera, Preeti Verma, Jaithra Varma Manthena, Sriparna Saha, Pushpak Bhattacharyya, Minakshi Dhar, Sarbajeet Tiwari
| Challenge: | Existing models of disease diagnosis using AI do not use knowledge infusion. |
| Approach: | They propose a transformer-based, knowledge-infused multi-modal medical dialogue generation framework . they propose 'discourse-aware' image identifier that recognizes signs and their severity . |
| Outcome: | The proposed model outperforms state-of-the-art models by 7.84% in the english language. |