Papers by Carolina Scarton
ATLAS: Improving Lay Summarisation with Attribute-based Control (2024.acl-short)
Copied to clipboard
| Challenge: | Lay summarisation aims to produce scientific summaries that are comprehensible to non-experts. |
| Approach: | They propose an abstractive summarisation approach that can control properties contributing to overall "layness" they evaluate ATLAS on a combination of biomedical lay summarization datasets. |
| Outcome: | The proposed approach outperforms state-of-the-art summarisation metrics on biomedical datasets and shows that it can be discriminatory and emergently influenced. |
MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on providing individual, well-defined types of context in translation, such as the surrounding text or discrete external variables like the speaker’s gender. |
| Approach: | They introduce a novel neural machine translation framework that interprets all context as text. |
| Outcome: | The proposed framework outperforms a baseline that matched the parameters and significantly outperformed it in English translation. |
Probing for idiomaticity in vector space models (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextualised word representation models are used to represent idiomaticity in language. |
| Approach: | They propose probing measures to assess if some of the expected linguistic properties of noun compounds are readily available in some standard and widely used representations. |
| Outcome: | The proposed models show that idiomaticity is not yet accurately represented by contextualised models. |
Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels (2021.acl-long)
Copied to clipboard
| Challenge: | Existing resources for idiomaticity annotation only include ratings at type level . idioms such as noun compounds have been considered a challenge for NLP . |
| Approach: | They present a dataset with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token levels. |
| Outcome: | The proposed dataset shows that human annotations are not capturing idiomaticity as human annotation models. |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for sentence simplification are focused on a single transformation, such as lexical paraphrasing or splitting. |
| Approach: | They propose a dataset for assessing sentence simplification in English using a crowdsourced multi-reference corpus. |
| Outcome: | The proposed dataset shows that it captures characteristics of simplicity better than other datasets. |
Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss (2024.findings-acl)
Copied to clipboard
| Challenge: | Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in natural language processing (NLP). |
| Approach: | They propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning by using adaptive contrastive learning and resampling miners. |
| Outcome: | The proposed model outperforms previous models significantly on a SemEval challenge and outperformed previous alternatives in many metrics. |
Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for lay summarisation are limited in size and scope, hindering the development of data-driven approaches. |
| Approach: | They propose to use two new datasets for the lay summarisation of biomedical research articles to characterise their lay summaries. |
| Outcome: | The proposed datasets are compared with existing datasets and show they can be leveraged to support different audiences and applications. |
AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets are limited to providing the degree of idiomaticity of expressions along with the literal and, where applicable, (a single) non-literal interpretation of MWEs. |
| Approach: | They propose to use a dataset to test the effectiveness of a language model in generating representations of sentences containing idioms. |
| Outcome: | The proposed model performs reasonably well on the one-shot and few-shot scenarios, but there is scope for improvement in the zero-shot scenario. |
Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis (2020.aacl-main)
Copied to clipboard
| Challenge: | Hate speech and toxic comments are a common concern of social media platform users . identifying toxic comments is important for studying and preventing the proliferation of toxicity in social media. |
| Approach: | They propose to use Brazilian Portuguese to analyze toxic or non-toxic tweets . they propose to analyze tweets as toxic or in different types of toxicity . |
| Outcome: | The proposed model achieves 76% macro-F1 score using monolingual data in the binary case. |
Enhancing Biomedical Lay Summarisation with External Knowledge Graphs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to lay summarisation are reliant on the source article, which is unlikely to include all the information necessary for a lay audience. |
| Approach: | They augment existing biomedical lay summarisation dataset with article-specific knowledge graphs that contain detailed information on relevant biomedically related concepts. |
| Outcome: | The proposed methods improve readability and explanation of technical concepts by integrating graph-based domain knowledge within lay summarisation models. |
Text Simplification from Professionally Produced Corpora (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to Text Simplification rely on the Wikipedia-Simple Wikipedia parallel corpus, which is used for many tasks. |
| Approach: | They propose to use the Newsela corpus to extract 550, 644 complex-simple sentence pairs from the corpus and introduce a lexical simplifier that uses the corpu to generate candidate simplifications. |
| Outcome: | The proposed model outperforms state-of-the-art approaches and generates candidate simplifications from the newsela corpus. |
Learning Simplifications for Specific Target Audiences (P18-2)
Copied to clipboard
| Challenge: | Text simplification is a monolingual text-to-text transformation task . data from TS data can contain multiple simplifications of the same original text . |
| Approach: | They propose to use sequence-to-sequence neural models to build models tailored for specific grade levels. |
| Outcome: | The proposed model outperforms state-of-the-art approaches for a monolingual text-to-text transformation task. |
Analysing State-Backed Propaganda Websites: a New Dataset and Linguistic Study (2023.emnlp-main)
Copied to clipboard
| Challenge: | a network of doppelganger websites (impersonating genuine news sites) was discovered in 2022 . a novel dataset enables studies of disinformation networks and the training of NLP tools for disinformation detection. |
| Approach: | They analyze two hitherto unstudied sites sharing state-backed disinformation . they perform cross-site topic clustering and perform linguistic and temporal analysis . |
| Outcome: | The proposed dataset includes 14,053 articles, annotated with each language version, and additional metadata such as links and images. |
Measuring the Impact of Readability Features in Fake News Detection (2020.lrec-1)
Copied to clipboard
Roney Santos, Gabriela Pedro, Sidney Leal, Oto Vale, Thiago Pardo, Kalina Bontcheva, Carolina Scarton
| Challenge: | Recent efforts to detect fake news use language-based approaches to detect news articles . authors show that readability features can improve classification accuracy . |
| Approach: | They propose to use readability features to detect fake news in the Brazilian Portuguese language . they show that such features can achieve up to 92% classification accuracy . |
| Outcome: | The proposed features achieve up to 92% accuracy and may improve previous classification results. |
SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain (L18-1)
Copied to clipboard
| Challenge: | lexical simplification is the task of reducing lexically and/or structural complexity of texts. |
| Approach: | They propose to collect manual simplifications for 1,100 original sentences using a sentence-level corpus from the Public Administration domain. |
| Outcome: | The proposed corpus contains 1,100 original sentences with manual simplifications collected through a two-stage process. |
Measuring What Counts: The Case of Rumour Stance Classification (2020.aacl-main)
Copied to clipboard
| Challenge: | Numerous methods have been proposed to predict the stance of replies towards a given rumour, but their performance is not optimal for the four-class imbalanced task of rumor stance classification. |
| Approach: | They propose to use a four-class problem to predict the stance of replies towards a given rumour to help identify the most informative minority classes. |
| Outcome: | The proposed methods are robust to imbalanced data and score higher systems capable of recognising the two most informative minority classes (support and deny). |
Improving Tokenisation by Alternative Treatment of Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | Subword tokenisation is a key initial step in processing natural language . it uses a number of different methods to tokenise text, including a stringsearching technique and a word-matching technique. |
| Approach: | They propose to use a vocabulary-based approach to tokenise text using a numerical ID and a mathematical function to manipulate it. |
| Outcome: | The method is based on a set of training data and learning from it to build a vocabulary and tokenise it at inference time using this vocabulary and learnt parameters. |
Can We Identify Stance without Target Arguments? A Study for Rumour Stance Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing target-aware models underperform in cases where the context of the target is crucial. |
| Approach: | They propose a framework to enhance reasoning with the targets and propose 'target-aware' models without awareness of the target. |
| Outcome: | The proposed framework achieves state-of-the-art on two benchmark datasets. |
It’s All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Low-resource languages, especially those written in rare scripts, remain unsupported by large language models due to lack of training data. |
| Approach: | They evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs and compare their methods to parameter-efficient fine-tuning. |
| Outcome: | The proposed methods compare with parameter-efficient fine-tuning (PEFT) on low-resource languages. |
Label Set Optimization via Activation Distribution Kurtosis for Zero-Shot Classification with Generative Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) performance is highly sensitive to prompt design, yet the impact of class label options (e.g. lexicon or order) in zero-shot classification remains underexplored. |
| Approach: | They propose a post-hoc method for selecting optimal label sets in zero-shot ICL with large language models. |
| Outcome: | The proposed method consistently achieves performance gains of 0.54 to 0.76 compared to the conventional method. |
Don’t waste a single annotation: improving single-label classifiers through soft labels (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for annotating data are limited by ambiguity and lack of context in data samples. |
| Approach: | They challenge the traditional approach of annotating data by only providing a single label for each sample and annotator disagreement is discarded . instead, they use additional annotation information such as confidence, secondary label and disagreement to generate soft labels. |
| Outcome: | The proposed method improves model performance and calibration on the hard label test set. |
Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science (2024.lrec-main)
Copied to clipboard
Yida Mu, Ben P. Wu, William Thorne, Ambrose Robinson, Nikolaos Aletras, Carolina Scarton, Kalina Bontcheva, Xingyi Song
| Challenge: | Existing instruction-tuned Large Language Models (LLMs) have impressive language understanding and the capacity to generate responses that follow specific prompts. |
| Approach: | They evaluate the zero-shot performance of two publicly accessible LLMs, ChatGPT and OpenAssistant, in the context of six Computational Social Science classification tasks. |
| Outcome: | The proposed LLMs perform better than state-of-the-art models on social science tasks. |
EASSE: Easier Automatic Sentence Simplification Evaluation (D19-3)
Copied to clipboard
| Challenge: | EASSE provides access to a broad range of evaluation resources including standard automatic metrics, word-level accuracy scores and reference-independent quality estimation features. |
| Approach: | They propose to provide a Python package that provides access to automatic evaluation and comparison of Sentence Simplification (SS) systems. |
| Outcome: | The proposed tool allows comparison and understanding of the performance of Sentence Simplification (SS) systems. |
Reference-less Analysis of Context Specificity in Translation with Personalised Language Models (2024.lrec-main)
Copied to clipboard
Sebastian Vincent, Rowanne Sumner, Alice Dowek, Charlotte Prescott, Emily Preston, Chris Bayliss, Chris Oakley, Carolina Scarton
| Challenge: | Conventional approaches to NLP tasks build models in a one-size-fits-all fashion disregarding the context of the processed text. |
| Approach: | They build LMs which leverage rich contextual information to reduce perplexity by up to 6.5% compared to a non-contextual model. |
| Outcome: | The proposed models reduce perplexity by up to 6.5% compared to non-contextual models and generalise well to a scenario with no speaker-specific data. |