Papers by Filip Ilievski
Contextualizing Argument Quality Assessment with Relevant Knowledge (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for assessing argument quality in isolation analyze their quality in the absence of context, which affects their accuracy and generalizability. |
| Approach: | They propose a method for scoring argument quality based on contextualization via relevant knowledge that leverages large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. |
| Outcome: | The proposed method outperforms existing methods across multiple metrics in both in-domain and zero-shot setups. |
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not have commonsense knowledge or cannot reason with it. |
| Approach: | They propose to augment a general commonsense QA framework with a knowledgeable path generator by extrapolating existing paths from a KG with 'state-of-the-art' language model. |
| Outcome: | The generated paths are interpretable, novel, and relevant to the task. |
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | lateral thinking tasks require implicit and complex reasoning, relying on human-like commonsense mechanisms. |
| Approach: | They propose a lateral thinking benchmark to test models' ability to exhibit lateral reasoning and defy default commonsense associations. |
| Outcome: | The proposed model exhibits lateral thinking and defies default commonsense associations. |
Robust Text Classification: Analyzing Prototype-Based Networks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Language models exhibit a drop in performance on noisy data, which can cause classifiers to incorrectly change their predictions. |
| Approach: | They propose to use Prototype-Based Networks to classify examples based on their similarity to prototypical examples of a class (prototypes) they show that PBNs offer more robustness under both targeted and static adversarial attacks. |
| Outcome: | The proposed model is robust to noise and targets both targeted and static attacks. |
Coalescing Global and Local Information for Procedural Text Understanding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for procedural text understanding have low precision or low recall . et al., 2012, pp. 106-106. |
| Approach: | They propose a model that builds entity- and timestep-aware input representations . they extend the model with additional output layers and integrate it into a story reasoning framework . |
| Outcome: | The proposed model achieves state-of-the-art on a popular procedural text understanding dataset and on 'story reasoning benchmark' it integrates the model with additional output layers and improves on the previous models. |
Systematic Study of Long Tail Phenomena in Entity Linking (C18-1)
Copied to clipboard
| Challenge: | Existing systems for entity linking are based on frequent 'head' cases, while performance drops when moving towards rare 'long tail' entities. |
| Approach: | They propose to use a long tail to investigate the properties of entity linking datasets. |
| Outcome: | The proposed systems overfit to popular/frequent and non-ambiguous cases and find the most difficult cases among the infrequent candidates of ambiguous forms. |
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work proposes lightweight updates to improve commonsense reasoning models . fine-tuning can cause models to overfit to task-specific data and forget knowledge gained during training . |
| Approach: | They propose to use lightweight models to update pre-trained language models to learn commonsense background knowledge. |
| Outcome: | The proposed models learn from commonsense reasoning datasets, but they are overfitted and limited generalized. |
Representing Numbers in NLP: a Survey and a Vision (2021.naacl-main)
Copied to clipboard
| Challenge: | Numeracy is an essential skill for language understanding since numbers are often interspersed in text. |
| Approach: | They propose a comprehensive taxonomy of tasks and methods to represent numbers in text . they synthesize best practices for representing numbers in texts and articulate a vision for holistic numeracy . |
| Outcome: | The proposed model synthesizes best practices for representing numbers in text . it argues that the model is more effective than other approaches . |
Do Language Models Perform Generalizable Commonsense Inference? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has applied pretrained language models to populate commonsense knowledge graphs (CKGs) but there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. |
| Approach: | They analyze the ability of pretrained language models to perform generalizable commonsense inference in terms of knowledge capacity, transferability and induction. |
| Outcome: | The proposed models can adapt to different schemas defined by multiple CKGs but fail to generalize to new relations. |
Don’t Annotate, but Validate: a Data-to-Text Method for Capturing Event Data (L18-1)
Copied to clipboard
| Challenge: | Existing methods to create event data are limited by ambiguity and variation in the data. |
| Approach: | They propose a method to obtain large volumes of text corpora with event data . they use a tool to annotate texts and enrich the reference texts with event coreference annotations. |
| Outcome: | The proposed method obtains large volumes of high-quality text corpora with event data . the data obtained with this method have high precision and at a large scale . |
PaCo: Preconditions Attributed to Commonsense Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models can reason with circumstantial preconditions of commonsense knowledge, but they do not understand the circumstancial precondition. |
| Approach: | They propose to use a dataset to examine the ability of existing language models to understand circumstantial preconditions to improve their reasoning with commonsense knowledge. |
| Outcome: | The proposed task shows that human reasoning with preconditions is an open challenge. |
Numeracy enhances the Literacy of Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Specialized number representations have shown improvements on numerical reasoning tasks like arithmetic word problems and masked number prediction. |
| Approach: | They propose to use six different number encoders to improve masked word prediction by avoiding conflating nominal and ordinal number occurrences. |
| Outcome: | The proposed representations improve masked word prediction accuracy and generalize to contexts without annotated numbers. |
Large-scale Cross-lingual Language Resources for Referencing and Framing (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing corpora that capture language but do not represent actual situations hinder development of systems to resolve cross-document coreference. |
| Approach: | They introduce the concept of cross-lingual referential corpora and propose a framework to analyze framing . they expect to capture larger variation in framation compared to traditional approaches . |
| Outcome: | The proposed project will analyze the framing of incidents in different languages and texts . it expects to capture larger variation in framation compared to traditional approaches . |