Papers by Prodromos Malakasiotis
LEGAL-BERT: The Muppets straight out of Law School (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing guidelines for pre-training and fine-tuning do not always generalize well in the legal domain. |
| Approach: | They propose to use BERT out of the box, adapt it by additional pre-training on domain-specific corpora, and pre-train it from scratch on domains. |
| Outcome: | The proposed strategies are: use the original BERT out of the box, adapt it by additional pre-training on domain-specific corpora, and pre-train it from scratch on domain specific corpors. |
SUM-QE: a BERT-based Summary Quality Estimation Model (D19-1)
Copied to clipboard
| Challenge: | SUM-QE is a quality estimation model for summarization that captures linguistic qualities that traditional evaluation metrics fail to capture. |
| Approach: | They propose a new quality estimation model based on BERT that addresses linguistic quality aspects that are only indirectly captured by content-based approaches to summary evaluation without comparison with human ratings. |
| Outcome: | The proposed model outperforms existing models on linguistic quality aspects that are only indirectly captured by content-based summarization evaluations without comparison with human ratings. |
Should I try multiple optimizers when fine-tuning a pre-trained Transformer for NLP tasks? Should I tune their hyperparameters? (2024.eacl-long)
Copied to clipboard
| Challenge: | NLP research has explored different neural model architectures and sizes, datasets, training objectives, and transfer-learning techniques. |
| Approach: | They propose to use a variant of Stochastic Gradient Descent (SGD) to select among numerous variants, often with minimal or no tuning of the optimizer’s hyperparameters. |
| Outcome: | Experiments with five GLUE datasets, two models and seven popular optimizers show that tuning just the learning rate is as good as tuning all the hyperparameters. |
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases (2021.naacl-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis
| Challenge: | Interpretability or explainability is an emerging field of research in NLP . experimental results indicate that the newly introduced task is very challenging . |
| Approach: | They propose to extract rationales as paragraphs in multi-paragraph structured court cases . they also propose a constraint that allows models to be more specific . |
| Outcome: | The proposed task is very challenging and there is a large scope for further research. |
FiNER: Financial Numeric Entity Recognition for XBRL Tagging (2022.acl-long)
Copied to clipboard
Lefteris Loukas, Manos Fergadiotis, Ilias Chalkidis, Eirini Spyropoulou, Prodromos Malakasiotis, Ion Androutsopoulos, Georgios Paliouras
| Challenge: | Publicly traded companies are required to submit periodic reports with eXtensive Business Reporting Language (XBRL) word-level tags. |
| Approach: | They propose to use XBRL tagging as a new entity extraction task for the financial domain and release FiNER-139, a dataset of 1.1M sentences with gold X brl tags. |
| Outcome: | The proposed solution replaces numeric expressions with pseudo-tokens reflecting original token shapes and numeric magnitudes. |
Large-Scale Multi-Label Text Classification on EU Legislation (P19-1)
Copied to clipboard
| Challenge: | Large-Scale multi-label text classification is a task of assigning to each document all the relevant labels from a large set, typically containing thousands of labels (classes). |
| Approach: | They propose to use a dataset of 57k English EU legislative documents annotated with 4.3k EUROVOC labels for LMTC, few-shot learning and contextual embeddings. |
| Outcome: | The proposed dataset is suitable for LMTC, few- and zero-shot learning and bypasses the maximum text length limit. |
Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations (2021.eacl-main)
Copied to clipboard
| Challenge: | Major scandals in corporate history have urged the need for regulatory compliance, where organizations need to ensure that their controls (processes) comply with relevant laws, regulations, and policies. |
| Approach: | They introduce regulatory information retrieval (REG-IR) an application of document-to-document information retrievals where the query is an entire document making the task more challenging than traditional IR where the queries are short. |
| Outcome: | The proposed approach is more challenging than traditional IR where the query is an entire document making the task more challenging. |
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels (2020.emnlp-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos
| Challenge: | Large-scale Multi-label Text Classification (LMTC) is a type of classification that assigns labels to a large set of labels. |
| Approach: | They propose to use probabilistic label trees to improve frequent, few and zero-shot learning . they propose to combine a new state-of-the-art method with pre-trained Transformers . |
| Outcome: | The proposed models outperform existing models on frequent, few and zero-shot learning on three datasets from different domains. |
Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prompting Large Language Models (LLMs) performs impressively in zero- and few-shot settings. |
| Approach: | They propose a framework that allows reducing calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side. |
| Outcome: | The proposed framework reduces calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side. |