Papers by Gerasimos Lampouras
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results indicate that TARG achieves state-of-the-art performance in knowledge selection and response generation, outperforming previous state- of-the art by 3.2, 3.6, and 4.2 points in EM, F1 and BLEU-4 respectively on Doc2Dial. |
| Approach: | They propose to integrate topical information into knowledge-grounded task-oriented dialogue systems by using multiple topic-aware attention mechanisms to derive the importance weighting scheme over dialogue utterances and external knowledge sources. |
| Outcome: | The proposed model outperforms existing models in knowledge selection and response generation. |
Informed Sampling for Diversity in Concept-to-Text NLG (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to encourage lexical diversity for language generation tasks produce repetitive outputs, but this often comes at a cost to the perceived fluency and adequacy of the output. |
| Approach: | They propose to augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output. |
| Outcome: | The proposed method achieves a high level of diversity with minimal effect on the output’s fluency and adequacy. |
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches for NLU use CL to improve in-distribution data performance via heuristic-oriented or task-agnostic difficulties. |
| Approach: | They propose to use CL to improve in-distribution data performance by taking advantage of training dynamics as difficulty metrics instead of heuristic-oriented or task-agnostic difficulties. |
| Outcome: | The proposed model schedulers improve on in-distribution, out-of-distortion and zero-shot cross-lingual transfer datasets while being 20% faster on average. |
Conversation Graph: Data Augmentation, Training, and Evaluation for Non-Deterministic Dialogue Management (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing datasets limited in size con- sidering complexity of dialogues . current trends lean towards end-to-end models while modular systems tend to be preferred in industrial applications. |
| Approach: | They propose a graph-based representation of dialogues that can be exploited for data augmentation, multi- reference training and evaluation of non-deterministic agents. |
| Outcome: | The proposed graph-based representation of dialogues can be exploited for data augmentation, multi- reference training and evaluation of non-deterministic agents. |
DReSD: Dense Retrieval for Speculative Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding (SD) uses an efficient draft model to propose the next few tokens, which are verified by the LLM in a single forward call, reducing latency while preserving its outputs. |
| Approach: | They propose a draft model that proposes the next few tokens from a non-parametric datastore and uses a framework that uses approximate nearest neighbour search with contextualised token embeddings to retrieve the most semantically relevant sequences for SD. |
| Outcome: | The proposed framework achieves (on average) 87% higher acceptance rates, 65% longer accepted tokens and 19% faster generation speeds compared to sparse retrieval (REST). |
Process Evaluation for Agentic Systems (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent adoption of LLM-based assistants has led to premature assumptions about their reliability and general capability. |
| Approach: | They propose to assess the feasibility of automatic process evaluation for critical applications such as medicine, finance, law and infrastructure. |
| Outcome: | The proposed evaluations are based on a small-scale study to assess the feasibility of automated process evaluation, present a compliance score, analyse use cases of bad and good behaviours, and offer recommendations for more holistic evaluation. |
Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have shown remarkable performance on various benchmarks, treating the problem of Code Generation in a fashion similar to Natural Language Generation, trained with a Language Modelling (LM) objective. |
| Approach: | They propose a method to automatically obtain function signatures and associated Unit Tests, suitable for RL training of Code Synthesis models. |
| Outcome: | The proposed model improves on a pre-trained model on various benchmarks and is more efficient than previous models. |
Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation (2021.acl-long)
Copied to clipboard
| Challenge: | Concept-to-text Natural Language Generation requires a delexicalisation of the input, but this often requires that the input appears verbatim in the output text. |
| Approach: | They propose a method that uses multilingual pretrained embeddings to inflect words in their correct form during relexicalisation. |
| Outcome: | The proposed method outperforms monolingual models in concept-to-text and in low resource environments. |
Hierarchical Recurrent Aggregative Generation for Few-Shot NLG (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches do not account for the fact that some sub-tasks, specifically aggregation and lexicalisation, can benefit from transfer learning in different extents. |
| Approach: | They propose a hierarchical approach for few-shot and zero-shot generation using a three-moduled jointly trained architecture. |
| Outcome: | The proposed approach achieves state-of-the-art on few-shot and zero-shot settings compared to previous approaches. |
Text-to-Code Generation with Modality-relative Pre-training (2024.eacl-long)
Copied to clipboard
| Challenge: | Large pre-trained language models have been applied to programming language tasks with great success, often through further pre-training of a strictly-natural language model. |
| Approach: | They propose to map programming language modalities into the same embedding space by separating embeddable spaces between modality and modality-relative training objectives. |
| Outcome: | The proposed model can be adapted and represented differently depending on which modality they belong to and to the ultimate benefit of the downstream task. |
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current direct preference optimization algorithms focus on a strict set of tokens contributing signals of KL divergence and rewards to the loss function. |
| Approach: | They propose a method that automatically learns to weight the KL divergence and reward corresponding to each token during PO training. |
| Outcome: | The proposed method achieves +10% and +3% win-rate points in two PO scenarios. |
Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that CLMs can generate accurate solutions with no regard for runtime, but at a substantial cost to correctness (down by up to 30%) |
| Approach: | They propose a framework that incorporates correctness and runtime as learning signals via self-generated preference data. |
| Outcome: | The proposed framework reduces the baseline runtimes by 6% and the average length of the generated solutions is reduced by up to 48% on MBPP and 23% on HumanEval. |
HumanRankEval: Automatic Evaluation of LMs as Conversational Assistants (2024.naacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) are popular conversational assistants, but evaluation of such models is not scalable. |
| Approach: | They propose a task that performs automatic evaluation using human judgement and a large-scale set of questions with multiple answers authored and scored by humans. |
| Outcome: | The proposed task performs well with human judgements and is particularly responsive to model changes following instruction-tuning. |