Papers by Laura Aina
Putting Words in Context: LSTM Language Models and Lexical Ambiguity (P19-1)
Copied to clipboard
| Challenge: | In language, a word can contribute a very different meaning depending on the context . lexical ambiguity involves both morphosyntactic and semantic aspects . |
| Approach: | They propose a method to probe hidden representations for lexical and contextual information about words. |
| Outcome: | The proposed method shows that both types of information are represented to a large extent, but there is room for improvement for contextual information. |
Understanding and Improving Information Preservation in Prompt Compression for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled their successful application to a broad range of tasks. |
| Approach: | They propose a framework that allows for in-depth analysis of prompt compression methods. |
| Outcome: | The proposed framework analyzes state-of-the-art soft and hard compression methods . it shows that some fail to preserve key details from the original prompt, limiting performance on complex tasks. |
Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) tend to be unreliable on fact-based answers. |
| Approach: | They propose a framework for comparing LLMs' confidence over fact-based answers with hidden-state probes that are more reliable than hidden-status probes. |
| Outcome: | The proposed methods show that hidden-state probes provide the most reliable confidence estimates despite requiring access to weights and supervision data. |
Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) are state-of-the-art when adapted to text classification tasks. |
| Approach: | They compare fine-tuning, prompting, and knowledge distillation procedures to train pre-trained language models to downstream tasks. |
| Outcome: | The proposed training procedures perform better when trained with fine-tuning or prompting on large train sets than when trained by prompting or fine-untun. |
What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue (N19-1)
Copied to clipboard
| Challenge: | a recent study suggests that models that incorporate a bias towards learning entity representations are not effective at modeling entities. |
| Approach: | They propose to use two entity-centric models for a referential task . they show they outperform the state of the art and do better on lower frequency entities . |
| Outcome: | The proposed models outperform the state of the art on a referential task . they do better on lower frequency entities than a counterpart model not entity-centric . |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |