| Challenge: | In a large-scale observational study of a spoken corpus, we find that language production at a time point preceding a word is sped up or slowed down depending on activation of that word. |
| Approach: | They propose a cognitive model of fluency in which lexical memory retrievals may explain some of the variability in speech rates. |
| Outcome: | The proposed model predicts that language production is sped up or slowed down depending on activation of a word . |
Similar Papers
Are Large Language Model Temporally Grounded? (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent large language models lack a consistent temporal model of textual narratives . sentence ordering in unlabelled texts is only weakly correlated with event ordering . |
| Approach: | They evaluate LLMs with textual narratives and evaluate their common-sense knowledge . they find that LLM models struggle the most with self-consistency . |
| Outcome: | The proposed models lack a consistent temporal model of textual narratives. |
Analysing Lexical Semantic Change with Contextualised Word Representations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on lexical semantic change have focused on detecting and characterising word meaning shifts using distributional semantic models. |
| Approach: | They propose a method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. |
| Outcome: | The proposed method captures a variety of synchronic and diachronic linguistic phenomena and is highly reproducible and reproducible. |
Lexicosyntactic Inference in Neural Models (D18-1)
Copied to clipboard
| Challenge: | lexicosyntactic inferences are triggered by surprising aspects of the syntactical context that a word occurs in. |
| Approach: | They build a factuality judgment dataset for English clause-embedding verbs in various syntactic contexts and use it to probe the behavior of current state-of-the-art neural systems. |
| Outcome: | The proposed model makes systematic errors that are visible through the lens of factuality prediction. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |
Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change (P19-1)
Copied to clipboard
| Challenge: | State-of-the-art lexical semantic change detection models suffer from noise stemming from vector space alignment. |
| Approach: | They propose a method to simulate lexical semantic change and control for possible biases by avoiding alignment. |
| Outcome: | The proposed method outperforms state-of-the-art models on a synthetic task and a manual testset. |
More room for language: Investigating the effect of retrieval on language models (2024.naacl-short)
Copied to clipboard
| Challenge: | Retrieval-augmented language models are a promising alternative to standard pretraining, but little attention has been put into understanding what this type of training scheme does to the underlying language model when analyzed as a standalone -separated from the overall retrieval pipeline. |
| Approach: | They propose an ‘ideal retrieval’ methodology to study these models in a fully controllable setting and propose a retrieval augmentation methodology to examine their effects. |
| Outcome: | The proposed model saves substantially less world knowledge in their weights, but is worse at comprehending global context. |
A Neural Model of Adaptation in Reading (D18-1)
Copied to clipboard
| Challenge: | Several studies suggest that readers do adapt their lexical and syntactic predictions to the current context. |
| Approach: | They propose to add a simple adaptation mechanism to a neural language model to improve predictions of reading times. |
| Outcome: | The proposed model improves predictions of human reading times compared to a non-adaptive model. |
Remember This Event That Year? Assessing Temporal Information and Understanding in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly ubiquitous, yet their ability to effectively retain and reason about temporal information remains limited. |
| Approach: | They propose six metrics to assess three learning paradigms to enhance temporal knowledge acquisition. |
| Outcome: | The proposed methods improve performance and reduce incorrect outputs. |
Retrieval-based Language Models and Applications (2023.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will provide a comprehensive overview of retrieval-based language models. |
| Approach: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
| Outcome: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
The impact of lexical and grammatical processing on generating code from natural language (2022.findings-acl)
Copied to clipboard
| Challenge: | Yin and Neubig (2018) identify four key components of importance for natural language to code translation. |
| Approach: | They propose a seq2seq-based architecture that relies on a grammar-based decoder and a lexical substitution component for natural language to code translation. |
| Outcome: | The proposed architecture relies on a grammar-based decoder and a BERT encoder . the proposed architecture is based on lexical substitutions in natural language to code translation . |