| Challenge: | Existing methods to evaluate text data are rarely reported by taking the timestamp of the document into account. |
| Approach: | They propose methods that make better use of temporally-diverse training data with a focus on named entity recognition. |
| Outcome: | The proposed models make better use of temporally-diverse training data, with a focus on named entity recognition. |
Similar Papers
Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts (2022.aacl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a longstanding NLP task that consists of identifying an entity in a sentence or document. |
| Approach: | They construct a dataset of seven entity types annotated over 11,382 tweets . they provide a set of language model baselines and analyze the performance of the model . |
| Outcome: | The proposed dataset contains seven entity types annotated over 11,382 tweets . the authors focus on short-term degradation of NER models over time and strategies to fine-tune a language model over different periods . |
Temporal Effects on Pre-trained Models for Language Processing Tasks (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study shows that language models can be improved as time passes . a number of approaches to solving language tasks have evolved rapidly without a model . |
| Approach: | They examine temporal effects on model performance on downstream language tasks . they also examine the efficacy of two approaches for temporal domain adaptation without human annotations . |
| Outcome: | The proposed methods improve self-labeling and named entity recognition on new data. |
Entity Cloze By Date: What LMs Know About Unseen Entities (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing literature provides benchmarks to measure LMs' knowledge about entities . |
| Approach: | They propose a framework to analyze what language models can infer about new entities that did not exist when they were pretrained. |
| Outcome: | The proposed framework shows that models more informed about the entities achieve lower perplexity on this benchmark. |
Neural Temporality Adaptation for Document Classification: Diachronic Word Embeddings and Domain Adaptation Models (P19-1)
Copied to clipboard
| Challenge: | Recent studies show that document classifiers can become more stable over time when trained in ways that account for temporal variations. |
| Approach: | They propose a method for embedding diachronic word embedds into document classification models . they propose 'time-driven neural classification model' that accounts for temporal variations . |
| Outcome: | The proposed model can be trained on six corpora and make it more robust over time. |
Time-Aware Language Models as Temporal Knowledge Bases (2022.tacl-1)
Copied to clipboard
Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
| Challenge: | Existing language models are trained on snapshots of data collected at a specific moment in time. |
| Approach: | They propose a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time. |
| Outcome: | The proposed method improves memorization of seen facts and calibration on unseen facts from future time periods. |
A Twitter Corpus for Named Entity Recognition in Turkish (2022.lrec-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a subtask of information extraction that uses predefined named entities to identify NEs in noisy texts. |
| Approach: | They propose to use a Turkish Twitter Named Entity Recognition dataset to identify predefined named entities (NEs) the dataset contains 5000 tweets from a year-long period with a high agreement score. |
| Outcome: | The proposed dataset contains 5000 tweets from a year-long period and has high agreement scores. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Tackling Temporal Questions in Natural Language Interface to Databases (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Temporal aspect is one of the most challenging areas in Natural Language Interface to Databases (NLIDB). |
| Approach: | They propose a dataset with accompanied databases supporting temporal questions in NLIDB. |
| Outcome: | The proposed dataset helps two models learn and improve in temporal aspect. |
Is Your LLM Outdated? A Deep Look at Temporal Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate large language models are limited due to their inherent dynamic nature and the inherent dynamicity of language and information. |
| Approach: | They introduce a new evaluation framework that employs fresh text and event prediction for assessing LLMs’ temporal adaptability. |
| Outcome: | The proposed framework shows significant temporal biases and a decline in performance over time. |
On the Impact of Temporal Concept Drift on Model Explanations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation faithfulness of model predictions is typically evaluated on held-out data from the same temporal distribution as the training data. |
| Approach: | They examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. |
| Outcome: | The proposed method shows the most robust faithfulness scores across datasets and in asynchronous settings. |