When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages (2021.naacl-industry)
Copied to clipboard
| Challenge: | Prior-message context provides the greatest lift in Teams (chat) scenario. |
| Approach: | They compare prior-message context with email and chat messages from Microsoft Teams and Outlook. |
| Outcome: | The proposed model outperforms existing models on two of the largest commercial communication platforms: Microsoft Teams and Outlook. |
Similar Papers
On the Role of Context in Reading Time Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new perspective on how readers integrate context during reading time prediction is presented . a recent study shows that the proportion of variance in reading times explained by context is smaller when context is represented by the orthogonalized predictor. |
| Approach: | They propose a technique where they project surprisal onto the orthogonal complement of frequency. |
| Outcome: | The proposed method shows that the proportion of variance in reading times explained by context is smaller when context is represented by the orthogonalized predictor. |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |
Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers (P18-2)
Copied to clipboard
| Challenge: | cloze deletion test is a test that requires the learner to understand the context and vocabulary in order to identify the correct word. |
| Approach: | They collect data from human participants and test various models in a local and a global context condition to examine the role of linguistic context in predicting quantifiers. |
| Outcome: | The proposed models outperform humans in a local and global context and are only slightly better in the latter. |
An Empirical Investigation of Contextualized Number Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large scale empirical investigation of contextualized number prediction in running text is needed. |
| Approach: | They propose a suite of output distribution parameterizations that incorporate latent variables to add expressivity and better fit the natural distribution of numeric values in running text. |
| Outcome: | The proposed models outperform flow-based models on two numeric datasets in the financial and scientific domain. |
Joint Effects of Context and User History for Predicting Online Conversation Re-entries (P19-1)
Copied to clipboard
| Challenge: | Existing methods for predicting online conversation re-entry focus on modeling engagement patterns in ongoing conversations or ignoring the rich information in users' previous chatting history. |
| Approach: | They propose a neural framework with three main layers to model the conversation context and user history and their interactions with Twitter and Reddit to predict whether a user will return to a conversation they once participated in. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two large-scale Twitter and Reddit conversations, and achieves an F1 score of 61.1 on Twitter conversations. |
Context versus Prior Knowledge in Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies have investigated how often a model will rely on prior knowledge over conflicting contextual information in answering questions. |
| Approach: | They propose two mutual information-based metrics to measure a model’s dependency on a context and on its prior about an entity. |
| Outcome: | The proposed metrics show that language models can integrate prior knowledge and new information in a predictable way across different questions and contexts. |
Survival text regression for time-to-event prediction in conversations (2021.findings-acl)
Copied to clipboard
| Challenge: | a recent study has modelled time-to-event prediction tasks as classification tasks . authors: this is contrived and less informative than traditional classification models . |
| Approach: | They propose to frame time-to-event prediction tasks as classification tasks . they use survival regression techniques commonly used in healthcare and reliability engineering . |
| Outcome: | The proposed models outperform text regression methods and comparable classification models on three datasets. |
In-Context Learning (and Unlearning) of Length Biases (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing work has demonstrated the ability of large language models to learn lexical and label biases in-context negatively impacts performance and robustness of models. |
| Approach: | They investigate the impact of length biases on in-context learning by analyzing model length information in-constext. |
| Outcome: | The proposed model learns length biases in the context window without parameter updates. |
Are Emergent Abilities in Large Language Models just In-Context Learning? (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models have been claimed to acquire certain capabilities without having been specifically trained on them. |
| Approach: | They propose a theory that explains emergent abilities by taking into account their potential confounding factors and rigorously substantiate this theory through over 1000 experiments. |
| Outcome: | The proposed theory proves that emergent abilities are not truly emergental, but result from a combination of in-context learning, model memory, and linguistic knowledge. |
Improving Backchannel Prediction Leveraging Sequential and Attentive Context Awareness (2024.findings-eacl)
Copied to clipboard
| Challenge: | Backchannels are short and often affirmative or empathetic responses from a listener during a conversation . et al. (2010) showed that timely backchanneling can enhance storytelling ability . |
| Approach: | They propose a context-aware backchannel prediction approach that leverages a pretrained wav2vec model to enhance backchannel performance. |
| Outcome: | The proposed approach improves performance in Korean and English datasets . it leverages the pretrained wav2vec model for encoding audio signal . |