| Challenge: | Pre-trained systems are able to capture advice better than rule-based systems, but advice identification is challenging. |
| Approach: | They analyze a dataset of advice posts on two reddit forums and annotate whether they contain advice. |
| Outcome: | The proposed models show that pre-trained models capture advice better than rule-based systems, but advice identification is challenging. |
Similar Papers
How Can We Know What Language Models Know? (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work examines knowledge contained in language models by having the LM fill in the blanks of prompts such as “Obama is a __ by profession”. |
| Approach: | They propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. |
| Outcome: | The proposed methods improve accuracy from 31.1% to 39.6% on the LAMA benchmark for extracting relational knowledge from LMs. |
Exploring Self-Identified Counseling Expertise in Online Support Forums (2021.findings-acl)
Copied to clipboard
Allison Lahnala, Yuntian Zhao, Charles Welch, Jonathan K. Kummerfeld, Lawrence C An, Kenneth Resnicow, Rada Mihalcea, Verónica Pérez-Rosas
| Challenge: | Increasing number of people engage in online health forums, making it important to understand the quality of the advice they receive. |
| Approach: | They examine the role of expertise in responses to help-seeking posts . they find that a classifier can distinguish between peer and self-identified mental health professionals' interactions . |
| Outcome: | The findings show that experts' language use differs between groups, and that their comments engage the support-seeker further. |
TuringAdvice: A Generative and Dynamic Evaluation of Language Use (2021.naacl-main)
Copied to clipboard
| Challenge: | Empirical results show that today’s language models struggle at TuringAdvice . language models are getting ever-larger, and are being trained on ever-increasing quantities of text . |
| Approach: | They propose a task task that requires models to generate helpful advice in natural language. |
| Outcome: | The proposed model outperforms even multibillion parameter models on 600k in-domain training examples. |
Reasoning with Language Model Prompting: A Survey (2023.acl-long)
Copied to clipboard
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Huajun Chen
| Challenge: | Reasoning is an essential ability for complex problem-solving and can provide back-end support for various real-world applications. |
| Approach: | They present cutting-edge research on reasoning with language model prompting and provide systematic resources to help beginners. |
| Outcome: | The proposed approaches have not been systematically reviewed and analyzed. |
A Study on Accessing Linguistic Information in Pre-Trained Language Models by Using Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to access linguistic information in pre-trained multilingual language models are difficult to use. |
| Approach: | They propose prompting and formulate linguistic tasks to test the LM's access to explicit grammar principles and find out what type of information can be obtained . |
| Outcome: | The proposed method can provide access to linguistic features in pre-trained models, but some are harder to capture . |
Improving Natural Language Interaction with Robots Using Advice (N19-1)
Copied to clipboard
| Challenge: | Recent studies focus on learning models for physically grounded language understanding tasks such as the blocks world domain. |
| Approach: | They propose a protocol for including advice, high-level observations about the task, which can help constrain the agent’s prediction. |
| Outcome: | The proposed approach can be extended to include advice, high-level observations about the task, and reduce the effort involved in supplying the advice. |
TalkUp: Paving the Way for Understanding Empowering Language (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Empowerment has rarely been studied in NLP because of its implicit nature . linguistics and psychology research shows how empowerment can impact people by increasing their sense of self-efficacy and self-esteem. |
| Approach: | They crowdsource Reddit posts labeled for empowerment and use it to train language models that capture empowering and disempowering language. |
| Outcome: | The proposed dataset can be used to train language models that capture empowering and disempowering language. |
How Much Knowledge Can You Pack Into the Parameters of a Language Model? (2020.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that large neural language models trained on unstructured text can attain competitive results on open-domain question answering benchmarks without access to external knowledge. |
| Approach: | They propose to fine-tune pre-trained neural language models to answer questions without external knowledge . they show that this approach scales with model size and performs competitively . |
| Outcome: | The proposed approach scales with model size and performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source when answering questions. |
Learning Strategies for Robust Argument Mining: An Analysis of Variations in Language and Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument mining is a complex process that requires a large amount of resources and time. |
| Approach: | They propose to analyze arguments in three different languages and domains to understand their robustness to natural language variations. |
| Outcome: | The proposed systems are more robust to natural language variations than existing arguments mining systems. |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |