| Challenge: | a project aims to create a system that uses automatic speech recognition (ASR) to produce real-time text captions of spoken English during in-person meetings with hearing individuals. |
| Approach: | They propose to use automatic speech recognition to produce captions in real-time . they add word-importance annotations to a transcript of a conversational dialogue corpus . |
| Outcome: | The proposed system would produce captions in real-time for people who are deaf or hard-of-hearing . the best performing model has an F-score of 0.60 in an ordinal 6-class word-importance classification task with an agreement (concordance correlation coefficient) of 0.89 with the human annotators. |
Similar Papers
Assessing Word Importance Using Models Trained for Semantic Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Many NLP tasks require to automatically identify the most significant words in a text. |
| Approach: | They propose to use attribution methods to explain the predictions of two NLP tasks to derive word significance from models trained to solve semantic tasks. |
| Outcome: | The proposed method is robust to the initial task and is able to identify important words in sentences without explicit word importance labeling in training. |
Document-Level Relation Extraction with Sentences Importance Estimation and Focusing (2022.naacl-main)
Copied to clipboard
| Challenge: | Document-level relation extraction models are not robust and exhibit bizarre behaviors when non-evidence sentences are removed. |
| Approach: | They propose a document-level relation extraction framework that uses a sentence importance score and a focusing loss to encourage DocRE models to focus on evidence sentences. |
| Outcome: | The proposed framework improves overall performance and makes DocRE models more robust. |
End-to-end ASR to jointly predict transcriptions and linguistic annotations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models generate audio transcripts by sequentially producing likely graphemes, or multi-graphemic units, from which lexical items of a language can be recovered. |
| Approach: | They propose a Transformer-based sequence-to-sequence model for automatic speech recognition that can produce high-quality transcriptions and linguistic annotations. |
| Outcome: | The proposed model can produce high-quality transcriptions and linguistic annotations on Japanese and English audio datasets. |
On Importance Sampling-Based Evaluation of Latent Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to evaluate language models using latent structures are intractable as they require marginalizing over the latent space. |
| Approach: | They propose to use importance sampling to evaluate latent language models . they elucidate subtle differences in how importance sampling is applied . |
| Outcome: | The proposed model performs better on tasks requiring structure and interpretability. |
Getting To Know You: User Attribute Extraction from Dialogues (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method to extract user attributes from dialogues is needed to improve user understanding. |
| Approach: | They propose to leverage dialogues with conversational agents to automatically extract user attributes from dialogues. |
| Outcome: | The proposed model surpasses retrieval and generation baselines on human evaluation. |
Designing Precise and Robust Dialogue Response Evaluators (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automated dialogue response evaluators have only moderate correlation with human judgement and are not robust. |
| Approach: | They propose to build a reference-free dialogue response evaluator that exploits the power of semi-supervised training and pretrained (masked) language models. |
| Outcome: | The proposed model achieves strong correlation with human judgement and generalizes robustly to diverse responses and corpora. |
Modeling Content Importance for Summarization with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on content importance do not consider semantics and context when evaluating importance. |
| Approach: | They apply information theory to pre-trained language models to define the concept of importance from the perspective of information amount. |
| Outcome: | Experiments on CNN/Daily Mail and New York Times show that the proposed model can model the importance of content better than previous methods based on F1 and ROUGE scores. |
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing dialogue summarization systems encode text with a number of general semantic features, but these are often not available in open-domain tools. |
| Approach: | They propose to use DialoGPT to label three types of features on two datasets . they propose to employ pre-trained and non-pre-tried models as dialogue annotators . |
| Outcome: | The proposed method improves on two dialogue summarization datasets and achieves state-of-the-art performance. |
LIDA: Lightweight Interactive Dialogue Annotator (D19-3)
Copied to clipboard
| Challenge: | Dialogue systems are dependent on the quality of the data used to train them. |
| Approach: | They propose to develop an annotation tool specifically for conversation data that handles the entire dialogue annotation pipeline from raw text to structured conversation data. |
| Outcome: | The proposed tool handles the entire dialogue annotation pipeline from raw text to structured conversation data and has a dedicated interface to resolve inter-annotator disagreements. |
Can LLMs Understand the Implication of Emphasized Sentences in Dialogue? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Emphasis is a crucial component in human communication, which indicates speaker’s intention and implication beyond pure text in dialogue. |
| Approach: | They propose a benchmark dataset with annotated dialogue samples capturing the implications of emphasis. |
| Outcome: | The proposed evaluation pipeline achieves high correlation with human scoring and commercial LLMs perform better than open-source LLM. |