Papers with end-to-end
Copied to clipboard
| Challenge: | This tutorial focuses on the evolution of voice-native LLMs . it reviews the adaptation of text LLM to audio, cross-modal alignment, and joint speech–text training . |
| Approach: | This tutorial examines the evolution of voice-native LLMs in conversational agents . it compares cascaded and voice-based LLM systems to end-to-end retrieval-and vision-grounded systems . |
| Outcome: | This tutorial examines the evolution of voice-native LLMs . it compares the performance of voice assistants to current open-domain agents . |
Copied to clipboard
| Challenge: | tutorial aims to familiarise research community with recent advances in statistical dialogue systems . focus of tutorial is on learning end-to-end from data and their relation to more common modular systems. |
| Approach: | This tutorial aims to familiarise the research community with the latest advances in statistical dialogue systems . the focus of the tutorial is on recently introduced end-to-end learning for dialogue systems and their relation to more common modular systems. |
| Outcome: | This tutorial aims to familiarise the research community with the recent advances in statistical dialogue systems for open-domain and task-based dialogue paradigms. |
Copied to clipboard
| Challenge: | Recent work on incorporating external knowledge into the response generation models has attracted great interest. |
| Approach: | They propose a neural entity linking approach to incorporate external knowledge into the response generation models to improve the relevancy of retrieved knowledge. |
| Outcome: | The proposed approach outperforms the baseline model by 62.8% relative to the baseline. |
Copied to clipboard
| Challenge: | Neural codec language models (or codec LMs) are emerging as a powerful framework for text-to-speech (TTS) despite the close interdependence of codecs and LM, research on codec and lms has largely remained siloed. |
| Approach: | They propose a frame-wise codec encoder that improves both LM log-likelihood and TTS metrics . they also propose LM codebook level dropout to efficiently navigate a portion of codec-LM design space . |
| Outcome: | The proposed codec-LM co-design improves intelligibility, audio quality and speaker control compared to a siloed baseline. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have made it difficult to build an automated debate system that helps people to synthesise persuasive arguments. |
| Approach: | They propose to use an argument mining dataset to capture the end-to-end process of preparing an argumentative essay for a debate. |
| Outcome: | The proposed dataset shows that it performs better on individual tasks than on human-centred evaluations. |
Copied to clipboard
| Challenge: | Existing methods to plan in textual environments rely on a fully-observed environment where all entity states are known, but are not interpretable. |
| Approach: | They propose to use LLMs to generate a formal representation of the environment that can be solved by a symbolic planner. |
| Outcome: | The proposed model outperforms existing methods in the Coin Collector simulation and Cooking World simulations. |
Copied to clipboard
| Challenge: | evaluating and troubleshooting production TOD systems is still a largely manual process requiring large amount of human conversations with the systems. |
| Approach: | They propose a data-efficient end-to-end Bot SIMulation framework for commercial task-oriented dialog systems that can generate user queries and generate semantic-level dialog acts and entities from bot definitions. |
| Outcome: | The proposed framework is able to infer semantic-level dialog acts and entities from bot definitions and generate user queries via model-based paraphrasing. |
Copied to clipboard
| Challenge: | Indic languages are resource-scarce and do not have such parallel data due to low volume of queries. |
| Approach: | They propose a sequence-to-sequence deep learning model which trains end-to end for Indic languages, Hindi and Telugu. |
| Outcome: | The proposed model is competitive with existing spell checking and correction techniques for Indic languages. |
Copied to clipboard
| Challenge: | Recent deep learning methods for anomalies in images learn better features of normality in an end-to-end self-supervised setting. |
| Approach: | They propose to use a novel pretext task to learn a deep learning model for Anomaly Detection in text to train a model to discriminate between different transformations applied to visual data. |
| Outcome: | The proposed method outperforms state-of-the-art methods on 20Newsgroups and AG News datasets in the semi-supervised setting and in the unsupervised setting. |
Copied to clipboard
| Challenge: | Existing studies on syntactic injection in Variational AutoEncoders (VAEs) are limited to LSTM-based VAEs. |
| Approach: | They propose to use latent space separation techniques to inject syntactic information into Variational AutoEncoders (VAEs) using graph-based models. |
| Outcome: | The proposed end-to-end VAE architecture can improve the organisation of the latent space, alleviating the information loss occurring in standard VAE setups, and resulting in enhanced performances on language modelling and downstream generation tasks. |
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task of associating a word in context with its most appropriate sense from a predefined sense inventory. |
| Approach: | They propose to use a state-of-the-art neural model to integrate WSD into real-world applications. |
| Outcome: | The proposed system offers high-quality sense information in 40 languages through a state-of-the-art neural model for WSD. |
Copied to clipboard
| Challenge: | Existing systems treat this task as a pipeline of two separate subtasks, i.e., event extraction and temporal relation classification. |
| Approach: | They propose a joint event and temporal relation extraction model with shared representation learning and structured prediction. |
| Outcome: | The proposed method improves both event extraction and temporal relation extraction over state-of-the-art systems. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks require large labeled datasets to perform . compared to prior work, relative improvements in F1 of up to 16% are found . |
| Approach: | They propose to use self-training, knowledge distillation, and transfer learning to learn SLU models . they compare pipeline and pipeline approaches to find out how to use external data . |
| Outcome: | The proposed models improve performance beyond pre-trained models in resource-constrained settings . the best baseline model is a pipeline approach, while the best performance is achieved by an E2E model. |
Copied to clipboard
| Challenge: | Recent advances in text generation systems often produce incoherent and unfaithful outputs . a novel automated text generation system takes into account content selection, text planning, and surface realization. |
| Approach: | They propose an end-to-end trained two-step text generation model that considers sentence-level content planners and language styles. |
| Outcome: | The proposed model outperforms competing models in three domains with diverse topics and varying language styles. |
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) benefit from external knowledge stored in graph structures for various downstream tasks. |
| Approach: | They propose a graph-guided self-attention mechanism that integrates token-level structural information into PLMs without additional alignment or concatenation efforts. |
| Outcome: | The proposed model outperforms baseline models and achieves comparable results to state-of-the-art models on WebNLG datasets. |
Copied to clipboard
| Challenge: | Experimental results show that an E2E S2I model outperforms a conventional pipeline by 27% on the F1 score . multi-component pipelines require large annotated datasets, have higher latency, and have complex deployment. |
| Approach: | They propose an end-to-end (E2E) S2I model that uses pre-trained automatic speech recognition with slight modification and fine-tuning on small annotated datasets. |
| Outcome: | The proposed model outperforms a conventional pipeline on an annotated dataset by 27% on the F1 score. |
Copied to clipboard
| Challenge: | Content-based collaborative filtering (CF) predicts user-item interactions based on both items’ interaction history and item content information. |
| Approach: | They propose to combine item encodings with a multi-modality approach to improve training efficiency by 146x . |
| Outcome: | The proposed model improves training efficiency (up to 146x) on five datasets from two task domains of Knowledge Tracing and News Recommendation. |
Copied to clipboard
| Challenge: | Conversational Agents (Agents) often fail to understand how to start a conversation or what to ask next . a novel approach to recommending highly relevant follow-up question suggestions is proposed . |
| Approach: | They propose a method to recommend highly relevant follow-up question suggestions . they use offline QBs to fetch the most-relevant candidate questions . |
| Outcome: | The proposed system produces a ranked list of highly relevant follow-up question recommendations within 1 sec. |
Copied to clipboard
| Challenge: | Existing frameworks for streaming and non-streaming ASR models have been used to reduce development, training and deployment costs. |
| Approach: | They propose to use dynamic right-context through chunked attention masking to train zipformer-based ASR models. |
| Outcome: | The proposed framework reduces word error by relative 7.9% with a small degradation in user-perceived latency. |
Copied to clipboard
| Challenge: | Recent approaches rely on expensive annotations and are unavailable in low resource scenarios (e.g., rare languages or domains). |
| Approach: | They propose an end-to-end SRL model which leverages unlabeled data and propose to reduce the annotation effort involved via semi-supervised learning. |
| Outcome: | The proposed model outperforms the state-of-the-art in English and consistently improves performance in other languages, including Chinese, German, and Spanish. |
Copied to clipboard
| Challenge: | End-to-end ASR systems that eschew linguistic resources but are more dependent on large-data settings are suggested as a solution to EL documentation bottlenecks. |
| Approach: | They propose to build an end-to-end ASR system that is reproducible by the ASR community and propose a novice transcription correction task. |
| Outcome: | The proposed method would mitigate bottlenecks and shortages in transcribers . it is based on a Yoloxóchitl Mixtec corpus and is reproducible by the ASR community. |
Copied to clipboard
| Challenge: | Current textual question answering models fail to generalize to out-of-domain settings. |
| Approach: | They propose to decompose question and context into smaller units and align them to find the answer. |
| Outcome: | The proposed model is more robust than the standard BERT QA model on adversarial and out-of-domain datasets. |
Copied to clipboard
| Challenge: | Currently, the dominant end-to-end reinforcement learning paradigm for agents in Large Language Models (LLMs) employs multi-objective optimization that jointly trains both planning and answer summarization capabilities. |
| Approach: | They propose a framework that decouples the training process to enable a focused, single-objective optimization of the planning module. |
| Outcome: | The proposed framework achieves an 8%–12% improvement in planning performance compared to end-to-end baselines. |
Copied to clipboard
| Challenge: | Existing language models pre-trained on large-scale corpora often do not work well for user-generated data where the language styles differ significantly from the mainstream text. |
| Approach: | They propose a method for formality style transfer by injecting multiple rules into an end-to-end BERT-based encoder and decoder model. |
| Outcome: | The proposed method outperforms existing rule-based FST approaches on tweet sentiment analysis tasks. |
Copied to clipboard
| Challenge: | Argumentation is an essential tool in various domains, including law, public policy, and artificial intelligence. |
| Approach: | They propose to evaluate LLMs on various computational argumentation tasks . they organize existing tasks into six main categories and standardize the format of 14 datasets . |
| Outcome: | The proposed model performs well on argument mining and argument generation tasks. |
Copied to clipboard
| Challenge: | Existing studies on ERC focus on context modeling but ignore representation of contextual emotional tendency. |
| Approach: | They propose to use Emoformer to extract multi-modal emotion vectors from different modalities and fuse them with sentence vector to be an emotion capsule. |
| Outcome: | The proposed model outperforms the state-of-the-art models on two benchmark datasets. |
Copied to clipboard
| Challenge: | Existing generic prompt injection detectors lack domain specificity needed for nuanced resume attacks. |
| Approach: | They propose a scalable detection framework that uses a synthetically generated dataset to address data scarcity in this domain. |
| Outcome: | The proposed framework outperforms the best off-the-shelf detector by over 50% in relative F1 and approaches frontier LLM accuracy. |
Copied to clipboard
| Challenge: | a dialogue state tracker is a core component in most of today's spoken dialogue systems . slot-filling dialogues are composed of a predefined set of slots that need to be filled through the conversation . |
| Approach: | They propose an E2E architecture that extracts unknown slot values while still achieving state-of-the-art accuracy on the standard DSTC2 benchmark. |
| Outcome: | The proposed architecture achieves state-of-the-art accuracy on the DSTC2 benchmark while retaining predefined slot values. |
Copied to clipboard
| Challenge: | Existing methods for fine-grained opinion mining (OM) are based on span-based annotations, but they are not effective. |
| Approach: | They propose a unified span-based approach for the end-to-end OM setting using syntactic constituents and multi-task learning to integrate them into the proposed model. |
| Outcome: | The proposed approach achieves significant improvements over previous work on the MPQA 2.0 dataset and reduces the number of wrongly-predicted opinion expressions and roles. |
Copied to clipboard
| Challenge: | eschewing separate architecture and training for knowledge-intensive tasks is cumbersome . end-to-end training only based on supervision from the end task is awkward . |
| Approach: | They propose a single Transformer that performs retrieval as attention and end-to-end training solely based on supervision from the end QA task. |
| Outcome: | The proposed model outperforms state-of-the-art retrievers and readers on in-domain datasets. |
Copied to clipboard
| Challenge: | Non-autoregressive encoder-decoder models improve decoding speed, but generation quality suffers . editing at the level of output sequences limits model flexibility. |
| Approach: | They propose *iterative realignment* which iteratively realigns connectionist temporal alignments. |
| Outcome: | The proposed model matches an autoregressive baseline with a 14x speedup on the WSJ dataset; on LibriSpeech, it achieves an LM-free test-other WER of 9.0% (19% relative improvement on comparable work). |
Copied to clipboard
| Challenge: | Modular neural networks without additional training have been shown to surpass end-to-end neural networks on challenging vision–language tasks. |
| Approach: | They propose to use BLIP-2-based modular neural networks without additional training to build programs and a number of skill-specific, task-oriented modules to execute them. |
| Outcome: | The proposed methods outperform end-to-end neural networks on vision language tasks and retain performance when they use task-agnostic selections. |
Copied to clipboard
| Challenge: | Biomarkers are crucial indicators for early cancer detection and prognosis, but extracting biomarkers from clinical texts remains a challenge. |
| Approach: | They propose a pipeline that integrates a domain-adapted biomarker entity recognition model and a relation extraction model to link biomarkers to their respective mutations. |
| Outcome: | The proposed pipeline achieves an F1 score of 0.86 for oncology and 0.87 for neuroscience domains on 5,000 clinical texts. |
Copied to clipboard
| Challenge: | Sentence simplification involves a sentence being transformed into a simpler version of itself while preserving its core meaning. |
| Approach: | They propose a controllable-simplification model that tailors simplifications to four global operations . they propose to use a dataset to train highly accurate classification systems for these operations based on syntactic or discourse structure . |
| Outcome: | The proposed model outperforms both end-to-end and controllable approaches in sentence simplification tasks. |
Copied to clipboard
| Challenge: | Unstructured and ambiguous Standard Operating Procedures suffer from ambiguity, missing information, and inconsistency, all of which hinder automation. |
| Approach: | They propose a three-stage LLM framework that transforms unstructured SOPs into a structured plan and an executable code template. |
| Outcome: | The proposed framework shows an 88.4% accuracy and significant reduction in inconsistency on real-world SOPs and synthetic variants. |
Copied to clipboard
| Challenge: | a new schema for NLP knowledge about tasks, datasets and metrics is proposed. |
| Approach: | They propose a new schema that represents knowledge about tasks, datasets and metrics in the NLP domain. |
| Outcome: | The proposed framework can be automatically built into scientific leaderboards . the proposed system achieves reasonable results for all relation types on this small-scale graph . |
Copied to clipboard
| Challenge: | Text simplification (TS) is a technique that makes written information more accessible to all people, especially those with cognitive or language impairments. |
| Approach: | They propose to use English as a pivot language for simplification of Dutch medical and municipal texts. |
| Outcome: | The proposed approach improves on Dutch medical text, while the existing pipeline performs better on all metrics. |
Copied to clipboard
| Challenge: | Document-level contextual information has shown benefits to text-based machine translation, but whether and how it helps end-to-end speech translation is still under-studied. |
| Approach: | They propose a concatenation-based ST model with adaptive feature selection for computational efficiency. |
| Outcome: | The proposed model improves translation quality and robustness to (artificial) audio segmentation errors. |
Copied to clipboard
| Challenge: | Existing data pruning methods for active learning are expensive and time-consuming. |
| Approach: | They propose a plug-and-play data pruning strategy that leverages language models to prune the unlabeled pool. |
| Outcome: | The proposed pruning strategy outperforms existing pruning methods on translation, sentiment analysis, topic classification, and summarization tasks on diverse datasets. |
Copied to clipboard
| Challenge: | Past work on sentence embedding models faces issues determining the causal impact of implicit syntax representations. |
| Approach: | They construct a neural module net based on a transformer model and train it end-to-end to approximate the sentence’s embedding. |
| Outcome: | The proposed model captures whether syntax is a strong model of its compositional ability. |
Copied to clipboard
| Challenge: | Current methods for evaluating meeting effectiveness rely on post-hoc surveys that yield a single coarse-grained score for an entire meeting. |
| Approach: | They propose a new paradigm for evaluating meeting effectiveness centered on novel criteria and temporal fine-grained approach . they introduce a meta-evaluation dataset containing 2,459 human-annotated segments from 130 AMI Corpus meetings. |
| Outcome: | The proposed framework can be used to evaluate meeting effectiveness across different meeting types and from business scenarios to unstructured discussions. |
Copied to clipboard
| Challenge: | End-to-end speech translation (E2E) models that directly maps audio to a foreign text are not efficient. |
| Approach: | They propose adaptive feature selection (AFS) for encoder-decoder based E2E ST. |
| Outcome: | The proposed model outperforms the existing model on LibriSpeech En-Fr and MuST-C with a BLEU score of 18.56. |
Copied to clipboard
| Challenge: | End-to-end speech-to speech (S2S) dialogue systems face key challenges in incorporating external knowledge into their models. |
| Approach: | They propose a framework that directly retrieves relevant textual knowledge from speech queries. |
| Outcome: | The proposed framework improves the performance of end-to-end speech-tospeech dialogue systems while achieving higher retrieval efficiency. |
Copied to clipboard
| Challenge: | Existing methods to determine the knowledge an LLM already possesses and the knowledge that requires the help of a search engine are expensive and require excessive computational costs. |
| Approach: | They propose a slim proxy model that detects missing knowledge in LLMs with a proxy model and use it to perform retrieval for the missing knowledge. |
| Outcome: | The proposed approach detects missing knowledge in LLMs with a slim proxy model and takes its answers as heuristic answers. |
Copied to clipboard
| Challenge: | Existing methods for multi-speaker speech recognition require isolated source signals or senone alignments for effective learning. |
| Approach: | They propose a sequence-to-sequence framework to decode multiple label sequences from a single speech sequence by unifying source separation and speech recognition functions in an end-to end manner. |
| Outcome: | The proposed model improves on existing models by 83.1% relative to previous models with explicit separation and recognition modules. |
Copied to clipboard
| Challenge: | Knowledge base completion (KBC) is a human-interpretable dialect . rule-based KBC has a high quality but low accuracy . |
| Approach: | They propose to use logical neural networks to learn both kinds of rules in a common framework using gradient-based optimization. |
| Outcome: | The proposed method improves by 10% relative to SotA rule-based methods and by combining it with knowledge graph embeddings it achieves an additional 7.5% relative improvement. |
Copied to clipboard
| Challenge: | Traditional goal-oriented dialogue systems allow execution of validation rules as a post-processing step after slots have been filled which can lead to error accumulation. |
| Approach: | They propose a task of constraint violation detection based on knowledge-driven slot constraints . they propose methods to integrate external knowledge into the system and compare it to traditional rule-based pipeline approach . |
| Outcome: | The proposed task compares to the existing system and a rule-based pipeline. |
Copied to clipboard
| Challenge: | Recent trend for dependency parsing is adopting neural networks due to their significant success in a wide range of applications. |
| Approach: | They propose a sequence to sequence (seq2seque) dependency parser that predicts the relative position of head for each word. |
| Outcome: | The proposed parser achieves 94.11% UAS on PTB and 88.78% UAS . |
Copied to clipboard
| Challenge: | Modern artificial intelligence is characterized by large pretrained language models with strong language capabilities to be adapted to various downstream tasks. |
| Approach: | They propose to use the task of speech translation (ST) to pretrain speech models for end-to-end SLU on intra- and cross-lingual scenarios. |
| Outcome: | The proposed model achieves higher performance over baselines on monolingual and multilingual intent classification as well as spoken question answering using SLURP, MINDS-14, and NMSQA benchmarks. |
Copied to clipboard
| Challenge: | In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image with translations in target language. |
| Approach: | They propose an end-to-end IIMT model with four modules that translate images . they propose a two-stage training framework to assist the model in learning alignment across languages . |
| Outcome: | The proposed model outperforms cascaded models with only 70.9% of parameters and is highly accurate. |
Copied to clipboard
| Challenge: | Existing approaches to lexical substitution tend to overlook good substitute candidates that are not the synonyms of the target words in the lexicals and fail to take into account the substitution’s influence on the global context of the sentence. |
| Approach: | They propose an end-to-end BERT-based lexical substitution approach which proposes and validates substitute candidates without using annotated data or manually curated resources. |
| Outcome: | The proposed approach performs well in proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks. |
Copied to clipboard
| Challenge: | Existing research on text image machine translation (TIMT) lacks recognized source language information resulting in a decrease in translation performance. |
| Approach: | They propose a cross-modal cross-lingual interactive model which incorporates source language information by synchronizing source and target language results. |
| Outcome: | The proposed model outperforms end-to-end models and has faster decoding speed with smaller model size than cascade models. |
Copied to clipboard
| Challenge: | a new study examines how accent information is encoded and propagated in an end-to-end ASR system. |
| Approach: | They propose to use phone probes to analyze phonetic content of representations at each layer. |
| Outcome: | The proposed model is based on a large amount of US-accented English speech and is compared with other models using phone probes. |
Copied to clipboard
| Challenge: | Cross-lingual summarization (CLS) has attracted increasing interest due to the availability of large-scale web-mined datasets and the advancements of multilingual language models. |
| Approach: | They propose a dataset of cross-lingual code-switched summaries in Chinese and English . they show that leveraging existing CLS resources does not improve performance . |
| Outcome: | The proposed method does not improve on CroCoSum, indicating the limited generalizability of existing approaches. |
Copied to clipboard
| Challenge: | End-to-end spoken language understanding systems model sequence labeling as a sequence prediction task causing a divergence from its well-established token-level tagging formulation. |
| Approach: | They propose to model sequence labeling as a sequence prediction task . their systems explicitly separate the added complexity of recognizing spoken mentions from the NLU task of sequence labelling . |
| Outcome: | The proposed systems outperform both cascaded and direct models on a labeling task of named entity recognition across SLU benchmarks. |
Copied to clipboard
| Challenge: | In this work, we address the challenge of cross-lingual proper noun recognition in automatic speech recognition systems where proper nodes in an utterance may originate from a language different from the language in which the ASR system is trained. |
| Approach: | They propose a dictionary-based method to correct ASR predictions in a large language model . |
| Outcome: | The proposed method significantly reduces word error rates across cross-lingual proper noun recognition tasks involving three secondary languages. |
Copied to clipboard
| Challenge: | Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence. |
| Approach: | They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary. |
| Outcome: | Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training. |
Copied to clipboard
| Challenge: | Different Open Information Extraction (OIE) tasks require different types of information. |
| Approach: | They propose to adapt an OIE Graph to different OIE tasks with simple rules . they implement an end-to-end OIA generator and make it open-accessible . |
| Outcome: | The proposed system achieves new SOTA performance on three popular OIE tasks. |
Copied to clipboard
| Challenge: | Existing approaches to financial report generation are insufficient to handle dynamic uncertainties of real-world financial environments. |
| Approach: | They propose a cognitively grounded agentic framework for professional financial report generation that is driven by Dynamic Graph of Thoughts and a social collaboration mechanism to facilitate coordinated agent interaction. |
| Outcome: | The proposed framework is based on a dynamic reasoning model and social collaboration mechanism. |
Copied to clipboard
| Challenge: | Empirical results show that BeeSL’s speed and accuracy makes it a viable approach for large-scale real-world scenarios. |
| Approach: | They propose a joint end-to-end neural information extraction model that recasts the task as sequence labeling and jointly models intermediate tasks via multi-task learning. |
| Outcome: | Empirical results show that BeeSL outperforms the current best system on the Genia 2011 benchmark by 1.57% absolute F1 score reaching 60.22% F1 . |
Copied to clipboard
| Challenge: | Existing work on question-answer extraction fails to integrate incomplete utterances from dialog context for composite QA retrieval. |
| Approach: | They propose a task where questions and corresponding answers might be separated across different utterances. |
| Outcome: | The proposed methods perform well on 5 customer service datasets and set a benchmark for N-to-N DialogQAE with utterance and session level evaluation metrics. |
Copied to clipboard
| Challenge: | Existing approaches to summarize text using end-to-end content selectors have had mixed success in content selection, for example copying full sentences from the source document. |
| Approach: | They propose to use content selectors to over-determine phrases in a source document that should be part of the summary. |
| Outcome: | The proposed model over-determines phrases in a source document that should be part of the summary while generating fluent summaries. |
Copied to clipboard
| Challenge: | Degradation in performance across underrepresented accents is a severe deterrent to inclusive adoption of ASR. |
| Approach: | They propose an approach to adapt speech accents to unseen accents by using cross-attention with a trainable set of codebooks. |
| Outcome: | The proposed approach yields significant performance gains on the seen English accents and unseen accents on the Mozilla Common Voice dataset. |
Copied to clipboard
| Challenge: | Existing approaches to optimize RAG generators fail to align with RAG requirements thoroughly. |
| Approach: | They propose a method for optimizing the RAG generator from multiple preference perspectives to align with RAG requirements comprehensively. |
| Outcome: | The proposed method improves the performance of RAG generators by incorporating retrieved documents into the prompt. |
Copied to clipboard
| Challenge: | Text Image Machine Translation (TIMT) is a critical subfield of machine translation . it requires accurate optical character recognition, robust visual-text reasoning, and high-quality translation a challenge . |
| Approach: | They propose a multi-task optimization framework to specialize MLLMs into expert TIMT models. |
| Outcome: | The proposed model outperforms baselines on the latest in-domain MIT-10M benchmark. |
Copied to clipboard
| Challenge: | Depending on the size of transformer-based models, they can be restricted from deployment in resource-constrained environments. |
| Approach: | They propose to combine neural architecture search and network pruning techniques to generate and train weight-sharing super-networks that contain efficient transformer-based models. |
| Outcome: | The proposed model achieves high-performing, high-performance subnetworks on the general language understanding evaluation and the Stanford Question Answering Dataset. |
Copied to clipboard
| Challenge: | Existing approaches to building cross-lingual summarization systems on dialogue documents are limited. |
| Approach: | They propose a benchmark dataset for building cross-lingual summarization systems on dialogue documents. |
| Outcome: | The proposed model outperforms pipeline models on ClidSum and mDialBART. |
Copied to clipboard
| Challenge: | Modern Hawaiian orthography includes characters for long vowels and glottal stops . manual transliteration is laborious when performed manually . |
| Approach: | They propose two methods to help transliterate Hawaiian between older and newer texts automatically using finite state transducers and a recurrent neural network language model. |
| Outcome: | The proposed method solves the transliteration problem automatically using finite state transducers and a neural network language model. |
Copied to clipboard
| Challenge: | Existing methods to combine hybrid and end-to-end ASRs with confidence measures are limited and neither can achieve optimal performance. |
| Approach: | They propose to combine the hybrid Kaldi-based Automatic Speech Recognition system with the end-to-end wav2vec 2.0 XLS-R ASR using confidence measures. |
| Outcome: | The proposed method reduces the word error rate by 14% on the primary test set and 20% on other noisy and imbalanced data. |
Copied to clipboard
| Challenge: | A financial analyst's work involves manually reviewing lengthy filings and financial news articles in order to extract relevant pieces of information. |
| Approach: | They propose an end-to-end, fully unsupervised method for knowledge discovery from financial texts that integrates existing resources to construct a knowledge graph of companies and related entities. |
| Outcome: | The proposed method calculates the environmental rating for companies in the S&P 500 based on company filings with the SEC and provides an independent assessment of its outputs with an independent MSCI source. |
Copied to clipboard
| Challenge: | Large language models suffer from instability or lack of coherence when receiving diverse input variations. |
| Approach: | They analyze the behavior of large language models when dealing with multiple lexical variations of the same info-seeking questions. |
| Outcome: | The proposed model generates equivalent outputs when receiving diverse input variations. |
Copied to clipboard
| Challenge: | Multilingual speech translation (ST) and machine translation (MT) in the medical domain enhances patient care by enabling efficient communication across language barriers. |
| Approach: | They present a large-scale ST dataset for the medical domain spanning all translation directions in Vietnamese, English, German, French, and Simplified/Traditional Chinese, together with the models. |
| Outcome: | The multi-language speech translation (ST) and machine translation (MT) in the medical domain is the largest medical MT dataset and the largest many-to-many multilingual ST among all domains. |
Copied to clipboard
| Challenge: | Existing models for visual language reasoning require tens of thousands of training examples and their reasoning capabilities are limited. |
| Approach: | They propose a one-shot solution to visual language reasoning by combining plot-to-text translation and reasoning over the translated text into a modality conversion module. |
| Outcome: | The proposed method improves on human-written queries on plots and charts compared with a fine-tuned SOTA model on human queries. |
Copied to clipboard
| Challenge: | Recent advances in AM models overlook the integration of supplementary discourse structure information, resulting in suboptimal outcomes. |
| Approach: | They propose a framework which generates discourse structure-aware prefixes for each layer of the generation model. |
| Outcome: | The proposed framework achieves state-of-the-art performance on two AM benchmarks. |
Copied to clipboard
| Challenge: | Current research on in-image machine translation focuses on synthetic data with simple background, single font, fixed text position, and bilingual translation. |
| Approach: | They propose an end-to-end model to handle the challenge of practical conditions in PRIM . they annotate a real-world one-line text image with complex background, fonts, diverse text positions . |
| Outcome: | The proposed model improves translation quality and visual effect compared to other models. |
Copied to clipboard
| Challenge: | Argument mining (AM) is a challenging task as it requires recognizing complex argumentation structures involving multiple subtasks. |
| Approach: | They propose a generative framework where expected outputs of AM are framed as a simple target sequence. |
| Outcome: | The proposed framework achieves state-of-the-art on two AM benchmarks. |
Copied to clipboard
| Challenge: | Existing joint neural models for Information Extraction use local task-specific classifiers to predict labels for individual instances. |
| Approach: | They propose a joint neural framework that extracts the optimal IE result as a graph from an input sentence. |
| Outcome: | The proposed model achieves new state-of-the-art on all subtasks and does not use any language-specific feature. |
Copied to clipboard
| Challenge: | Recent large language models have shown indications of mathematical reasoning ability on competition-level problems. |
| Approach: | They propose a benchmark dataset to enable such analyses using large language models. |
| Outcome: | The proposed model performs better with concepts and hints than with the best model, but it is difficult to verify. |
Copied to clipboard
| Challenge: | Mainstream of automatic speech recognition (ASR) has shifted from pipeline methods to end-to-end (E2E) methods. |
| Approach: | They propose to integrate a pre-trained speech representation model and a large language model (LLM) for automatic speech recognition in an end-to-end manner. |
| Outcome: | The proposed model achieves comparable performance to modern E2E ASR models by utilizing powerful pre-training models with the proposed integrated approach. |
Copied to clipboard
| Challenge: | Recent advances in open-domain question answering have demonstrated impressive accuracy on general-purpose domains like Wikipedia. |
| Approach: | They propose a more realistic end-to-end domain shift evaluation setting covering five diverse domains to assess model adaption. |
| Outcome: | The proposed model improves by 24 points when adapted to unsupervised datasets. |
Copied to clipboard
| Challenge: | rynSpeech corpus is a dataset that can be used for both Text-to-Speecher (TTS) and Automatic Speech Recognition (ASR) speakers of many African languages have no access to voice-enabled applications in their native languages. |
| Approach: | They propose a dataset to collect Yorùbá speech data that can be used for both TTS and ASR tasks. |
| Outcome: | The proposed dataset can generate a good quality model with as little as 5 hours of speech . the results are consistent with previous studies on the Yorùbá language . |
Copied to clipboard
| Challenge: | Existing approaches to document-based Opendomain Question Answering (ODQA) use flat text chunks or page-level images to locate the correct document. |
| Approach: | They propose a hierarchical tree-based multimodal retrieval framework that elevates document hierarchy to a first-class retrieval signal. |
| Outcome: | The proposed framework outperforms page- and chunk-based baselines on ODQA benchmarks and improves retrieval recall by 12.9% and end-to-end QA performance by 6.8%. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are difficult to explain and understand due to long input contexts and autoregressive output generation. |
| Approach: | They propose a post-hoc explanation method which incorporates LLM-specific techniques. |
| Outcome: | The proposed method improves retrieval recall and prediction accuracy significantly on open-domain question answering benchmarks. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) problem is performed under extremely weak supervision . XWS setting is considered weaker than 1-shot since example entity is given in context-free way . |
| Approach: | They propose a method that uses extremely weak supervision to train named entity recognition models. |
| Outcome: | The proposed method outperforms the state-of-the-art few-shot methods with 1-shot supervision and ChatGPT annotations significantly. |
Copied to clipboard
| Challenge: | Existing OIE systems split a sentence into tokens and recognize token spans as tuple relations and arguments. |
| Approach: | They propose to split a sentence into tokens and recognize token spans as tuple relations and arguments. |
| Outcome: | The proposed model achieves state-of-the-art on multiple OIE datasets showing that SaC has better properties than sentence as token sequence. |
Copied to clipboard
| Challenge: | Existing benchmarks focus more on end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization. |
| Approach: | They propose a benchmark specifically designed to explore the problem-solving principles by decomposing 6.5K visual math problems into 10.9K step-level questions for evaluation. |
| Outcome: | The proposed benchmark covers 6.5K visual math problems and 10.9K step-level questions spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts. |
Copied to clipboard
| Challenge: | Document logical structuring is crucial for document intelligence due to the complexity of text segment dependencies in the document. |
| Approach: | They propose an end-to-end, generation-based method for document logical structuring that generates the action sequence via a global context-aware generative model and updates its global context and current logical structure based on the generated actions. |
| Outcome: | Experiments on ChCatExt and HierDoc datasets show that Seg2Act performs better than previous methods in both supervised and transfer learning settings. |
Copied to clipboard
| Challenge: | a neuro-symbolic approach allows zero-shot adaptation to unseen tasks and domains . a neural LM keeps track of events that occur during a conversation and a symbolic program implements dialog policy is executed to recommend actions. |
| Approach: | They propose an end-to-end, zero-shot task-oriented dialog system . it is designed to adapt to unseen tasks or domains without prior training . |
| Outcome: | The proposed system can be programmed to adapt to unseen tasks without training . it reduces data collection and training requirements for enabling new TOD 1 16189 tasks . |
Copied to clipboard
| Challenge: | Existing retrieval methods struggle with highly specialized situations that require extensive domain expertise. |
| Approach: | They propose a method that integrates additional information from an LLM-based generator to enhance query performance and train the retriever to better discriminate the relevant documents identified by the generator. |
| Outcome: | The proposed method outperforms existing domain adaptation methods by a large margin and leads to substantial improvements in retrieval quality across a wide range of application scenarios. |
Copied to clipboard
| Challenge: | Open-domain question answering (ODQA) systems typically adopt a retriever-reader architecture, where the retriever finds relevant documents, and the reader extracts or synthesizes answers. |
| Approach: | They propose a method that iteratively adjusts the importance weights of QE terms based on their relevance, refining term distinction and enhancing the separation of relevant terms. |
| Outcome: | The proposed method improves retrieval accuracy and overall performance on four ODQA datasets and five QE methods. |
Copied to clipboard
| Challenge: | Existing knowledge retrieval methods fail to account for interrelationship between knowledge pieces . however, current methods fail in a situation where multiple knowledge pieces are relevant . |
| Approach: | They propose an energy-based retriever that directly models the candidate retrieval results as a whole instead of modeling the knowledge pieces separately. |
| Outcome: | The proposed retriever outperforms the baseline energy-based retriever in knowledge retrieval tasks. |
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is employed to tackle these challenges . a Knowledge Boundary Model (KBM) is used to express the known/unknown of a given question . |
| Approach: | They propose a Knowledge Boundary Model to express the known/unknown of a given question . they find that not all questions need to trigger RAG to improve performance . |
| Outcome: | The proposed model reduces time and computational costs by retrieving parts of unknown knowledge . the proposed model can express the known/unknown of a given question and determine whether a RAG needs to be triggered . |
Copied to clipboard
| Challenge: | Existing benchmarks decompose the end-to-end professional report generation into individual components. |
| Approach: | They propose a benchmarking tool that evaluates 20 real-world professional report generation tasks grounded in multimodal document collections. |
| Outcome: | The proposed model outperforms closed-source models on executive summarization tasks but drops significantly on long-horizon synthesis tasks. |
Copied to clipboard
| Challenge: | Existing methods for fraud detection rely on transcribed text, lacking acoustic cues . a proposed framework for audio-based slow-thinking fraud detection eliminates transcription errors . |
| Approach: | They propose a framework for audio-based slow-thinking fraud detection that eliminates transcription errors and rewards slow-thought reasoning by capturing fine-grained audio details. |
| Outcome: | The proposed method improves accuracy, inference efficiency, and real-time processing capabilities. |
Copied to clipboard
| Challenge: | Using a partially reannotated subset of the Armed Conflict Location & Event Data, we analyze 39,786 conflict events across 20 languages and 171 countries. |
| Approach: | They propose a large-scale conflict event dataset with extensive coverage of region-specific entities. |
| Outcome: | The proposed method detects event arguments and entities through holistic document understanding and normalizes them across the multilingual dataset. |
Copied to clipboard
| Challenge: | idioms are defined as words with a figurative meaning not deducible from their individual components. |
| Approach: | They compare idiom translation as compared to conventional news translation in two languages . they compare MT and SLT systems with MT, Large Language Models and cascaded alternatives . |
| Outcome: | The proposed systems show better handling of idioms than standard news translation systems. |
Copied to clipboard
| Challenge: | NLCO evaluates large language models for combinatorial optimization (CO) . existing evaluations emphasize relatively simple reasoning competencies . |
| Approach: | They propose a combinatorial optimization benchmark that evaluates large language models on CO reasoning. |
| Outcome: | The proposed model can handle combinatorial optimization without writing code or calling external solvers. |