Papers by Roi Reichart
Copied to clipboard
| Challenge: | Using frameworks such as Universal Dependencies (UD) to transfer knowledge between languages can be challenging because of variation in syntactic structures. |
| Approach: | They propose a typologically driven method which reduces anisomorphism in UD treebanks by considering both morphological and structural properties. |
| Outcome: | The proposed method is effective for machine translation and cross-lingual sentence similarity. |
Copied to clipboard
| Challenge: | a series of bilingual lexicon induction experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods fail for a large number of language pairs. |
| Approach: | They propose to use fully unsupervised approaches to project monolingual embeddings into a shared cross-lingual space without any cross-linguistic signal. |
| Outcome: | The proposed methods fail for a large number of language pairs, but never surpass weakly supervised methods. |
Copied to clipboard
| Challenge: | a key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language. |
| Approach: | They propose to use a full-vocabulary setup to test the performance of language modeling (LM) on 50 typologically diverse languages. |
| Outcome: | The proposed language modeling task is based on a full vocabulary setup focused on word-level prediction on 50 typologically diverse languages. |
Copied to clipboard
| Challenge: | Few-shot crosslingual transfer outperforms zero-shot with pretrained encoders like multilingual BERT. |
| Approach: | They conduct an experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on lexical features and a full model finetuning approach outperformed several state- of-the art approaches. |
Copied to clipboard
| Challenge: | Existing methods for comparing DNNs on unseen data are not suitable for this task. |
| Approach: | They propose to adapt a test for the Almost Stochastic Dominance relation between two distributions to the problem by comparing their performance on unseen data. |
| Outcome: | The proposed method meets all criteria while previously proposed methods fail to do so. |
Copied to clipboard
| Challenge: | PERL is a representation learning model that uses labeled data from the source domain and unlabeled data not necessarily drawn from the target domain. |
| Approach: | They propose a model that extends contextualized word embedding models with pivot-based fine-tuning to address this bottleneck. |
| Outcome: | The proposed model outperforms strong baselines across 22 sentiment classification domain adaptation setups and improves in-domain model performance. |
Copied to clipboard
| Challenge: | Existing work on persuasion games assumes communication with stylized messages that do not consist of real-world natural language. |
| Approach: | They propose to use a repeated sender-decision maker game to persuade a receiver to accept a deal by sending one of several possible natural language reviews to the expert. |
| Outcome: | The proposed expert is superior to baselines and adaptable to different decision makers and potential proposed deals. |
Copied to clipboard
| Challenge: | Existing representations of hallucinations limit the types of errors that can be expressed, so we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. |
| Approach: | They propose a benchmark for localizing hallucinations using LLMs with a human annotation of over 1,000 examples and a protocol to verify its quality in a humans evaluation. |
| Outcome: | The proposed representation captures the full range of possible errors, and the best model achieves an F1 score of 0.67. |
Copied to clipboard
| Challenge: | causality has not had the same importance in natural language processing, says aaron e. smith . he says research on causality in NLP remains scattered across domains without unified definitions . |
| Approach: | They propose to consolidate research on causality in NLP across academic areas . they explore potential uses of causal inference to improve robustness, fairness, interpretability . |
| Outcome: | The proposed method is a unified overview of causal inference for the NLP community. |
Copied to clipboard
| Challenge: | Existing evaluation strategies for analyzing economic data with narratives are limited due to the complexity of the interplay of numerous factors and the difficulty in isolating causal relationships. |
| Approach: | They propose to use two Twitter datasets to capture economy-related narratives and use them to construct models using large language models. |
| Outcome: | The proposed models are able to predict macroeconomic fluctuations using the extracted or extracted narratives in two Twitter datasets. |
Copied to clipboard
| Challenge: | Recent advances in large language models have led to the development of systems 2 models that can solve complex tasks and predict human behavior. |
| Approach: | They compare the performance of four state-of-the-art LLMs to human participants and compare their results to stumpers, a unique single-step intuition problem that humans can easily verify. |
| Outcome: | The proposed models excel in solving stumpers and surpass human performance on stumpers, while humans exhibit superior skills in verifying solutions to the same problems. |
Copied to clipboard
| Challenge: | Domain Adaptation (DA) algorithms suffer degradation when applied to out-of-distribution examples. |
| Approach: | They propose an example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation . the algorithm is trained to generate a unique prompt that maps the test example to a semantic space . |
| Outcome: | The proposed model outperforms baselines in 14 multi-source adaptation scenarios. |
Copied to clipboard
| Challenge: | Existing methods for pre-training can be sub-optimal in some cases . for example, aspect extraction tasks require domain and category invariant representations . |
| Approach: | They propose a domain-invariant learning scheme for BERT to fine-tune pre-trained language models on a source domain and then apply it to a different target domain. |
| Outcome: | The proposed scheme improves performance over state-of-the-art models while using fraction of the unlabeled data. |
Copied to clipboard
| Challenge: | Existing models show impressive results on a common CQA benchmark, but are they robust to domain, setting and domain? |
| Approach: | They propose a prompt-based history modeling approach that adds textual prompts directly to the text of a passage. |
| Outcome: | The proposed model is simple, easy to plug into practically any model and highly effective. |
Copied to clipboard
| Challenge: | Existing methods for generating concept-based explanations of preferences are poorly understood. |
| Approach: | They propose a method for generating local and global concept-based explanations of preferences across multiple domains using an LLM. |
| Outcome: | The proposed method outperforms baselines while also being explainable. |
Copied to clipboard
| Challenge: | Multi-task learning requires annotating the same text with multiple annotation schemes, which can be costly and laborious. |
| Approach: | They propose to use multi-task active learning paradigm to optimize annotation processes by iteratively selecting unlabeled examples whose annotation is most valuable for the NLP model. |
| Outcome: | The proposed model minimizes annotation efforts for multi-task NLP models by iterating on the most valuable examples. |
Copied to clipboard
| Challenge: | Semantic specialization is not available in many languages because of their incomplete or non-existent structure. |
| Approach: | They propose a method that transfers specialization from a resource-rich source language to virtually any target language. |
| Outcome: | The proposed method performs lexical simplification, dialog state tracking, and textual similarity tasks in 5 languages. |
Copied to clipboard
| Challenge: | Statistical significance testing is a standard statistical tool designed to ensure that experimental results are not coincidental. |
| Approach: | They propose a protocol for statistical significance test selection in NLP setups . they propose he proposes a survey of the most relevant tests to help guide the protocol . |
| Outcome: | The proposed protocol includes a survey of the most relevant tests. |
Copied to clipboard
| Challenge: | Concept-based explanations quantify how high-level concepts influence model behavior . existing benchmarks rely on costly human-written counterfactuals that serves as imperfect proxy . |
| Approach: | They propose a framework for constructing datasets containing structural counterfactual pairs . they use a structured Causal Model to generate a concept-based explanation . |
| Outcome: | The proposed framework compares concept-based explanations to causal effects estimated from counterfactuals. |
Copied to clipboard
| Challenge: | Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages. |
| Approach: | They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. |
| Outcome: | The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English. |
Copied to clipboard
| Challenge: | a deep neural network can be used to improve bilingual text embeddings . a novel approach is proposed to optimize text embed-ings on shared visual information . |
| Approach: | They propose a deep neural network that leverages images to improve bilingual text embeddings. |
| Outcome: | The proposed model outperforms previous methods on word similarity and cross-lingual image description retrieval. |
Copied to clipboard
| Challenge: | Existing domain adaptation (DA) algorithms are not able to handle out-of-distribution examples due to the costly and labor-intensive data labeling process. |
| Approach: | They propose a controllable generation approach to deal with domain adaptation challenge by generating a domain-counterfactual textual example from an input text. |
| Outcome: | The proposed approach outperforms baselines and improves accuracy of state-of-the-art unsupervised DA algorithm. |
Copied to clipboard
| Challenge: | Recent advances in NLP systems have led to widespread adoption by a broad spectrum of users across various domains, impacting decision-making, the job market, society, and scientific research. |
| Approach: | They examine existing interpretability paradigms, their properties, and their relevance to different stakeholders by analyzing trends from the past decade across multiple research fields. |
| Outcome: | The proposed models are complex and opaque and are often overlooked by technical surveys. |
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
Copied to clipboard
| Challenge: | Recent years have seen an increasing number of applications that have a natural language interface, such as chatbots or "intelligent personal assistants" |
| Approach: | They propose a new training algorithm that trains a semantic parser on examples from a set of source domains and augment it with features and a logical form candidate filtering logic to support zero-shot adaptation. |
| Outcome: | The proposed framework performs better than a non-adapted parser with features and logical form candidate filtering logic. |
Copied to clipboard
| Challenge: | Several approaches for dependency parsing in the small data regime have been proposed. |
| Approach: | They propose to use stochastic gradient Langevin dynamics to generate samples from the approximated posterior to overcome the computational and statistical costs of the approximate inference step. |
| Outcome: | The proposed model outperforms the biaffine model on 6 languages with less than 5k training instances and improves across five languages. |
Copied to clipboard
| Challenge: | Existing approaches to learning word associations rely on distributional hypothesis and learn a single word vector space based on word co-occurrences in large text corpora. |
| Approach: | They propose to induce a joint function-specific word vector space where vectors of plausible SVO compositions lie close together. |
| Outcome: | The proposed model outperforms existing models on selectional preference and event similarity tasks while reducing the number of parameters by up to 95%. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are widely used in NLP research but there is no standard or rigorous procedure to determine whether they can replace humans. |
| Approach: | They propose a statistical procedure that requires only a modest subset of annotated examples to justify using LLM annotations. |
| Outcome: | The proposed procedure compares LLMs with open-source LLM annotators and judges on ten language and vision-language tasks. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) offer new opportunities to enhance the annotation process, particularly for detecting label errors in existing datasets. |
| Approach: | They propose to use an ensemble of large language models to flag mislabeled examples by using an LLM-as-a-judge approach to detect label errors in existing datasets. |
| Outcome: | The proposed method improves label accuracy and consistency in large language models. |
Copied to clipboard
| Challenge: | Pre-training Large Language Models (LLMs) on textual corpora embeds substantial factual knowledge in their parameters, which is essential for excelling in various downstream applications. |
| Approach: | They propose to use supervised fine-tuning to align large language models to new factual information that is not acquired through pre-training. |
| Outcome: | The proposed model is trained to generate facts that are not grounded in pre-existing knowledge, but hallucinates when examples with new knowledge are learned. |
Copied to clipboard
| Challenge: | Existing methods for compressing language representation models are not interpretable and do not consider the differences in the predictive power of various model components or the generalizability of the compressed models. |
| Approach: | They propose a model compression scheme that estimates the average treatment effect of a single layer on the model's predictions. |
| Outcome: | The proposed model compression scheme outperforms strong baselines on dozens of domain pairs across three text classification and sequence tagging tasks. |
Copied to clipboard
| Challenge: | Existing approaches to domain adaptation (DA) require labeled data that can be found in only a handful of domains. |
| Approach: | They propose a task-refinement learning approach to solve pivot detection problems . they propose to train PBLM models with gradually increasing information exposed about each pivot . |
| Outcome: | The proposed approach achieves state-of-the-art accuracy in six domain adaptation setups for sentiment classification. |
Copied to clipboard
| Challenge: | a framework for cross-domain and cross-language transfer has hardly been explored . cross-linguistic and cross language transfer methods are used for multilingual applications . |
| Approach: | They propose a framework that builds on pivot-based learning, structure-aware Deep Neural Networks and bilingual word embeddings to train a model on labeled data from one language pair. |
| Outcome: | The proposed model outperforms existing models even when trained in the lazy setup . the proposed model can be applied to nine English-German and nine English - french domain pairs without retraining . |
Copied to clipboard
| Challenge: | Existing approaches to zero-shot learning are limited in two ways: Query-dependence and richness of language description. |
| Approach: | They propose a task-agnostic approach to image classification using only text descriptions . they train a hypernetwork that receives class descriptions and outputs a multi-class model . |
| Outcome: | The proposed approach generates non-linear classifiers, handles rich textual descriptions, and may be adapted to produce lightweight models efficient enough for on-device applications. |
Copied to clipboard
| Challenge: | Queer youth face increased mental health risks, such as depression, anxiety, and suicidal ideation. |
| Approach: | They propose a scale that is inspired by psychological standards and expert input to evaluate LLM's interactions with queer-related content. |
| Outcome: | The proposed scale outperforms human responses to queer-related content and outperformed LLMs in the qualitative and quantitative analysis. |
Copied to clipboard
| Challenge: | In order to achieve unprecedented performance, many out-of-distribution generalization approaches use unlabeled data from the target distribution. |
| Approach: | They propose a framework that leverages labeled data from multiple source domains to generalize to unknown target domains at training. |
| Outcome: | The proposed framework outperforms existing models in two tasks, and it is compared to few-shot GPT-3. |
Copied to clipboard
| Challenge: | Existing work on domain adaptation does not exploit the structure of the input text . PBLM can naturally feed structure aware text classifiers such as LSTM and CNN . |
| Approach: | They propose a model that integrates pivot-based and NN modeling in a structure aware manner. |
| Outcome: | The proposed model can naturally feed structure aware text classifiers such as LSTM and CNN. |
Copied to clipboard
| Challenge: | a number of natural questions have been asked about the inductive biases of neural networks on core NLP tasks. |
| Approach: | They construct an informative prior for held-out languages on a task of character-level, open-vocabulary language modelling. |
| Outcome: | The proposed model outperforms baseline models with an uninformative prior in both zero-shot and few-shot settings, showing that it is imbued with universal linguistic knowledge. |
Copied to clipboard
| Challenge: | Existing systems for visual question answering are overfitted to training data and are sensitive to small perturbations. |
| Approach: | They propose a robustness measure to augment visual question answering datasets to measure generalization capabilities. |
| Outcome: | The proposed model can quantify failure cases which reveal that current systems are still brittle. |
Copied to clipboard
| Challenge: | Modern Natural Language Generation models come with massive computational and storage requirements. |
| Approach: | They propose a method that applies word-level knowledge distillation to multiple PTs generated by both teacher and student. |
| Outcome: | The proposed techniques can be used to compress natural language models while preserving their performance. |
Copied to clipboard
| Challenge: | Currently, there are only 24 languages in the world that have not been annotated . transferring knowledge across domains is a common solution . |
| Approach: | They propose a Bayesian generative model for the space of neural parameters that factorizes into latent variables for each language and each task. |
| Outcome: | The proposed model can perform better than state-of-the-art methods with a typologically diverse sample of 33 languages from 4 continents and 11 families. |
Copied to clipboard
| Challenge: | Sentence fusion is the task of joining related sentences into coherent text. |
| Approach: | They propose a method where ground-truth solutions are automatically expanded into multiple references via curated equivalence classes of connective phrases. |
| Outcome: | The proposed approach improves on state-of-the-art models by expanding ground-truth solutions into multiple references. |
Copied to clipboard
| Challenge: | Existing studies have suggested that bilingual lexicon induction is influenced by the (dis)similarity of the languages at hand. |
| Approach: | They propose to measure the isomorphism of monolingual embedding spaces based on their spectra and introduce isometric measures to measure their similarity. |
| Outcome: | The proposed measures outperform standard isomorphism measures while being more tractable and easier to interpret. |
Copied to clipboard
| Challenge: | Existing query parsers that account for the unique grammar of web queries rely on resources not available outside of big web corporations. |
| Approach: | They propose a biLSTM query parser that explicitly accounts for the unique grammar of queries. |
| Outcome: | The proposed query parser outperforms existing state-of-the-art parsers on 2500 annotated queries. |
Copied to clipboard
| Challenge: | a new study examines how dementia is perceived by non-experts . human perception of dementia is inconsistent and relies on a narrow set of cues compared to LLMs based on broader clinical patterns . |
| Approach: | They propose a method that uses LLMs to extract high-level, expert-guided features . human perception of dementia is inconsistent and relies on a narrow set of cues, they say . |
| Outcome: | The proposed method analyzes picture descriptions to assess whether they were produced by non-experts or by nonexperts. |
Copied to clipboard
| Challenge: | Existing research on domain robustness (DR) relies on the Source Drop (SD) but lacks a complementary metric, a new study finds . |
| Approach: | They propose to use the Target Drop (TD) to measure domain DR . they use a DR benchmark consisting of 7 diverse tasks to measure both metrics . |
| Outcome: | The proposed model types excel in-domain, but few-shot LLMs often surpass them cross-domain showing better robustness. |