Papers by Jinan Xu
Copied to clipboard
| Challenge: | Knowledge distillation (KD) approaches focus on homogeneous architectures with identical tokenizers, constraining their applicability in cross-architecture scenarios. |
| Approach: | They propose a framework that uses contextual information to enhance sequence alignment precision and dynamically improves vocabulary mapping. |
| Outcome: | The proposed framework shows significant advantages over existing methods for model compression . it can be used across multiple model families and across multiple benchmarks . |
Copied to clipboard
| Challenge: | Existing IE tools lack multi-task support and automatic updates for KG and EKG construction. |
| Approach: | They propose a human-machine-cooperative IE toolkit for KG and EKG construction that unifies different IE subtasks and integrates LLMs as the assistant machine. |
| Outcome: | The proposed tool improves annotation quality, efficiency, and stability simultaneously. |
Copied to clipboard
| Challenge: | Multi-domain learning is a good solution for solving domain tasks but it requires retraining when adding a new domain. |
| Approach: | They propose to exploit unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
| Outcome: | The proposed framework exploits unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
Copied to clipboard
| Challenge: | Document-level event argument extraction aims to identify event arguments beyond sentence level, where a significant challenge is to model long-range dependencies. |
| Approach: | They propose a chain reasoning paradigm which captures long-range interdependence due to the chains’ compositional nature and generates decomposable first-order logic rules for reasoning. |
| Outcome: | The proposed method outperforms previous methods on two benchmarks and is robust enough to defend against adversarial attacks. |
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms the previous state-of-the-art model GloVe on STS tasks. |
| Approach: | They propose a simple and effective prompt template that is able to obtain the knowable information of input sentences from LLMs. |
| Outcome: | The proposed model outperforms the previous state-of-the-art model PromptBERT on STS tasks. |
Copied to clipboard
| Challenge: | Neural Chat Translation (NCT) models that use dialogue characteristics of chat are often incoherent and speakerirrelevant. |
| Approach: | They propose to introduce the modeling of dialogue characteristics into the NCT model by capturing the inherent dialogue characteristics. |
| Outcome: | The proposed model can translate conversational text between speakers of different languages. |
Copied to clipboard
| Challenge: | Existing knowledge distillation techniques for neural machine translation lack special treatment on the top-1 information, which is limiting the potential of KD. |
| Approach: | They propose a method to distill knowledge from top-1 predictions of teachers and a technique to infuse more additional knowledge by distilling on the data without ground-truth targets. |
| Outcome: | The proposed method outperforms the vanilla word-level KD and outperfies the existing methods on three different students with different capacity gaps. |
Copied to clipboard
| Challenge: | Existing approaches to improve neural machine translation use token-level adaptive training . however, standard models make predictions on condition of previous contexts . |
| Approach: | They propose a target-context-aware metric which can be supplemented by statistical metrics . they propose an adaptive training approach based on token- and sentence-level CBMI . |
| Outcome: | The proposed model outperforms the Transformer baseline and other similar approaches on English-German and Chinese-English tasks. |
Copied to clipboard
| Challenge: | Existing models use aspect-independent encoders for sentence representation generation. |
| Approach: | They propose an aspect-guided deep transition model which guides the sentence encoding from scratch with a specially-designed deep transition architecture. |
| Outcome: | The proposed model outperforms existing models on multiple datasets on aspect-category sentiment analysis and aspectterm sentiment analysis without additional features. |
Copied to clipboard
| Challenge: | Existing studies on cross-lingual summarization focus on pipeline methods or jointly training an end-to-end model through an auxiliary MT or MS objective. |
| Approach: | They propose a hierarchical model for the cross-lingual summarization task . the model is based on the conditional variational auto-encoder . |
| Outcome: | The proposed model generates better cross-lingual summaries than comparison models in the few-shot setting. |
Copied to clipboard
| Challenge: | Existing studies focus on improving MMS models by filtering summary-unrelated visual features with implicit learning or explicitly complex training objectives. |
| Approach: | They propose a multimodal multimodal summarization task that aims to generate summaries in any language with document inputs in any languages and the corresponding image sequence. |
| Outcome: | The proposed task can generate summaries in any language with document inputs in any languages and the corresponding image sequence. |
Copied to clipboard
| Challenge: | Existing studies decompose complex tasks into intermediate steps by prompting strategies, such as Chain-of-Thought and its variants. |
| Approach: | They propose to use code comments as natural logic pivot between natural language and code language to boost the code generation ability of code LLMs. |
| Outcome: | The proposed method significantly improves the code pass rate on humanEval and MBPP, while the robustness of the logical comment decoding strategy is higher than the Chain-of-thoughts prompting. |
Copied to clipboard
| Challenge: | Existing methods to ED see no differences between event types and use a single model to address them all. |
| Approach: | They propose a new concept termed trigger salience attribution which can explicitly quantify the underlying patterns of events. |
| Outcome: | The proposed model can distinguish between trigger-dependent and context-dependent types and achieve promising performance on two benchmarks. |
Copied to clipboard
| Challenge: | Paraphrase generation is of great importance for many downstream tasks in natural language processing. |
| Approach: | They propose a method to generate sentences as learning objectives from the learned data distribution and employ reinforcement learning to combine these new learning objectives for model training. |
| Outcome: | The proposed method gains significant diversity and improves generation quality over state-of-the-art datasets. |
Copied to clipboard
| Challenge: | Existing datasets are too small to train a model for capturing regularities underlying how event arguments are extracted. |
| Approach: | They propose to bridge implicit EAE with machine reading comprehension (MRC) by building a unified training framework and explicit data augmentation regimes via MRC. |
| Outcome: | The proposed method obtains state-of-the-art performance on two benchmarks and demonstrates superior results in a data-low scenario. |
Copied to clipboard
| Challenge: | Existing multi-agent systems have shown strong potential for machine translation (MT) but their performance in multidomain translation remains unsatisfactory due to cross-domain word ambiguity . |
| Approach: | They propose a multi-agent collaborative disambiguation framework for MDT that leverages the collaborative capabilities of LLMs for disambiguations. |
| Outcome: | The proposed framework improves translation performance across multiple domains and improves disambiguation accuracy. |
Copied to clipboard
| Challenge: | Multimodal machine translation and textual chat translation have received considerable attention . however, little research has been devoted to multimodal machine translator in conversations . |
| Approach: | They propose a task to generate more accurate translations with the help of dialogue history and visual context. |
| Outcome: | The proposed task can generate more accurate translations with the help of dialogue history and visual context. |
Copied to clipboard
| Challenge: | Traditional machine translation evaluation relies on reference written by humans . reference-free evaluation gets rid of labor-intensive annotations, which can pivot easily to new domains . |
| Approach: | They propose a reference-free evaluation approach that characterizes evaluation as two aspects: fluency and faithfulness. |
| Outcome: | The proposed approach outperforms SOTA reference-fee metrics on machine translation datasets. |
Copied to clipboard
| Challenge: | Existing LT strategies cannot indicate the desired target language on zero-shot translation, i.e., the off-target issue. |
| Approach: | They propose a language converter strategy that embeds the target language into the top encoder layers to mitigate confusion in the encoder and ensures stable language indication for the decoder. |
| Outcome: | The proposed language converter strategy significantly mitigates off-target issue on multiUN, TED, and OPUS-100 datasets. |
Copied to clipboard
| Challenge: | Neural chat translation aims to translate bilingual conversational text due to its inherent characteristics such as role preference, dialogue coherence, and translation consistency. |
| Approach: | They propose to model the translation quality of conversational text by learning distributions of bilingual conversational characteristics. |
| Outcome: | The proposed approach outperforms baseline models and is widely available. |
Copied to clipboard
| Challenge: | Neural machine translation models are trained to maximize the likelihood of the next token given previous golden tokens as inputs, but at the inference stage, golden token is unavailable. |
| Approach: | They propose a scheduled sampling method that randomly replaces groundtruth tokens with predicted ones during training, ignoring real-time model competence. |
| Outcome: | The proposed method outperforms the Transformer and vanilla scheduled sampling on large-scale translations. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have made significant progress in knowledge-intensive applications, but they may face a multi-stage continuous learning scenario. |
| Approach: | They propose a multi-stage continuous learning paradigm that includes a preference-based learning bias to identify potential knowledge conflicts and a self-distillation-based data augmentation strategy to expand and enrich the training corpus. |
| Outcome: | The proposed learning paradigm achieves a significant improvement in accuracy after 7 stages of fine-tuning compared to previous methods while preserving general knowledge. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) is one of the fundamental tasks in natural language processing. |
| Approach: | They propose four practical guidelines to guide knowledge transfer and task finetuning . they propose a framework to exploit data from three aspects in a unified training manner . |
| Outcome: | The proposed framework improves on six benchmarks and shows that it is state-of-the-art in five languages. |
Copied to clipboard
| Challenge: | Current large language models (LLMs) show a significant performance gap in alignment between English and other languages. |
| Approach: | They propose a consistency-based method to construct high-quality multilingual preference data for improving multilingual alignment. |
| Outcome: | The proposed method is based on three LLMs and three common tasks and shows that it performs better than current methods. |
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Copied to clipboard
| Challenge: | Sentence matching is a key issue in natural language inference and paraphrase identification. |
| Approach: | They propose a semantics-oriented attention and deep fusion network (OSOA-DFN) that is oriented to the original semantic representation of another sentence and propagates attention information at each matching layer. |
| Outcome: | The proposed model can model sentence matching more precisely on three sentence matching benchmark datasets. |
Copied to clipboard
| Challenge: | Existing knowledge editing methods retain outdated responses for reasoning questions . naively retraining LLMs can be computationally intensive and can lead to catastrophic forgetting . |
| Approach: | They propose a simple yet effective decoding strategy to enhance edited models on reasoning questions. |
| Outcome: | The proposed method outDates ISsue aware deCOding (DISCO) to improve models on reasoning questions. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks require large datasets with accurate annotations that are labor-intensive and time-consuming. |
| Approach: | They propose a method to leverage domain gaps to model cross-domain few-shot named entity recognition (NER) NER is a natural language processing task to detect entity mentions and classify them into predefined labels . |
| Outcome: | The proposed method achieves state-of-the-art or competitive results on standard datasets. |
Copied to clipboard
| Challenge: | Existing approaches to generate adversarial examples for NMT use the meaning-preserving restriction. |
| Approach: | They propose a new definition for adversarial examples based on the Doubly Round-Trip Translation (DRTT) they introduce masked language models to construct bilingual adversarials based upon DRTT . |
| Outcome: | The proposed approach significantly improves the robustness of the NMT model on clean and noisy test sets. |
Copied to clipboard
| Challenge: | Existing conversation models produce meaningless and generic responses, which significantly reduce the user experience. |
| Approach: | They propose to fuse knowledge to improve informativeness and adopt latent variables to enhance the diversity of responses. |
| Outcome: | The proposed model can generate syntactically diverse and knowledge-accurate responses while maintaining the knowledge accuracy. |
Copied to clipboard
| Challenge: | Existing approaches to multilingual retrieval-augmented generation (MRAG) use a single-turn retrieval and subsequent optimization to acquire and integrate beneficial external knowledge from multilingual collections. |
| Approach: | They propose a multilingual search-augmented reinforcement learning framework that integrates a language-coupled Group Relative Policy Optimization into the policy and reward models. |
| Outcome: | The proposed framework achieves competitive performance and is appropriate for various practical scenarios such as constrained training data and retrieval over collections encompassing a large number of languages. |
Copied to clipboard
| Challenge: | Existing methods to update factual knowledge overlook connections of same knowledge between different languages, resulting in knowledge conflicts and limited edit performance. |
| Approach: | They propose a method to edit multilingual knowledge simultaneously that avoids knowledge conflicts and improves edit performance. |
| Outcome: | The proposed method avoids knowledge conflicts and improves edit performance on bi-ZsRE and MzsRE benchmarks. |
Copied to clipboard
| Challenge: | Existing syntactically-controlled paraphrase generation models perform well with human-annotated or well-chosen syntaktic templates. |
| Approach: | They propose a quality-based Syntactic Template Retriever to retrieve templates based on the quality of the to-be-generated paraphrases. |
| Outcome: | The proposed algorithm can generate high-quality paraphrases without sacrificing quality. |
Copied to clipboard
| Challenge: | Existing back-translation methods focus on in-domain lexical knowledge, which may lead to poor translation of unseen in- domain words. |
| Approach: | They propose an iterative constrained back-translation method to incorporate in-domain lexical knowledge into synthetic parallel data from BT. |
| Outcome: | The proposed method improves the BLEU score by up to 3.08 on four domains. |
Copied to clipboard
| Challenge: | Currently, Large Language Models (LLMs) have achieved remarkable results in machine translation, but their performance in multidomain translation (MDT) is less satisfactory. |
| Approach: | They propose to evaluate the disambiguation ability of Large Language Models in multi-domain translation . they construct a translation test set with multi- domain ambiguous word annotation . |
| Outcome: | The proposed framework evaluates LLMs on disambiguation in multi-domain translation (DMDTEval) the results show that LLM's perform poorly in multidomain translation, highlighting ambiguity in translation. |
Copied to clipboard
| Challenge: | Current Large Reasoning Models exhibit two critical limitations when processing non-English languages: (1) They struggle to maintain input-output language consistency; (2) They generally perform poorly with wrong reasoning paths and lower answer accuracy compared to English. |
| Approach: | They propose a language-consistency reward and a cross-lingual thinking alignment reward to improve the model's interpretability and accuracy. |
| Outcome: | The proposed model achieves nearly 100% language consistency and superior performance on two multilingual benchmarks (MMATH and PolyMath). |
Copied to clipboard
| Challenge: | Existing studies on syntactically controlled paraphrase generation rely on large-scale parallel data. |
| Approach: | They propose a syntactically-informed unsupervised paraphrasing model based on conditional variational auto-encoder which can generate texts in a specified syntastic structure. |
| Outcome: | The proposed model can generate diverse paraphrases with specified syntactic structure using non-parallel data. |
Copied to clipboard
| Challenge: | Large Language Models struggle with multilingual reasoning tasks due to resource constraints . a training-free method improves performance on multilingual thinking tasks . |
| Approach: | They propose a training-free method that transforms language-specific semantic information into language-agnostic structured representations. |
| Outcome: | The proposed method outperforms strong baselines on multilingual reasoning tasks. |
Copied to clipboard
| Challenge: | Existing models for slot filling and intent detection fail to fully utilize cooccurrence relations between slots and intents, which restricts their potential performance. |
| Approach: | They propose a novel Collaborative Memory Network (CM-Net) that captures slot-specific and intent-specific features in a collaborative manner. |
| Outcome: | The proposed network outperforms existing models on two benchmarks and a self-collected corpus. |
Copied to clipboard
| Challenge: | Current dialogue systems face diverse user requests and rapid change domains, making quickly adapt to scenarios with previous unseen slot types becomes a major challenge. |
| Approach: | They propose an incremental novel slot detection task which separates the dialogue system to deal with novel types as two major phrases: 1) model discovers unknown slots; 2) training model to possess the capability to handle new classes. |
| Outcome: | The proposed approach overcomes catastrophic forgetting during the process of INSD and is highly effective. |
Copied to clipboard
| Challenge: | Existing methods for fine-tuning domain adaptation have overfitting problem in low-resource domains . lack of parallel data makes it difficult for model to learn domain-specific knowledge . |
| Approach: | They propose a Reinforcement Learning Domain Adaptation method for Neural Machine Translation that uses in-domain source monolingual data to make up for the lack of parallel data. |
| Outcome: | The proposed method can alleviate overfitting and reinforce the model to learn domain-specific knowledge. |
Copied to clipboard
| Challenge: | Existing studies focus on fine-tuning multilingual dense retrieval models, but data scarcity for low-resource languages makes it difficult to align representations in a shared vector space. |
| Approach: | They propose to obtain high-quality hard negative samples and effective mini-batch data to boost data utilization for multilingual dense retrieval by obtaining high-quality negative samples. |
| Outcome: | The proposed method outperforms existing baselines on a multilingual retrieval benchmark, MIRACL, with 16 languages. |
Copied to clipboard
| Challenge: | Named entity recognition datasets are notorious for their noisy nature due to annotation errors, inconsistencies, and subjective interpretations. |
| Approach: | They propose a method that considers NER as a constituency tree parsing problem and uses a tree-structured Conditional Random Fields with uncertainty evaluation for integration. |
| Outcome: | The proposed model exhibits superb performance even in extreme scenarios with 90% annotation noise. |
Copied to clipboard
| Challenge: | a recent study focuses on generating impartial and interpretable judicial judgments based on established criminal fact. |
| Approach: | They propose a law reasoning schema enriched with hierarchical factum probandum, evidence, and implicit experience that enables public scrutiny and preventing bias. |
| Outcome: | The proposed schema enables public scrutiny and prevents bias in the "Intelligent Court" it employs a suite of legal analysis tools to address the challenge task. |
Copied to clipboard
| Challenge: | Existing studies on multimodal abstractive summarization focus on how to use extracted visual features to produce a concise summary given the multimodal data. |
| Approach: | They propose to improve the visual quality of the multimodal abstractive summarization model by capturing summary-oriented visual features. |
| Outcome: | The proposed approach achieves state-of-the-art under 44 languages and is highly effective on high-resource English datasets. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have remarkable ability in high-resource languages, but their performance in multilingual scenarios is still limited. |
| Approach: | They propose a layer-wise expert allocation algorithm to determine the appropriate number of new experts for each layer. |
| Outcome: | The proposed method outperforms the previous state-of-the-art baseline with 60% fewer experts in the single-expansion setting and 33.3% fewer in the lifelong-expanding setting. |
Copied to clipboard
| Challenge: | Existing studies have focused on learning domain knowledge from multiple domains, but task-specific parameters hinder mutual transfer of knowledge between new domains. |
| Approach: | They propose an iterative Continual learning framework for multi-domain neural machine translation that leverages previously acquired domain knowledge. |
| Outcome: | The proposed model outperforms baseline models on UM-Corpus and OPUS datasets. |
Copied to clipboard
| Challenge: | Neural machine translation models are trained to maximize the likelihood of next token given previous golden tokens as inputs, but at the inference stage, golden token is unavailable. |
| Approach: | They propose to use scheduled sampling to replace ground-truth tokens with predicted tokens to bridge the gap between training and inference. |
| Outcome: | The proposed methods outperform the Transformer baseline and vanilla scheduled sampling on three large-scale WMT tasks. |
Copied to clipboard
| Challenge: | Existing safeguards for Large Language Models are vulnerable to "jailbreaking" harmful queries. |
| Approach: | They propose a learning method that optimizes a continuous soft safety prompt automatically to facilitate multilingual safeguarding of LLMs. |
| Outcome: | The proposed method outperforms previous approaches in multilingual jailbreak defense while exhibiting strong cross-lingual generalization. |
Copied to clipboard
| Challenge: | Existing methods for LLM alignment optimize tokens using a sparse, response-level reward or preference annotation. |
| Approach: | They propose an RLHF-equivalent distillation method for token-level reward optimization that incorporates the reward learned by DPO into the RLHG objective and builds a token-based teacher distribution. |
| Outcome: | The proposed method bridges the accuracy gap between the reward from the DPO model and the pure reward model by building a contrastive DPO reward with a normal and a reverse DPO. |
Copied to clipboard
| Challenge: | Safety impact of Multimodal Large Language Models (MLLMs) on human behavior is evaluated in this study. |
| Approach: | They propose a safety-warning-based evaluation framework that encourages models to provide clear and informative safety warnings, rather than generic refusals. |
| Outcome: | The proposed safety-warning-based evaluation framework encourages models to provide clear and informative safety warnings, rather than generic refusals. |
Copied to clipboard
| Challenge: | Existing parameter-efficient fine-tuning methods such as Low-Rank Adaptation ignore the differential parameter budget requirements across weight matrices, which may lead to suboptimal fine-uning outcomes. |
| Approach: | They propose a parameter-efficient low-rank Adaptation method that decomposes high-rank LoRA layers into structured single-rank components and allows dynamic pruning of parameter budget . |
| Outcome: | The proposed method outperforms LoRA and LoRA with the same parameter budget and performance. |
Copied to clipboard
| Challenge: | Existing approaches to token-level adaptive training only use static word frequency information without considering the source language. |
| Approach: | They propose a bilingual mutual information based adaptive objective that assigns weights to target tokens with higher BMI . they propose to use this approach to improve token-level adaptive training . |
| Outcome: | The proposed method improves token-level adaptive training on two languages. |
Copied to clipboard
| Challenge: | Sentence-level translation, document-level and terminology constrained translations are important in machine translation. |
| Approach: | They propose a multi-task machine translation model that integrates translation memory sentences . they propose 'in-context learning' paradigm that allows translation-specific context learning . |
| Outcome: | The proposed model improves translation memory, document-level translation, and document-constrained translation tasks. |
Copied to clipboard
| Challenge: | Recent studies have shown that many well-developed Visual Question Answering systems suffer from bias problem. |
| Approach: | They propose a way to mitigate bias problem by subtracting bias score from standard VQA base score. |
| Outcome: | The proposed method improves on the VQA v2.0 and VQA-CP V2,0 datasets. |
Copied to clipboard
| Challenge: | Existing methods for robustness against homophone errors are limited to homophones . substitution errors are the most common errors in NMT models . |
| Approach: | They propose an adversarial example generation method based on confusion sets that contain words easily confusable with a target word by ASR to conduct adversarially training for NMT models. |
| Outcome: | The proposed method improves on the clean test set and can be used in real-world scenarios. |
Copied to clipboard
| Challenge: | Existing approaches to Aspect-based sentiment analysis do not exploit the interactive relations among subtasks and do not utilize document-level labeled domain/sentiment knowledge, which restricts their performance. |
| Approach: | They propose an iterative multi-knowledge transfer network for end-to-end ABSA that leverages the inter-task interaction between subtasks. |
| Outcome: | The proposed approach improves on three benchmark datasets. |
Copied to clipboard
| Challenge: | Recent work focuses on the modeling of discourse relation, resulting in discrete codes learning shallow semantics. |
| Approach: | They propose a topic-aware latent code-guided text generation model that encourages discrete codes to model information about topics. |
| Outcome: | The proposed model generates more topic-relevant and coherent texts. |
Copied to clipboard
| Challenge: | Temporal knowledge graph reasoning is a crucial task for answering time-dependent questions within a knowledge graph (KG). |
| Approach: | They propose a temporal KG reasoning benchmark with over 200k entities and 960k questions that facilitate complex, multi-relational and multi-hop reasoning. |
| Outcome: | The proposed model is able to conduct pattern-aware and time-sensitive reasoning across temporal KGs and is scalable to a wide range of data conditions. |
Copied to clipboard
| Challenge: | Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios. |
| Approach: | They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge. |
| Outcome: | The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models. |
Copied to clipboard
| Challenge: | Existing Large Reasoning Models have demonstrated broad application potential, yet their safety and reliability remain critical concerns. |
| Approach: | They conduct a safety evaluation of 13 MLRMs across 5 benchmarks and examine their safety performance. |
| Outcome: | The proposed model improves safety on jailbreak and safety-awareness benchmarks. |
Copied to clipboard
| Challenge: | Existing methods to improve chat translation focus on modeling bilingual dialogue characteristics . however, there is insufficient data and simple joint training methods . |
| Approach: | They propose a scheduled multi-task learning framework to incorporate large-scale chat translation data into training by adding a second pre-training stage between the original pre- training and fine-tuning stages. |
| Outcome: | The proposed framework improves translation tasks on four language directions. |
Copied to clipboard
| Challenge: | Existing knowledge graph completion models only evaluated candidate triples from content information. |
| Approach: | They propose a multi-view classification model where multiple views are performed based on both content and context information for candidate triple evaluation. |
| Outcome: | The proposed model improves on two representative datasets and improves performance. |
Copied to clipboard
| Challenge: | Existing systems for sequence labeling are limited by shallow connections between consecutive hidden states and insufficient modeling of global information. |
| Approach: | They propose a global context enhanced deep transition architecture for sequence labeling . they deepen the state transition path at each position in a sentence and assign tokens with global representations . |
| Outcome: | The proposed architecture outperforms the best reported results on two standard sequence labeling tasks. |
Copied to clipboard
| Challenge: | Recent multilingual pre-trained models have been demonstrated effective in many cross-lingual tasks. |
| Approach: | They propose a framework that leverages code-switched data with multi-view learning to fine-tune XLM-R. |
| Outcome: | The proposed model achieves state-of-the-art on zero-shot cross-lingual sentiment classification and dialogue state tracking tasks. |
Copied to clipboard
| Challenge: | Existing V&L pre-training methods rely on strictly-aligned multilingual image-text pairs generated from English-centric datasets. |
| Approach: | They propose a regularized cross-lingual visual contrastive learning objective that constrains representation proximity of weakly-aligned multilingual image-text pairs. |
| Outcome: | The proposed model outperforms competing models with weak zero-shot capability on 5 multi-modal tasks across 6 languages. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have strong generalization abilities due to their huge model capacities. |
| Approach: | They propose a dual-space knowledge distillation framework that unifies the output spaces of the two models for KD. |
| Outcome: | The proposed framework outperforms existing white-box KD frameworks on task-agnostic instruction-following benchmarks and can automatically align representations of two models with different vocabularies. |
Copied to clipboard
| Challenge: | Syntax-controlled paraphrase generation aims to produce paraphrase conform to given syntactic patterns. |
| Approach: | They propose a model that captures parent-child and sibling relations and a syntax encoder to capture alignment relations. |
| Outcome: | The proposed model achieves state-of-the-art in terms of semantic and syntactic quality on two popular benchmark datasets. |