Papers by Zhuoyuan Mao
Textual Enhanced Contrastive Learning for Solving Math Word Problems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that current models rely on shallow heuristics to predict solutions . a textual Enhanced Contrastive Learning framework enforces the models to distinguish semantically similar examples while holding different mathematical logic. |
| Approach: | They propose a textual Enhanced Contrastive Learning framework which enforces models to distinguish semantically similar examples while holding different mathematical logic. |
| Outcome: | The proposed framework improves on benchmark and challenge datasets in English and Chinese. |
Seeking Diverse Reasoning Logic: Controlled Equation Expression Generation for Solving Math Word Problems (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing methods to solve Math Word Problems rely on human annotation . empirical results suggest that our method universally improves the performance on single-unknown and multiple-un unknown benchmarks. |
| Approach: | They propose a controlled equation generation solver by leveraging a set of control codes to guide the model to consider certain reasoning logic and decode the corresponding equations expressions transformed from the human reference. |
| Outcome: | The proposed method improves performance on single-unknown and multiple-un unknown benchmarks with 13.2% accuracy on the challenging multiple-unequal datasets. |
Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods ignore the intrinsic noise of distant supervision during the pre-training stage. |
| Approach: | They propose a weighted contrastive learning method that explicitly reduces noise . they leverage supervised data to estimate reliability and reduce noise compared to non-weighted baselines . |
| Outcome: | The proposed method reduces the noise of distant supervision and estimates reliability of pre-training instances. |
JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) requires large parallel corpora for training robust and high quality models. |
| Approach: | They propose a Japanese-specific sequence to sequence pre-training alternative to MASS for NMT . they use Japanese as the source or target language to train their models . |
| Outcome: | The proposed approach can give competitive results over MASS and BRSS, and significantly surpass the individual methods. |
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing RE models are incapable of handling implicit expressions and long-tail relation types due to language complexity and data sparsity. |
| Approach: | They propose a method to enhance relation extraction using k nearest neighbors (kNN-RE) kNN is a nearest-neighbor search tool that allows the model to consult training relations at test time . |
| Outcome: | The proposed model outperforms the best model to date on ACE05, SciERC, and Wiki80 datasets and outperformed the best on i2b2 and Wik80 dataset. |
Pre-training via Leveraging Assisting Languages for Neural Machine Translation (2020.acl-srw)
Copied to clipboard
| Challenge: | Sequence-to-sequence (S2S) pre-training with large monolingual data is not always available for the languages of interest (LOI). |
| Approach: | They propose to use monolingual corpora of other languages to complement the scarce monolingual LOI by script mapping (Chinese to Japanese) . Using only Chinese and French monolinguals, they improve Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios. |
| Outcome: | The proposed approach improves Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios. |
Lightweight Cross-Lingual Sentence Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for learning fixed-dimensional cross-lingual sentence representations are impractical due to memory limitations. |
| Approach: | They propose a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations. |
| Outcome: | The proposed model improves performance on training tasks and improves memory efficiency. |
LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale language-agnostic sentence embedding models suffer from inference speed and computation overhead. |
| Approach: | They propose to train a lightweight sentence embedding model to achieve this by incorporating knowledge from a teacher model. |
| Outcome: | The proposed model can build low-dimensional sentences for 109 languages with a thin-deep encoder. |
GPT-RE: In-context Learning for Relation Extraction using Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to in-context learning (ICL) are lacking in relation extraction (RE) . emergence of large language models (LLMs) such as GPT-3 represents a significant advancement in natural language processing. |
| Approach: | They propose to incorporate task-aware representations into demonstration retrieval and enrich the demonstrations with gold label-induced reasoning logic. |
| Outcome: | The proposed model achieves SOTA and competitive performances on the Semeval and SciERC datasets. |
When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation? (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to improve pre-training for many-to-many neural machine translation use manual cleaning of bilingual dictionaries, which are unavailable for most language pairs. |
| Approach: | They propose a word-level contrastive objective to leverage word alignments for many-to-many neural machine translation (NMT) Empirical results show that this leads to 0.8 BLEU gains for several language pairs. |
| Outcome: | Empirical results show that the proposed objective leads to 0.8 BLEU gains for several language pairs. |
DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in music large language models have significantly improved music understanding tasks, but the potential of incorporating additional modalities such as images, videos and textual music features remains unexplored. |
| Approach: | They propose a multimodal music understanding LLM fine-tuned via multi-way instruction tuning with multi-ways aligned music, text, image, and video data. |
| Outcome: | The proposed model achieves state-of-the-art performance across six music understanding tasks and zero-shot scenarios. |
Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that layer normalization (LayerNorm) overfits training data and therefore has low generalizability for ZST. |
| Approach: | They propose to use the Transformer architecture to set the default layer normalization setting for zero-shot translation (ZST) they also propose to set LayerNorm after residual connections to outperform PreNorm by 12.3 BLEU points. |
| Outcome: | The proposed model outperforms the current model by 12.3 BLEU points on 54 directions on OPUS, IWSLT, and Europarl datasets. |
BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing subword segmenters are frequency-based without semantics information or neural-based but trained on parallel corpora. |
| Approach: | They propose an unsupervised neural subword segmenter for neural machine translation that utilizes contextualized semantic embeddings of words from characterBERT and maximizes the generation probability of subword segments. |
| Outcome: | The proposed method improves translation performance on ALT, IWSLT15 Vi->En, WMT16 Ro->En and WMT15 Fi->En datasets. |