Papers by Guoqing Zheng
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for multilingual RS are limited by capacity and data distribution skew . we propose Conditional Generative Matching models (CGM) to overcome these challenges . |
| Approach: | They propose Conditional Generative Matching models to address multilingual RS challenges . they use expressive message conditional priors, mixture densities and latent alignment . results exceed ROUGE scores by 10% on average, and 16% for low resource languages . |
| Outcome: | The proposed model exceeds baselines in relevance by 10% on average and 16% for low resource languages. |
WALNUT: A Benchmark on Semi-weakly Supervised Learning for Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on weak supervision for NLU focus on a specific task or simulate weak supervision signals from ground-truth labels. |
| Approach: | They propose a benchmark to advocate and facilitate research on weak supervision for NLU . they use document-level and token-level prediction tasks as examples . |
| Outcome: | The proposed benchmark advocates and facilitates research on weak supervision for NLU tasks. |
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
Boosting Natural Language Generation from Instructions with Meta-Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that language models trained with multi-task instructional learning (MTIL) can solve diverse NLP tasks in zero-shot settings with improved performance compared to prompt tuning. |
| Approach: | They propose to adapt meta-learning to MTIL in three directions: 1) Model Agnostic Meta Learning (MAML), 2) Hyper-Network adaptation to generate task specific parameters conditioned on instructions. |
| Outcome: | The proposed approaches improve over strong baselines in zero-shot settings and are most impactful when the test tasks are strictly zero- shot and are "hard" |
Axiomatic Preference Modeling for Longform Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have helped bridge the "alignment gap" between the responses of raw pretrained language models and responses that resonate more closely with human preferences. |
| Approach: | They propose to use a axiomatic framework to generate a rich variety of preference signals to uphold these signals. |
| Outcome: | The proposed model outperforms GPT-4 and ChatGPT in preference scoring. |
Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) enhanced with retrieval augmentation have shown great performance in many applications, but their computational overhead and additional retrieval step limit their effectiveness in real-time tasks. |
| Approach: | They propose a system that combines a cloud-based LLM with a smaller client-side model through retrieval augmented memory to provide real-time text prediction. |
| Outcome: | The proposed system can generate better responses from the cloud-based model while maintaining low latency. |
A Dataset and Baselines for Multilingual Reply Suggestion (2021.acl-long)
Copied to clipboard
| Challenge: | Reply suggestion models help users process emails and chats faster. |
| Approach: | They present a multilingual reply suggestion dataset with ten languages . they build a generation model and a retrieval model as baselines for MRS . |
| Outcome: | The proposed model complements existing benchmarks for cross-lingual generalization . the model has different strengths in the English monolingual setting and requires different strategies to generalize across languages. |
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning (2021.naacl-main)
Copied to clipboard
Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
| Challenge: | Recent work shows that multilingual representations are disjointed across languages, bringing additional challenges for transfer onto extremely low-resource languages. |
| Approach: | They propose a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer. |
| Outcome: | The proposed framework learns to transform representations from auxiliary languages to a target language and brings their representation spaces closer for effective transfer. |