Papers by Chris Tar
Multilingual Universal Sentence Encoder for Semantic Retrieval (2020.acl-demos)
Copied to clipboard
Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-hsuan Sung, Brian Strope, Ray Kurzweil
| Challenge: | Using a multi-task trained dual-encoder, our models embed text from 16 languages into a shared semantic space. |
| Approach: | They propose retrieval focused multilingual sentence embedding models on TensorFlow Hub. |
| Outcome: | The models achieve state-of-the-art on monolingual and cross-lingual retrieval (SR) and retrieval question answering (ReQA) competitive performance is obtained on related tasks of translation pair bitext retrieval and retrieving question answering. |
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation (2024.findings-acl)
Copied to clipboard
Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, Thang Luong
| Challenge: | Modern large language models often "hallucinate" plausible but factually incorrect information, which reduces their trustworthiness especially in settings where accurate and up-to-date information is critical. |
| Approach: | They develop a human evaluation procedure to measure correctness and hallucination and use it to benchmark both closed and open-source LLMs. |
| Outcome: | The proposed method outperforms both competing search engine-augmented prompting methods and commercial systems on search-augmented QA. |
Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction (N19-1)
Copied to clipboard
| Challenge: | Modern NLP systems require high-quality annotations, but experts are expensive and lay annotators may not have the knowledge to provide high- quality annotations. |
| Approach: | They propose to directly model instance difficulty to improve model performance and to route instances to appropriate annotators. |
| Outcome: | The proposed model improves performance on a biomedical information extraction task using expert and lay annotations. |
Universal Sentence Encoder for English (D18-2)
Copied to clipboard
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, Ray Kurzweil
| Challenge: | TensorFlow Hub sentence embedding models have good task transfer performance . model variants allow for trade-offs between accuracy and compute resources . |
| Approach: | They propose easy-to-use TensorFlow Hub sentence embedding models with good task transfer performance. |
| Outcome: | The proposed models outperform models without transfer learning and those that use only word-level transfer on a number of NLP tasks. |
Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | evaluating large language models' output is difficult due to the high cost of human evaluation. |
| Approach: | They propose a family of foundational large autorater models that train on over 100 quality assessment tasks. |
| Outcome: | The proposed model outperforms models on 8 of 12 autorater benchmarks on 53 quality assessment tasks. |
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification (D19-1)
Copied to clipboard
| Challenge: | Existing work on adversarial data generation focuses on English . Existing multilingual datasets show effectiveness of deep, multilingual pre-training . |
| Approach: | They propose a dataset of 23,659 human translated PAWS evaluation pairs in six languages . they show the effectiveness of deep, multilingual pre-training while leaving considerable headroom . |
| Outcome: | The proposed model shows that multilingual training and evaluation regimes are more accurate than previous models. |