Papers by Sercan Arik
SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-SQL aims to automate the process of generating SQL queries on a database from natural language text. |
| Approach: | They propose a method to improve few-shot prompting capabilities of Text-to-SQL for Large Language Models (LLMs) they propose 'SQlPrompt' which aims to diversify the SQL proposals during consistency selection with different prompt designs and foundation models. |
| Outcome: | The proposed method outperforms previous approaches for in-context learning with zero labeled data by a large margin, closing the gap with finetuning state-of-the-art with thousands of labeles. |
TextGenSHAP: Scalable Post-Hoc Explanations in Text Generation with Long Documents (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are difficult to explain and understand due to long input contexts and autoregressive output generation. |
| Approach: | They propose a post-hoc explanation method which incorporates LLM-specific techniques. |
| Outcome: | The proposed method improves retrieval recall and prediction accuracy significantly on open-domain question answering benchmarks. |
Effective Large Language Model Adaptation for Improved Grounding and Citation Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models generate "hallucinated" answers that are not factual . despite their widespread adoption, they can generate plausiblesounding but nonfactual information. |
| Approach: | They propose a framework that tunes large language models to self-ground claims and provide citations to retrieved documents. |
| Outcome: | The proposed framework generates superior grounded responses with more accurate citations compared to prompting-based approaches and post-hoc citing-based methods. |
Search-Adaptor: Embedding Customization for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to embed text in large language models are limited to zero-shot setups and can be integrated with any LLM. |
| Approach: | They propose a method for customizing LLMs for information retrieval by modifying the embeddings generated by pre-trained LLM models and can be integrated with any LLM. |
| Outcome: | The proposed method improves performance on English, multilingual, and multimodal retrieval datasets by 5% over 14 BEIR datasets. |
Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive capabilities in many tasks, including natural language understanding and generation. |
| Approach: | They propose a framework for adaptation with self-evaluation to improve selective prediction performance of large language models. |
| Outcome: | The proposed framework outperforms state-of-the-art selective prediction methods on QA datasets and improves the AUACC from 91.23% to 92.63% and AUROC from 74.61% to 80.25%. |
Universal Self-Adaptive Prompting (2023.emnlp-main)
Copied to clipboard
| Challenge: | a hallmark of modern large language models is their impressive general zero-shot and few-shot abilities . however, zero- shot performances are weaker due to the lack of guidance and the difficulty of applying existing automatic prompt design methods in general tasks. |
| Approach: | They propose an automatic prompt design approach specifically tailored for zero-shot learning that categorizes a possible NLP task into one of three possible task types and then uses a selector to select the most suitable queries and zero- shot model-generated responses as pseudo-demonstrations. |
| Outcome: | The proposed approach is able to generalize ICL to zero-shot learning tasks while also allowing for a more efficient and efficient prompt design. |
Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Embeddings from Large Language Models (LLMs) have emerged as critical components in information retrieval applications. |
| Approach: | They propose a tuning framework for the customization of LLM embeddings. |
| Outcome: | The proposed framework reduces embedding dimensions while maintaining comparable performance levels. |
Better Zero-Shot Reasoning with Self-Adaptive Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans. |
| Approach: | They propose a new method that uses a set of examples from the LLM zero-shot outputs to improve performance. |
| Outcome: | The proposed method improves performance up to 15% compared to baselines and matches or exceeds few-shot baselines at a range of reasoning tasks. |