Papers by Yasaman Razeghi
EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning (2024.naacl-short)
Copied to clipboard
| Challenge: | Language models are adopting inference-time prompting techniques such as zero-shot and few-shot prompting. |
| Approach: | They propose a prompting technique that prompts the model to rephrase its queries before answering them. |
| Outcome: | The proposed prompt improves zero-shot-CoT performance of code-davinci-002 by 5% . the proposed prompt is tailored for four scenarios in both zero- and few-shot settings . |
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have demonstrated ability to perform numerical reasoning by extrapolating from a few examples in few-shot settings. |
| Approach: | They investigate how well pretrained language models reason with terms less frequent in pretraining data. |
| Outcome: | The models are more accurate on instances whose terms are more prevalent, in some cases above 70% more accurate than the bottom 10%. |
Nudging: Inference-time Alignment of LLMs via Guided Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) require alignment to effectively and safely follow user instructions. |
| Approach: | They propose a simple, training-free algorithm that aligns any base model at inference time using a small aligned model. |
| Outcome: | The proposed algorithm outperforms large aligned models on open-instruction tasks without training. |
Plot Twist: Multimodal Models Don’t Comprehend Simple Chart Details (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in multimodal models show remarkable performance in real-world benchmarks for chart and figure understanding like ChartQA that involve interpreting trends, comparing data points, and extracting insights from visuals. |
| Approach: | They propose to ask models basic questions about axes ranges and values to examine their visual understanding abilities in the context of charts. |
| Outcome: | The models perform well on complex tasks, but lack basic capabilities on basic questions. |
Selective Perception: Learning Concise State Descriptions for Language Model Actors (2024.naacl-short)
Copied to clipboard
| Challenge: | Recent large language models support longer contexts, but requiring them to process redundant or irrelevant data increases inference time and cost. |
| Approach: | They propose a method that leverages a small finetuned LM to sample the minimal set of input features that maximizes the performance of a downstream LM. |
| Outcome: | The proposed method reduces the length of LM actor input by 87% and 99% while improving task success rates by 158% and 54% on NetHack and robot planning. |
TurtleBench: A Visual Programming Benchmark in Turtle Geometry (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) are capable of reasoning about geometric patterns, but they are still a challenge to evaluate. |
| Approach: | They propose a benchmark to evaluate LMMs’ ability to interpret geometric patterns and generate precise code outputs. |
| Outcome: | The benchmark compared LMMs with turtle geometry and GPT-4V, and found that few-shot prompting improved their performance (2%). |
Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Snoopy allows researchers to analyze the impact of the overlap between pretraining corpus and test data on model performance statistics. |
| Approach: | They propose to align terms in test instances with their frequency in the Pile to explore correlations between model accuracy and model size and number. |
| Outcome: | Snoopy allows researchers to analyze term frequency statistics in large language models on NLP benchmarks. |
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have been successful when finetuned to downstream tasks . however, it is difficult to determine whether the knowledge that finetuning LMs contain is learned during the pretraining or the finetailing process. |
| Approach: | They propose a method to create prompts for a diverse set of tasks using a gradient-guided search. |
| Outcome: | The proposed method performs sentiment analysis and natural language inference without additional parameters and finetuning. |