Papers by Mirac Suzgun
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a new method for textual style transfer is proposed for text with a limited set of style choices . textual styles are a complex task that requires specialized models to perform . |
| Approach: | They propose a method for arbitrary textual style transfer using pre-trained language models . they use a mathematical formulation of the TST task, decomposing it into three components . |
| Outcome: | The proposed method performs on par with state-of-the-art large-scale models while using less compute and memory. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
Do Language Models Know When They’re Hallucinating References? (2024.findings-eacl)
Copied to clipboard
| Challenge: | State-of-the-art language models (LMs) are notoriously susceptible to generating hallucinated information. |
| Approach: | They propose to use hallucinated book and article references as "model organism" of hallucinism research . authors propose queries to the language model to identify hallucinous references . |
| Outcome: | The authors show that language models can identify hallucinated references without external resources . they show that LMs often produce inconsistent author lists for hallucinos, but also accurately recall the authors of real references . |
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them (2023.findings-acl)
Copied to clipboard
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, Jason Wei
| Challenge: | Language models have already made good progress on this benchmark, with the best model outperforming average reported human-rater results on 65% of the BIG-Bench tasks. |
| Approach: | They propose to use chain-of-thought prompting to challenge language models on 23 challenging BIG-Bench tasks which they call BIG-Bench Hard. |
| Outcome: | The proposed language models outperform the average human-rater on 65% of the BIG-Bench tasks. |
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2026.eacl-long)
Copied to clipboard
| Challenge: | Unlike fine-tuning or static retrieval methods, DC adapts LMs’ problem-solving skills on the fly, without modifying their underlying parameters. |
| Approach: | They propose a lightweight framework that endows a black-box LM with a persistent, evolving memory. |
| Outcome: | The proposed framework enables models to store and reuse accumulated strategies, code snippets, and general problem-solving insights at inference time. |
string2string: A Modern Python Library for String-to-String Algorithms (2024.acl-demos)
Copied to clipboard
| Challenge: | Notable algorithms include the Smith-Waterman algorithm for pairwise local alignment, the Hirschberg algorithm for global alignment, and the Wagner-Fischer algorithm for edit distance. |
| Approach: | **string2string** is an open-source library that offers efficient algorithms for string-to-string problems. |
| Outcome: | **string2string** is an open-source library that offers efficient algorithms for string-to-string problems. |
Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text decoding methods struggle to produce high-quality text . Greedy and beam search suffer from text degeneration and linguistic diversity issues . |
| Approach: | They propose a family of decoding methods based on minimum bayes risk minimization to address diversity-quality trade-offs in open-ended natural-language generation. |
| Outcome: | The proposed methods improve diversity-quality trade-offs on open-ended natural-language generation tasks. |