Papers by Michal Lukasik
Text Segmentation by Cross Segment Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Document and discourse segmentation are two fundamental NLP tasks pertaining to breaking up text into constituents. |
| Approach: | They propose three transformer-based NLP models that break up text into constituents and compare them to previous approaches. |
| Outcome: | The proposed architectures reduce errors by a large margin on three datasets and improve performance on real-world datasets. |
Content Explorer: Recommending Novel Entities for a Document Writer (D18-1)
Copied to clipboard
| Challenge: | Existing tools for exploratory search can be useful for document writers but they are not always effective for identifying topics for further research. |
| Approach: | They propose a supervised learning problem for recommending topics to a writer . they propose entropy loss function modification to improve the results . |
| Outcome: | The proposed model improves on a large dataset and can be used to identify topics for further research. |
Regression Aware Inference with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown strong results on a range of applications, including regression and scoring tasks. |
| Approach: | They propose alternative inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses. |
| Outcome: | The proposed approach significantly improves over baselines across datasets and models. |
Large Language Models with Controllable Working Memory (2023.findings-acl)
Copied to clipboard
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, Sanjiv Kumar
| Challenge: | Large language models (LLMs) have led to a series of breakthroughs in natural language processing due to the massive amounts of world knowledge they memorize during pretraining. |
| Approach: | They propose a method to inject counterfactual and irrelevant contexts into standard supervised datasets to strengthen both controllability and robustness. |
| Outcome: | The proposed method improves controllability and robustness across model architectures and sizes. |
TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning LLMs use cross-entropy (CE) loss . existing methods neglect the numeric nature of score prediction . |
| Approach: | They propose a method that fine-tunes large language models (LLMs) for automated text evaluation, assigning a score to the input based on scoring rubrics. |
| Outcome: | The proposed model outperforms existing methods in four LLM-as-a-judge datasets and two LLMs. |
Semantic Label Smoothing for Sequence to Sequence Problems (2020.emnlp-main)
Copied to clipboard
Michal Lukasik, Himanshu Jain, Aditya Menon, Seungyeon Kim, Srinadh Bhojanapalli, Felix Yu, Sanjiv Kumar
| Challenge: | Existing methods for seq2seq regularization use label smoothing, but it is difficult to extend it to other datasets. |
| Approach: | They propose a method that smooths over well formed relevant sequences that are semantically similar to the target sequence. |
| Outcome: | The proposed method shows a consistent and significant improvement over the state-of-the-art methods on different datasets. |