Papers by Petr Marek
Concept-aware Data Construction Improves In-context Learning of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work curating in-context learners assumes that ICL emerges from vast over-parametrization or the scale of multitask training. |
| Approach: | They propose a framework for constructing training scenarios that make it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations. |
| Outcome: | The proposed framework makes it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations and fares comparably to previous in-context learners trained in large-scale multitask learning requiring magnitudes of more training data. |
Towards Processing of the Oral History Interviews and Related Printed Documents (L18-1)
Copied to clipboard
Zbyněk Zajíc, Lucie Skorkovská, Petr Neduchal, Pavel Ircing, Josef V. Psutka, Marek Hrúz, Aleš Pražák, Daniel Soutner, Jan Švec, Lukáš Bureš, Luděk Müller
| Challenge: | a project aims to create an integrated archive of the recordings, scanned documents and photographs from totalitarian regimes in Czechoslovakia . the archive will be accessible online and provide multifaceted search capabilities . |
| Approach: | They propose to use automatic speech recognition and optical character recognition to build an archive of the interviews, scanned documents and photographs. |
| Outcome: | The proposed archive will be accessible online and provide multifaceted search capabilities. |
Soft Alignment Objectives for Robust Adaptation of Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Domain adaptation is a common approach for generative language models, but it is notorious for over-specialization to the target domain, resulting in catastrophic forgetting. |
| Approach: | They propose to build training objectives on a semantic similarity of predicted tokens to the reference and avoid catastrophic forgetting of adaptation by preserving adaptation in-domain quality. |
| Outcome: | The proposed objectives mitigate catastrophic forgetting while preserving the adaptation in-domain quality while reducing computational costs. |
Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing work shows that Large Language Models (LLMs) are not robust to complex language understanding tasks due to reliance on spurious correlations of training datasets. |
| Approach: | They propose a method for measuring model reliance on spurious features by exploiting chosen biases on out-of-distribution (OOD) datasets. |
| Outcome: | The proposed method shows that the reported OOD gains of debiasing methods can't be explained by mitigated reliance on biased features, suggesting that biases are shared among different QA datasets. |
Towards the Roots of the Negation Problem: A Multilingual NLI Dataset and Model Scaling Analysis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Negations are key to determining sentence meaning, making them essential for logical reasoning. |
| Approach: | They construct and publish two new textual entailment datasets in four languages with paired examples differing in negation. |
| Outcome: | The results show that increasing the model size may improve the models’ ability to handle negations. |
Flowstorm: Open-Source Platform with Hybrid Dialogue Architecture (2022.naacl-demo)
Copied to clipboard
| Challenge: | Flowstorm is an open-source conversational AI platform that can be used as a web app. |
| Approach: | They propose a conversational AI platform called Flowstorm that uses a combination of tree structures and generative models to handle specific dialogue scenarios. |
| Outcome: | The proposed platform can be used as a web app or run on their own . it uses tree structures and generative models to create and analyze conversational applications . |
OodGAN: Generative Adversarial Network for Out-of-Domain Data Generation (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing models for OOD detection work with text, but they do not work directly with the text. |
| Approach: | They propose to use a sequential generative adversarial network (SeqGAN) based model to generate OOD data for a given domain automatically. |
| Outcome: | The proposed model outperforms state-of-the-art in OOD detection metrics for ROSTD and OSQ datasets. |