Papers by Abulhair Saparov
World Models for Math Story Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts to solve math story problems have lacked accurate representations of mathematical concepts. |
| Approach: | They propose a graph-based semantic formalism for solving math story problems . they combine existing datasets and annotate a corpus of 1,019 problems with MathWorld . |
| Outcome: | The proposed model can be used to solve math story problems with pre-trained language models . the model can also be used for generating new problems by using the model as a design space . |
LLMs Are Prone to Fallacies in Causal Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that causal facts can be extracted from LLMs through prompting . but it is unclear if this success is limited to explicitly-mentioned causal facts in pretraining data . |
| Approach: | They fine tune LLMs on synthetic data and test whether they can infer causal relations . they find that LLM can correctly deduce absence of causal relations from temporal and spatial relations if order is randomized . |
| Outcome: | The proposed model outperforms existing methods on causal inference tasks. |
Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnostic Behavioral Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the robustness of LLMs with few-shot prompting techniques are limited. |
| Approach: | They propose to test the robustness of LLMs in multi-hop reasoning tasks via domain-agnostic perturbations. |
| Outcome: | The proposed model is more sensitive to certain perturbations such as replacing words with synonyms and more robust to few-shot prompting methods. |
Towards General Natural Language Understanding with Probabilistic Worldbuilding (2022.tacl-1)
Copied to clipboard
| Challenge: | Probabilistic worldbuilding model is a Bayesian model of semantic parsing and reasoning . large-scale language models are domain-general, despite training on text from virtually every domain . |
| Approach: | They propose a Bayesian probabilistic worldbuilding model that parses and abduces sentences . they use a dataset to test their method against heuristics and to generate a probability model . |
| Outcome: | The proposed model outperforms baselines on two out-of-domain question-answering datasets. |
Personas as a Way to Model Truthfulness in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models are trained on vast amounts of text from the internet, which contains factual and misleading information. |
| Approach: | They hypothesize that the pretraining data is generated by groups of (un)truthful agents whose outputs share common features and form a (un-truthfully persona) this allows the model to separate truth from falsehoods and controls the truthfulness of its generation. |
| Outcome: | The proposed model can infer truth from falsehoods by finetuning its model on a set of facts and finetuned it on unseen topics. |