Papers by Matthias Lindemann
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations (2023.acl-long)
Copied to clipboard
| Challenge: | Seq2seq models struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions or deeper recursion of phenomena that the model handles correctly in isolation. |
| Approach: | They propose a new way of parameterizing and predicting permutations by combining input tokens with multisets of output tokens and a method to backpropagate through the solver. |
| Outcome: | The proposed model outperforms pretrained models and prior work on realistic semantic parsing tasks that require generalization to longer examples. |
Normalizing Compositional Structures Across Graphbanks (2020.coling-main)
Copied to clipboard
| Challenge: | Graph-based meaning representations (MRs) exhibit structural differences that reflect different theoretical and design considerations, presenting challenges to uniform linguistic analysis and cross-framework semantic parsing. |
| Approach: | They propose a method to normalize MRs at the compositional level by linguistically-grounded rules. |
| Outcome: | The proposed method increases the match in compositional structure between MRs and improves multi-task learning in a low-resource setting. |
AMR dependency parsing with a typed semantic algebra (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representations (AMRs) are graphs which describe the predicate-argument structure of a sentence. |
| Approach: | They propose a semantic parser which parses strings into tree representations of the compositional structure of an AMR graph. |
| Outcome: | The proposed parser outperforms baselines and standard neural techniques for supertagging and dependency tree parsing. |
Cache & Distil: Optimising API Calls to Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models are expensive to run and expose the entire request stream to external providers. |
| Approach: | They propose to locally train a small private language model on the LLM's predictions to minimise the costs and data exposure associated with calling the API. |
| Outcome: | The proposed model can handle an increasing number of user requests independently and is able to perform better than other policies and baselines across tasks and budgets. |
Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Inductive biases play a critical role in NLP, especially in learning from limited data and generalizing systematically outside of the training distribution. |
| Approach: | They propose to strengthen the structural inductive bias of a Transformer by intermediate pre-training to perform syntactic transformations of dependency trees given a description of the transformation. |
| Outcome: | The proposed model can perform syntactic transformations and generalize semantic parsing with attention heads that keep track of which syntaktic transformation needs to be applied to which token. |
Compositional Semantic Parsing across Graphbanks (P19-1)
Copied to clipboard
| Challenge: | Existing semantic parsers that map sentences to graph-based meaning representations are hand-designed for specific graphbanks. |
| Approach: | They propose a compositional neural semantic parser which achieves competitive accuracies across graphbanks. |
| Outcome: | The proposed system achieves competitive accuracies across a variety of graphbanks. |
SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation (2024.acl-long)
Copied to clipboard
| Challenge: | Popular neural architectures lack strong structural inductive biases for seq2seq NLP tasks . previous work shows that these models struggle with systematic generalization . |
| Approach: | They propose to inject a structural inductive bias into a seq2seq model by pre-training it to simulate structural transformations on synthetic data. |
| Outcome: | The proposed method improves few-shot learning and generalization of FST-like models. |
Compositional Generalisation with Structured Reordering and Fertility Layers (2023.eacl-main)
Copied to clipboard
| Challenge: | Seq2seq models struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. |
| Approach: | They propose a flexible end-to-end differentiable neural model that composes two structural operations: a fertility step and a reordering step. |
| Outcome: | The proposed model outperforms seq2seq models on compositional splits of realistic semantic parsing tasks. |
Fast semantic parsing with well-typedness guarantees (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing algorithms for AM dependency parsing are slow and do not support linguistic principles. |
| Approach: | They propose an A* parser and a transition-based parsing algorithm which guarantee well-typedness and improve parse speed by up to 3 orders of magnitude. |
| Outcome: | The proposed algorithms guarantee well-typedness and improve parsing speed by up to 3 orders of magnitude while maintaining or improving accuracy. |