Papers by Miloš Stanojević
Do LLMs learn a true syntactic universal? (2024.emnlp-main)
Copied to clipboard
| Challenge: | linguistics literature has debated whether large multilingual language models learn language universals . Typological generalizations are a key battleground in such debates - e.g. van der Hulst, 2023, chapter 7). |
| Approach: | They consider a candidate universal for language universals, the Final-over-Final Condition . they suggest that modern language models may need additional sources of bias to become truly human-like . |
| Outcome: | The proposed model only seems to recognize the Final-over-Final Condition in German, Russian, Hungarian and Serbian . |
CCG Parsing Algorithm with Incremental Tree Rotation (N19-1)
Copied to clipboard
| Challenge: | Combinatory Categorial Grammar (CCG) is a mildly context sensitive grammar formalism that excels in incremental sentence processing. |
| Approach: | They propose a new incremental parsing algorithm that uses a syntactic approach . it uses right-branching constituent structures and optional constituents that adjoin on the right . |
| Outcome: | The proposed algorithm can cover the whole CCGbank with greater incrementality and accuracy than previous proposals. |
Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale (2022.tacl-1)
Copied to clipboard
| Challenge: | a novel class of Transformer language models that combine expressive power, scalability, and strong performance of Transformers and recursive syntactic compositions. |
| Approach: | They introduce Transformer Grammars, a class of Transformer language models that combine expressive power and recursive syntactic compositions. |
| Outcome: | The proposed model outperforms strong baselines on sentence-level language modeling perplexity and syntax-sensitive language evaluation metrics. |
Extracting structure from an LLM - how to improve on surprisal-based models of Human Language Processing (2025.coling-main)
Copied to clipboard
| Challenge: | Existing computational models capture prediction and reanalysis using Large Language Models (LLMs) and a statistical measure known as ‘surprisal’. |
| Approach: | They propose to extract structural information from Large Language Models and a statistical measure known as ‘surprisal’ to integrate it with their learnt statistics. |
| Outcome: | The proposed model achieved higher correlation with human reading times and better predicted the garden path effect and could distinguish between sentence types with different levels of difficulty. |
Multipath parsing in the brain (2024.acl-long)
Copied to clipboard
| Challenge: | a major unsolved problem in computational psycholinguistics is determining whether human comprehension considers a single analysis path 1 at a time. |
| Approach: | They compare syntactic surprisal from a state-of-the-art dependency parser with fMRI data . they find evidence for multipath parsing in English and Chinese data based on fm data a major unsolved problem in computational psycholinguistics is determining whether human sentence comprehension considers a single analysis path 1 at a time . |
| Outcome: | The proposed model shows that human parsing is multipath, with a higher r2 increase for multipath surprisal than single-path surpresal. |
Max-Margin Incremental CCG Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | a new incremental parser reduces the number of beam search violations and minimises the biggest violation. |
| Approach: | They propose to use beam search optimisation to minimise all beam search violations instead of minimising only the biggest violation. |
| Outcome: | The proposed parser outperforms existing non-incremental parsers and minimises all beam search violations instead of minimising the biggest violation. |
Unbiased and Efficient Sampling of Dependency Trees (2022.emnlp-main)
Copied to clipboard
| Challenge: | linguistic constraints in dependency trees are not part of the definition of spanning trees. |
| Approach: | They propose to use a constraint that requires a single root to be incorporated into dependency tree sampling . they propose to reduce the asymptotic runtime of sampling k trees without replacement to O(kn3) |
| Outcome: | The proposed algorithms are asymptotically and practically more efficient . they reduce the runtime of the fastest algorithm for sampling with replacement to O(kn3) |
SynJax: Structured Probability Distributions for JAX (2023.emnlp-demo)
Copied to clipboard
| Challenge: | a number of deep learning libraries have been developed to account for structured objects . a vectorized implementation of inference algorithms for structured distributions is difficult to implement . |
| Approach: | SynJax provides vectorized implementations of inference algorithms for structured distributions . authors propose to use a vectorized version of the algorithms to model structure in data . similar structures appear in biology and chemistry . |
| Outcome: | SynJax provides an efficient vectorized implementation of inference algorithms for structured distributions. |
A Root of a Problem: Optimizing Single-Root Dependency Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Graph-based dependency parsers can be improved without compromising on accuracy or accuracy. |
| Approach: | They propose two approaches to single-root dependency parsing that yield speed ups . they show that one approach is fully correct and finds the optimal dependency tree . |
| Outcome: | The proposed approach finds the optimal dependency tree without loss of accuracy or optimality. |
Wide-Coverage Neural A* Parsing for Minimalist Grammars (P19-1)
Copied to clipboard
| Challenge: | a new parser for wide-coverage parsing uses a linguistically expressive yet highly constrained grammar . the expected time complexity of the parsers is cubic in the length of the sentence . |
| Approach: | They propose to use a linguistically expressive yet highly constrained grammar to parse a wide-coverage sentence using a bi-LSTM neural network supertagger. |
| Outcome: | The proposed algorithm recovers unbounded long distance dependencies and can recover unbundled long distance dependents. |