Papers by Chao Lou
Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints (2023.acl-short)
Copied to clipboard
| Challenge: | Neural QCFG excels in interpretability and generalization but suffers from expensive inference. |
| Approach: | They propose to use a symbolic grammar to create QCFGs with a quasisynchronous context-free grammar that is parameterized by neural networks to perform faster inference. |
| Outcome: | The proposed models outperform vanilla Neural QCFG in most settings. |
RRAtention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference (2026.acl-long)
Copied to clipboard
Siran Liu, Guoxia Wang, Sa Wang, Jinle Zeng, Haoyang Xie, Siyu Lou, Jiabin Yang, Dianhai Yu, Haifeng Wang, Chao Yang
| Challenge: | Existing approaches to dynamic sparse attention require preprocessing, lack global evaluation, violate query independence, or incur high computational overhead. |
| Approach: | They propose a dynamic sparse attention method that achieves all desirable properties through a head **r**ound-**r**obin (RR) sampling strategy. |
| Outcome: | Experiments on natural language understanding and multimodal video comprehension show that the proposed method achieves 2.4 speedup at 128K context length outperforming existing methods. |
Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Syntactic Transformer language models aim to achieve better generalization through simultaneously modeling syntax trees and sentences. |
| Approach: | They propose a class of Transformer language models with explicit dependency-based inductive bias. |
| Outcome: | Experiments show that the proposed models outperform constituency-based models on sentences annotated with dependency trees and achieve better generalization. |
Spa: On the Sparsity of Virtual Adversarial Training for Dependency Parsing (2022.findings-aacl)
Copied to clipboard
| Challenge: | Virtual adversarial training (VAT) is a powerful approach to improving robustness and performance, leveraging both labeled and unlabeled data to compensate for the scarcity of labeles. |
| Approach: | They propose a Sparse Parse Adjustment algorithm which combines VAT and a graph-based dependency parsing model in an exact computational manner and enhances the dependency parsed with controllable and adjustable sparsity. |
| Outcome: | Empirical results show that the proposed algorithm outperforms other methods without sparsity regularization. |
Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to recognize named entities have been criticized for their performance on flat NER but fail to handle nested entities. |
| Approach: | They propose to use a span-based constituency parser to tackle nested NER . they use lexicalized constituency trees to model nesting entities . |
| Outcome: | The proposed method achieves state-of-the-art performance on ACE2004, ACE2005 and NNE, and competitive performance on the GENIA platform. |
Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point Process (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large-scale labeled support sets are not feasible in practical scenarios. |
| Approach: | They introduce a language model-based determinant point process that considers uncertainty and diversity of unlabeled instances for optimal selection. |
| Outcome: | The proposed method can effectively select canonical examples on 9 NLU and 2 Generation datasets. |
AMR Parsing with Causal Hierarchical Attention and Pointers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation of natural language sentences. |
| Approach: | They propose a new target form of AMR parsing and a model which integrates structural localities into the Transformer decoder. |
| Outcome: | The proposed model outperforms baseline models on four out of five benchmarks in the setting of no additional data. |