Papers by Ruoxi Ning
TAIL: A Toolkit for Automatic and Realistic Long-Context Large Language Model Evaluation (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Existing evaluation methods for long-context large language models are overly simplistic and require extensive human annotations. |
| Approach: | They propose an automatic toolkit to create realistic evaluation benchmarks . they use a document-grounded benchmark to generate question-answer pairs . |
| Outcome: | The proposed toolkit provides a way to create realistic evaluation benchmarks and visualize performance metrics of evaluated models. |
On the Effect of Hyperparameters in Language Modeling for Computational Linguistics (2026.acl-long)
Copied to clipboard
| Challenge: | Training language models and examining their linguistic behaviors is a common protocol in computational linguistics for studying linguistic phenomena and modeling human language processing. |
| Approach: | They replicate three prior studies with hyperparameters varied within a practical range and show that modest hyperparametric changes can alter qualitative conclusions about models’ linguistic abilities. |
| Outcome: | The results show that hyperparameter changes can alter qualitative conclusions and reverse the ranking of models. |
MAViS: A Multi-Agent Framework for Long-Sequence Video Storytelling (2026.eacl-long)
Copied to clipboard
| Challenge: | Experimental results show that MAViS achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. |
| Approach: | They propose an end-to-end multi-agent collaborative framework for long-sequence video storytelling that orchestrates specialized agents across multiple stages. |
| Outcome: | The proposed framework achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. |
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens (2025.emnlp-demos)
Copied to clipboard
Hala Sheta, Eric Haoran Huang, Shuyu Wu, Ilia Alenabi, Jiajun Hong, Ryker Lin, Ruoxi Ning, Daniel Wei, Jialin Yang, Jiawei Zhou, Ziqiao Ma, Freda Shi
| Challenge: | Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance. |
| Approach: | They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs. |
| Outcome: | The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic. |
Challenges to Open-Domain Constituency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing findings on cross-domain constituency parsing are only made on a limited number of domains. |
| Approach: | They manually annotate a high-quality constituency treebank containing five domains and analyze challenges to open-domain constituency parsing using a set of linguistic features. |
| Outcome: | The proposed model significantly improves the performance of the proposed model on the domain-variant features. |
FASTTRACK: Reliable Fact Tracing via Clustering and LLM-Powered Evidence Validation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to fact tracing rely on assessing the similarity between training samples and the query along a certain dimension, such as lexical similarity, gradient, or embedding space. |
| Approach: | They propose a new approach that harnesses the capabilities of Large Language Models to validate supportive evidence for queries and clusters the training database towards a reduced extent for LLMs to trace facts. |
| Outcome: | The proposed approach outperforms existing methods in accuracy and efficiency while being x33 faster than TracIn. |