Papers by Steven Li
CONTOR: Benchmarking Strategies for Completing Ontologies with Plausible Missing Rules (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations focus on distinguishing held-out ontologies from randomly corrupted ones, which often makes the task unrealistically easy. |
| Approach: | They propose to use the common description logic syntax for encoding ontology rules to test their effectiveness on manually annotated hard negatives. |
| Outcome: | The proposed models are compared with existing models and have been evaluated on different ontologies. |
Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction (2026.acl-long)
Copied to clipboard
Advait Gosai, Tyler Vuong, Utkarsh Tyagi, Steven Li, Wenjia You, Miheer Bavare, Arda Uçar, Zhongwang Fang, Brian Jang, Bing Liu, Yunzhong He
| Challenge: | End-to-end (E2E) spoken dialogue systems are replacing cascaded pipelines for voice-based human-AI interaction. Existing benchmarks evaluate these systems on synthetic speech and single-turn tasks, leaving multi-turn conversational ability underexplored. |
| Approach: | They propose an open-source benchmark to evaluate spoken dialogue systems under natural multi-turn interaction patterns. |
| Outcome: | The proposed model fails on the highest-performing model with 54.65% pass rate. |
What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases (2024.naacl-long)
Copied to clipboard
| Challenge: | Vision-language models have broad competence that is difficult to evaluate . current evaluation benchmarks focus on only assessing one or a few capabilities . |
| Approach: | They perform a large-scale transfer learning experiment to discover latent VL skills from data. |
| Outcome: | The results suggest that factor analysis can identify reasonable yet surprising VL skill factors . the results contribute to the design of balanced and broad-coverage vision-language evaluation methods. |
Two-Stage Fine-Tuning for Improved Bias and Variance for Large Pretrained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work challenges the bias-variance trade-off . large pretrained models can have large variance and overfit domain-specific data . |
| Approach: | They propose a bias-variance trade-off that implies learning methods need to balance complexity with data size to minimize under-fitting and over-fit. |
| Outcome: | The proposed method achieves strong results on SuperGLUE and clinical information extraction tasks. |
Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings (2024.findings-acl)
Copied to clipboard
| Challenge: | Concept embeddings are a useful and efficient mechanism for injecting commonsense knowledge into downstream tasks. |
| Approach: | They propose to model commonalities in concepts by capturing a more diverse range of commonsense properties. |
| Outcome: | The proposed model captures a more diverse range of commonsense properties and improves ontology completion and ultra-fine entity typing tasks. |
CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing code LLMs adopt a specific architecture or rely on a unified encoder-decoder network for downstream tasks, lacking flexibility to operate in the optimal architecture for a particular task. |
| Approach: | They propose to initialize code LLMs with frozen off-the-shelf LLM and explore instruction-tuning to align with natural language instructions. |
| Outcome: | The proposed model outperforms open-source LLMs on 20 code-related benchmarks. |
What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on decontextualised concept embeddings from language models has focused on capturing taxonomic structure in concepts. |
| Approach: | They propose a strategy for identifying what different concepts have in common with others and representing them in terms of their properties. |
| Outcome: | The proposed approach improves the performance of state-of-the-art models for a multi-label classification problem. |
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts (2023.emnlp-main)
Copied to clipboard
Truong Do, Le Khiem, Quang Pham, TrungTin Nguyen, Thanh-Nam Doan, Binh Nguyen, Chenghao Liu, Savitha Ramasamy, Xiaoli Li, Steven Hoi
| Challenge: | Recent studies suggest that fixing the routers can achieve competitive performance by alleviating the collapsing problem, where all experts eventually learn similar representations. |
| Approach: | They propose a method that dynamically generates router parameters through a fixed hypernetwork and trainable embeddings to achieve a balance between training the routers and freezing them to learn an improved routing policy. |
| Outcome: | Experiments on a wide range of tasks show that the proposed method performs better than existing methods. |
Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ultra-fine entity typing is a task of inferring the semantic types from a large set of fine-grained candidates that apply to a given entity mention. |
| Approach: | They propose to use pre-trained label embeddings to cluster the labels into semantic domains and treat them as additional types. |
| Outcome: | The proposed method improves the performance of existing models with high quality embeddings. |