Papers with BI
BI-Bench : A Comprehensive Benchmark Dataset and Unsupervised Evaluation for BI Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing benchmarks focus on isolated components rather than addressing the broader needs of BI users. |
| Approach: | They propose a holistic, end-to-end benchmarking framework that categorizes queries into descriptive, diagnostic, predictive, and prescriptive types, aligning with practical BI needs. |
| Outcome: | The proposed framework assesses BI systems on quality, relevance, depth of insights based on queries categorized into descriptive, diagnostic, predictive, and prescriptive types . |
Schema Aware Semantic Reasoning for Interpreting Natural Language Queries in Enterprise Settings (2020.coling-main)
Copied to clipboard
| Challenge: | Using ontology reasoning to understand natural language is a challenge for QA systems . a recent study shows that ontologies can improve natural language understanding . |
| Approach: | They propose to use ontology reasoning to translate natural language interpretation into a sequence of solvable tasks by an ontologist. |
| Outcome: | The proposed framework achieves better natural language understanding with a 30% accuracy improvement over the current state of natural language query interfaces. |
Representational Analysis of Binding in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has shown that LMs use a concept called Binding ID (BI) to mark entity-attribute pairs, but have not captured the information from entity activations. |
| Approach: | They propose to localize the Binding ID mechanism by localizing BI information in LMs by encoding it in a low-rank subspace. |
| Outcome: | The proposed model can infer attributes for a given entity from a container . |
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2025.findings-acl)
Copied to clipboard
Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, Weipeng Chen
| Challenge: | Recent studies have identified significant redundancy in large language models . quantization and pruning are two methods that reduce computational resources . |
| Approach: | They propose simple pruning methods that prune redundant layers based on their BI scores. |
| Outcome: | The proposed pruning methods demonstrate superior performance over previous pruning methods. |
MdEval: Massively Multilingual Code Debugging (2026.findings-acl)
Copied to clipboard
Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li
| Challenge: | Existing benchmarks primarily focus on Python and are limited in terms of language diversity. |
| Approach: | They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions. |
| Outcome: | The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task. |