Papers by See-Kiong Ng
A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the CHATGPT Era and Beyond (2024.eacl-long)
Copied to clipboard
Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Viktor Schlegel, Stefan Winkler, See-Kiong Ng, Soujanya Poria
| Challenge: | Sentence representations are a critical component in NLP applications such as retrieval, question answering, and text classification. |
| Approach: | They present a systematic review of the literature on sentence representations focusing mostly on deep learning models. |
| Outcome: | The proposed methods highlight the key contributions and challenges in this area and suggest potential avenues for improving the quality and efficiency of sentence representations. |
How Does Response Length Affect Long-Form Factuality (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite growing attention to LLM factuality, the effect of response length on factual accuracy remains underexplored. |
| Approach: | They propose an automatic and bi-level long-form factuality evaluation framework which achieves high agreement with human annotations while being cost-effective. |
| Outcome: | The proposed framework achieves high agreement with human annotations while being cost-effective. |
Media Source Matters More Than Content: Unveiling Political Bias in LLM-Generated Citations (2025.emnlp-main)
Copied to clipboard
| Challenge: | generative search engines rely on in-line citations as the key gateway to original webpages . a recent study shows that LLMs tend to cite left-leaning sources at higher rates compared to traditional retrieval systems . |
| Approach: | They construct a dataset of news articles labeled with left- or right-leaning stances . they find that LLMs tend to cite left-leansing sources at higher rates than traditional retrieval systems . |
| Outcome: | The proposed dataset shows that LLMs tend to cite left-leaning sources at higher rates than traditional retrieval systems. |
Encoding and Controlling Global Semantics for Long-form Video Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to find answers for long videos fail to reason over the whole sequence of video, leading to sub-optimal performance. |
| Approach: | They propose a state space layer to integrate global semantics into video . they use a gating unit to enable controllability over the flow of global semantic into visual representations. |
| Outcome: | The proposed framework is able to integrate global semantics into visual representations. |
Uncovering Scaling Laws for Large Language Models via Inverse Problems (2025.findings-emnlp)
Copied to clipboard
Arun Verma, Zhaoxuan Wu, Zijian Zhou, Xiaoqiang Lin, Zhiliang Chen, Rachael Hwee Ling Sim, Rui Qiao, Jingtan Wang, Nhung Bui, Xinyuan Niu, Wenyang Hu, Gregory Kang Ruey Lau, Zi-Yu Khoo, Zitong Zhao, Xinyi Xu, Apivich Hemachandra, See-Kiong Ng, Bryan Kian Hsiang Low
| Challenge: | Large Language Models (LLMs) have achieved remarkable success across diverse domains. |
| Approach: | inverse problems can efficiently uncover scaling laws that guide the building of LLMs, authors argue . authors propose brute-force approaches to improve LLM training costs due to high costs . |
| Outcome: | This paper advocates that inverse problems can efficiently uncover scaling laws that guide the building of LLMs to achieve the desirable performance with significantly better cost-effectiveness. |
QSTS: A Question-Sensitive Text Similarity Measure for Question Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing measures for question generation have been inadequately evaluated . current research uses QA datasets containing pairs of (reference question, passage context) elements. |
| Approach: | They propose a Question-Sensitive Text Similarity measure for comparing two questions . they also propose enabling question similarity research in QG contexts by using a dataset called SimQG. |
| Outcome: | The proposed measure overcomes shortcomings of existing measures that depend on n-gram overlap scores and obtains superior results compared to existing measures on publicly-available QG datasets. |
CodeArena: A Collective Evaluation Platform for LLM Code Generation (2025.acl-demo)
Copied to clipboard
Mingzhe Du, Anh Tuan Luu, Bin Ji, Xiaobao Wu, Yuhao Qing, Dong Huang, Terry Yue Zhuo, Qian Liu, See-Kiong Ng
| Challenge: | Large Language Models (LLMs) have reshaped code generation, but persistent challenges impede accurate assessment. |
| Approach: | They propose an online evaluation framework tailored for large language models to assess their coding capabilities. |
| Outcome: | a new evaluation framework for large language models (LLMs) provides unbiased, unbiased evaluations and open access to solutions and test cases. |
CorefDiffs: Co-referential and Differential Knowledge Flow in Document Grounded Conversations (2022.coling-1)
Copied to clipboard
| Challenge: | Document-grounded dialogs need smooth transitions between knowledge selected for generating responses. |
| Approach: | They propose a multi-document co-referential graph to capture inter- and intra-document relationships . they propose 'Coref-MDG' method to linearize static Coref-mDG into conversational sequence logic. |
| Outcome: | The proposed method outperforms the state-of-the-art by 9.5%, 7.4% and 8.2% on three public benchmarks. |
HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to maintain stable understanding performance and low GPU memory overhead. |
| Approach: | They propose a training-free architecture for real-time and accurate understanding of video streams . HERMES reuses a compact KV cache, enabling efficient streaming understanding . |
| Outcome: | The proposed architecture achieves 10 faster TTFT compared to prior SOTA. |
FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing fact-checking evaluation methods rely on static datasets and classification metrics, which fail to evaluate justification production and uncover the nuanced limitations of LLMs. |
| Approach: | They propose a framework that adaptively and dynamically assesses LLMs’ fact-checking capabilities by incorporating justification production alongside verdict prediction. |
| Outcome: | Experiments show that the framework differentiates among state-of-the-art LLMs, providing valuable insights into model strengths and limitations in model-centric fact-checking analysis. |
Ask-before-Plan: Proactive Language Agents for Real-World Planning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | despite the advancements of large language models, the potential of LLM-powered agents to comprehend ambiguous user instructions is still under exploration. |
| Approach: | They propose a task that requires agents to predict clarification needs based on conversation and agentenvironment interaction and generate a plan to fulfill the user's demands. |
| Outcome: | The proposed framework is based on a new ask-before-plan benchmark dataset. |
ESTeR: Combining Word Co-occurrences and Word Associations for Unsupervised Emotion Detection (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies list as many as 154 human emotions, but most researchers agree on basic emotions such as anger, fear, disgust, sadness, surprise, and happiness. |
| Approach: | They propose an unsupervised model for identifying emotions using a novel similarity function based on random walks on graphs. |
| Outcome: | The proposed model can be computed efficiently and avoids dependence on labeled datasets. |
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives (2024.findings-acl)
Copied to clipboard
Thong Nguyen, Yi Bin, Junbin Xiao, Leigang Qu, Yicong Li, Jay Zhangjie Wu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu
| Challenge: | Existing video-language understanding systems with human-like senses can mimic both our linguistic medium and visual environment with temporal dynamics. |
| Approach: | They propose to develop video-language understanding systems with human-like senses . they summarize their methods and highlight challenges associated with them . |
| Outcome: | The proposed models perform well in a variety of tasks and domains. |
SemRoDe: Macro Adversarial Training to Learn Representations that are Robust to Word-Level Attacks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to defend against word-level attacks have been limited. |
| Approach: | They propose a new approach called Semantic Robust Defence to enhance the robustness of language models by aligning the domains with a distance-based objective. |
| Outcome: | The proposed approach can be generalized across word embeddings, even when they share minimal overlap at both vocabulary and word-substitution levels. |
Knowledge Boundary of Large Language Models: A Survey (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) store vast amount of knowledge in their parameters, but they still have limitations in the memorization and utilization of certain knowledge. |
| Approach: | They propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. |
| Outcome: | The proposed definition of the LLM knowledge boundary and taxonomy categorizes knowledge into four distinct types . aims to offer a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM research. |
Identifying Early Maladaptive Schemas from Mental Health Question Texts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for identifying maladaptive schemas are not practical in 'offline' counseling scenarios such as community QA forums. |
| Approach: | They propose to use large language models and non-LLM methods to identify early maladaptive schemas (EMS) in Schema Therapy. |
| Outcome: | The proposed methods show complementary properties and can be used to further devise techniques for EMS identification. |
On Assigning Product and Software Codes to Customer Service Requests with Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | In a technology company, quality of customer service is a crucial asset. |
| Approach: | They propose to use Large Language Models to assign product names and software version labels to customer Service Requests (SRs) they frame assignment as multiple-choice question answering task instead of conventional prompts . |
| Outcome: | The proposed model can identify product names and software versions when they are mentioned with over 90% accuracy while cutting LLM costs by 40-60% on average. |
Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing image instruction fine-tuning datasets do not fully exploit visual information to enhance multimodal reasoning capabilities of Large language models (LLMs). |
| Approach: | They propose a LLaVA-based model fine-tuned with MathV360K to bridge this gap by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs. |
| Outcome: | The proposed model improves the multimodal reasoning capabilities of LLaVA-1.5 and demonstrates enhanced generalizability on the MMMU benchmark. |
Are All the Datasets in Benchmark Necessary? A Pilot Study of Dataset Evaluation for Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing benchmark datasets contribute little to discriminating top-scoring systems, while those less used datasets exhibit impressive discriminative power. |
| Approach: | They examine the distinguishability of benchmark datasets when comparing different systems . they find that existing benchmark dataset contribute little to discriminating top-scoring systems - whereas those less used datasets exhibit impressive discriminative power. |
| Outcome: | The proposed datasets are released on DataLab. |
Beyond Prompt: Fine-grained Simulation of Cognitively Impaired Standardized Patients via Stochastic Steering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training patients with cognitive impairment rely on discrete prompt engineering and fail to capture the heterogeneity of deficits across domains and severity levels. |
| Approach: | They propose to use steering vectors from contrastive pairs of instructions and responses to capture domain-specific features and introduce a Stochastic Token Modulation mechanism to regulate the intervention probability. |
| Outcome: | The proposed model outperforms baselines in clinical authenticity and severity controllability while remaining open-source. |
Non-Autoregressive Math Word Problem Solver with Unified Tree Structure (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing MWP solvers do not handle variants that can be derived via mathematical manipulation. |
| Approach: | They propose a non-autoregressive solver to present a solution expression and decode it from a given problem description. |
| Outcome: | The proposed solver is able to decode multiple expression variants and correct them . it is based on a unified tree structure and is available on Math23K and MAWPS. |
Pro-QuEST: A Prompt-chain based Quiz Engine for testing Specialized Technical Product Knowledge (2026.eacl-demo)
Copied to clipboard
| Challenge: | Specialized benchmarks can be leveraged to create quizzes that can effectively train engineering and marketing personnel on novel product offerings in a continually growing Cisco product space. |
| Approach: | They propose to generate multiple-choice questions using domain-specific prompts using a set of professional certification textbooks and a range of latest open-source and proprietary LLMs. |
| Outcome: | The proposed quiz engine generates multiple-choice questions using domain-specific prompts and a range of latest open-source, and proprietary LLMs. |
A Federated Framework for LLM-based Recommendation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated potential in building generative recommendation systems through fine-tuning user behavior data. |
| Approach: | They propose a federated framework for LLM-based recommendation that combines dynamic parameter aggregation and learning speed for different clients. |
| Outcome: | The proposed framework achieves a more balanced client performance and improved overall performance in a computational and storage-efficient way while safeguarding user privacy well. |
Polyglot Prompt: Multilingual Multitask Prompt Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | a monolithic framework for multilingual learning can be used without any task/language-specific module. |
| Approach: | They propose a framework to exploit prompting methods for learning a unified semantic space for different languages and tasks with multilingual prompt engineering. |
| Outcome: | The proposed framework can learn tasks from different languages in a monolithic framework without any task/language-specific module. |
DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Temporal Language Grounding (TLG) is a task to determine temporal boundaries of video moments that correspond to a language query. |
| Approach: | They propose an energy-based model framework to explicitly learn moment-query distributions. |
| Outcome: | The proposed model outperforms the state-of-the-art models on four public temporal language grounding datasets. |
Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models rely on sequential queries . however, existing methods rely heavily on sequential querying . |
| Approach: | They propose a training-free framework that transforms a single LLM into an effective inference-time ensemble. |
| Outcome: | The proposed framework outperforms existing models on reasoning benchmarks, such as MATH, and improves on a DIPPER ensemble of three Qwen2-MATH-1.5B instances. |
PIRsuader: A Persuasive Chatbot for Mitigating Psychological Insulin Resistance in Type-2 Diabetic Patients (2025.coling-main)
Copied to clipboard
| Challenge: | Psychological Insulin Resistance (PIR) is a psychological barrier in diabetic patients . many patients have deep-rooted fears and misgivings related to insulin which hinder them from initiation of insulin-based treatments. |
| Approach: | They propose a persuasive chatbot that can mitiga psychological insulin resistance in diabetic patients . they harness conversation generation capabilities of state-of-the-art Large Language Models . |
| Outcome: | The proposed chatbot improves the willingness of patients to try insulin and addresses concerns in an empathetic manner. |
Socratic Question Generation: A Novel Dataset, Models, and Evaluation (2023.eacl-main)
Copied to clipboard
| Challenge: | Socratic questioning is a form of reflective inquiry often employed in education to encourage critical thinking in students. |
| Approach: | They present a first large dataset of 110K questions, context pairs for Socratic Question Generation. |
| Outcome: | The proposed model produces realistic, type-sensitive, human-like Socratic questions . authors show that the model can be used in counseling and coaching . |
Towards Proactive Information Probing: Customer Service Chatbots Harvesting Value from Conversation (2026.findings-acl)
Copied to clipboard
| Challenge: | a new technology is transforming customer service chatbots into strategic bridges for business intelligence . a recent study shows that customer service bots are increasingly being used as reactive support tools . |
| Approach: | They propose a task of Proactive Information Probing which optimizes when to probe users for pre-specified information while minimizing conversation turns and user friction. |
| Outcome: | The proposed framework outperforms baselines in both information probing and service quality. |
METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks evaluate contextual causal reasoning in fragmented settings, failing to ensure context consistency or cover the full causal hierarchy. |
| Approach: | They use a unified context to benchmark large language models' contextual causal reasoning skills. |
| Outcome: | The proposed benchmarks show that LLMs are susceptible to distraction by irrelevant but factually correct information at lower level of causality. |
MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced natural language processing, demonstrating exceptional reasoning, tool usage, and memory capabilities. |
| Approach: | They propose a competition-based benchmark framework specifically designed to assess LLMs within multi-agent environments. |
| Outcome: | The proposed framework enhances the LLMs’ abilities in navigating complex social and cognitive dimensions by over threefold between the strongest and weakest LLM models. |
Chain-of-Exemplar: Enhancing Distractor Generation for Multimodal Educational Question Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multiple choice questions focus on text inputs and lack visual information. |
| Approach: | They propose a framework to generate subject-specific educational questions with plausible distractors based on multimodal content. |
| Outcome: | The proposed framework improves question generation and distractor generation over existing methods across subjects and educational levels. |
METRO: Towards Strategy Induction from Expert Dialogue Transcripts for Non-collaborative Dialogues (2026.acl-long)
Copied to clipboard
| Challenge: | Developing non-collaborative dialogue agents traditionally requires manual codification of expert strategies. |
| Approach: | They propose a method that formalizes expert knowledge into a Strategy Forest from raw transcripts. |
| Outcome: | The proposed method outperforms existing methods by 9%-10% in two benchmarks. |
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models (2025.acl-long)
Copied to clipboard
Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, WangYan WangYan, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, Bryan Hooi
| Challenge: | Large language models face significant challenges in handling long-context tasks because of their limited effective context window size during pretraining, which restricts their ability to generalize over extended sequences. |
| Approach: | They propose a training strategy for extending the context window of LLMs including impactful token analysis, position index transformation, and training optimization strategies. |
| Outcome: | Experiments on three types of LLMs show that LongRecipe can utilize long sequences while requiring only 30% of the target context window size. |
WASA: WAtermark-based Source Attribution for Large Language Model-Generated Data (2025.findings-acl)
Copied to clipboard
Xinyang Lu, Jingtan Wang, Zitong Zhao, Zhongxiang Dai, Chuan-Sheng Foo, See-Kiong Ng, Bryan Kian Hsiang Low
| Challenge: | Large language models (LLMs) have impressive performance but intellectual property concerns are looming . a framework that can be used to perform source attribution for LLMs can be developed. |
| Approach: | They propose a framework that enables an LLM to generate synthetic texts with embedded watermarks that contain information about their source. |
| Outcome: | The proposed framework achieves source attribution accuracy and robustness against adversaries. |
Don’t Just Say “I don’t know”! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies investigate ways to refuse to answer unknown questions . Large Language Models (LLMs) display a significant level of overconfidence when answering questions that they are aware of. |
| Approach: | They propose a self-alignment method to utilize Large Language Models to enhance its response-ability to unknown questions. |
| Outcome: | The proposed method is superior to baseline methods on four types of unknown questions. |
AgentPro: Enhancing LLM Agents with Automated Process Supervision (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks lack explicit supervision during the reasoning process, which may lead to error propagation across reasoning chains. |
| Approach: | They propose a framework which automates process supervision for large language model agents by automatically generating step-level annotations and developing a process reward model based on these annotations. |
| Outcome: | The proposed framework outperforms existing agent-based methods on four datasets and achieves a 6.32% increase in accuracy. |
GPTScore: Evaluate as You Desire (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks for text generation are not adequate to assess the quality of the generated outputs. |
| Approach: | They propose a framework that utilizes emergent abilities of generative pre-trained models to evaluate generated texts. |
| Outcome: | The proposed evaluation framework can achieve what one desires to evaluate for texts simply by natural language instructions. |
Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models (2026.acl-long)
Copied to clipboard
Jingwei Ni, Ekaterina Fadeeva, Tianyi Wu, Mubashara Akhtar, Jiaheng Zhang, Elliott Ash, Markus Leippold, Timothy Baldwin, See-Kiong Ng, Artem Shelmanov, Mrinmaya Sachan
| Challenge: | Existing verification approaches, such as Process Reward Models, are computationally expensive and limited to specific domains. |
| Approach: | They propose a transformer-based probe that uses internal states of frozen LLMs to estimate credibility of reasoning steps during generation. |
| Outcome: | The proposed probes match or exceed PRMs that are up to 810 larger. |
Investigating and Enhancing the Robustness of Large Multimodal Models Against Temporal Inconsistency (2025.acl-long)
Copied to clipboard
Jiafeng Liang, Shixin Jiang, Xuan Dong, Ning Wang, Zheng Chu, Hui Su, Jinlan Fu, Ming Liu, See-Kiong Ng, Bing Qin
| Challenge: | Large Multimodal Models (LMMs) have demonstrated impressive performance on general video comprehension benchmarks, but their robustness needs to be thoroughly investigated for broader applications. |
| Approach: | They propose a temporal robustness benchmark which introduces temporal inconsistency perturbations separately at the visual and textual modalities to assess the robustness of models. |
| Outcome: | The proposed method improves the model’s robustness and reliability in temporal analysis. |
Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs (2026.findings-acl)
Copied to clipboard
Zhiyuan Hu, Yucheng Wang, Yufei He, Jiaying Wu, Yilun Zhao, See-Kiong Ng, Cynthia Breazeal, Anh Tuan Luu, Hae Won Park, Bryan Hooi
| Challenge: | Reinforcement learning (RL) is a paradigm for post-training large language models, but it suffers from exploration collapse . a new study finds that RL fails to reward correct solutions that exhibit rare high-level strategies . |
| Approach: | They propose a method that rewards correct solutions that exhibit rare high-level strategies by clustering rollouts according to their high- level solution strategies. |
| Outcome: | The proposed approach improves pass@k across large sampling budgets and increases area under the pass@K curve (AUC@K) without sacrificing pass@1. |
On the Multi-turn Instruction Following for Conversational Web Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable abilities in planning and executing multi-step interactions within web-based environments. |
| Approach: | They propose a framework for conversational web navigation that uses multi-turn interactions with both the user and the environment. |
| Outcome: | The proposed framework is based on a multi-turn Mind2Web (MT-Mind2Web) it is designed to perform multi-step interactions with web-based environments . |