Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Copied to clipboard
| Challenge: | Interpreto is an open-source Python library for interpreting HuggingFace language models . it provides attribution methods and concept-based explanations . documentation or metrics are sometimes missing due to the complexity of the pipeline . |
| Approach: | Interpreto is an open-source Python library for interpreting HuggingFace language models . it provides attribution methods and concept-based explanations . authors welcome issues and pull requests . |
| Outcome: | Interpreto is an open-source Python library for interpreting HuggingFace language models . it provides attribution methods and concept-based explanations . the library welcomes issues and pull requests . |
Copied to clipboard
| Challenge: | **Copyright Detective** is the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. |
| Approach: | They propose a system that detects copyright infringements and visualizes them . they use content recall testing, paraphrase-level similarity analysis and persuasive jailbreak probing . |
| Outcome: | The proposed system detects, analyzes, and visualizes potential copyright risks in LLM outputs. |
Copied to clipboard
| Challenge: | Multi-agent LLM workflows are notoriously difficult to debug and refine. |
| Approach: | They propose a unified UI that closes the loop for offline, test-case–driven improvement of multi-agent LLM workflows. |
| Outcome: | PROTEA performs backward node evaluation and proposes a targeted prompt patch as an editable diff. |
Copied to clipboard
| Challenge: | Existing methods for constructing character relationships from plain text are time-consuming and low in coverage. |
| Approach: | They propose a human-in-the-loop framework that combines LLM-based extraction with symbolic reasoning. |
| Outcome: | The proposed framework improves annotation accuracy and consistency while significantly reducing time cost. |
Copied to clipboard
| Challenge: | Existing tree search methods are task-specific and require substantial reimplementation effort when adapting to new domains. |
| Approach: | They propose a Python framework for LLM reasoning via tree search that decomposes tree search into three reusable components that plug into algorithms like MCTS and BFS. |
| Outcome: | The proposed framework decomposes tree search into three reusable components that plug into algorithms like MCTS and BFS. |
Copied to clipboard
| Challenge: | Existing automated systems for scientific illustrations are limited in editability, stylistic controllability, and efficiency. |
| Approach: | They propose an end-to-end system that generates fully editable scientific illustrations from long-form scientific text while enabling flexible style adaptation through user-provided reference images. |
| Outcome: | The proposed system generates fully editable scientific illustrations from long-form scientific texts while enabling flexible style adaptation through user-provided reference images. |
Copied to clipboard
| Challenge: | Existing trading systems rely on fragmented and task-specific APIs, resulting in inconsistent schemas and limited reproducibility. |
| Approach: | They propose a unified trading environment for large language model (LLM) agents that standardizes three core capabilities . they argue that such a standardized trading environment is essential for scalable research on LLM-based financial agents. |
| Outcome: | The proposed trading environment reduces engineering overhead and supports reproducible evaluation through comprehensive logging and deterministic replay. |
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
Copied to clipboard
| Challenge: | a library for evaluating membership inference attacks against large language models (LLMs) has emerged as a crucial technique for auditing privacy risks and copyright infringement in LLMs. |
| Approach: | They propose a Python library for efficiently evaluating membership inference attacks against large language models (LLMs) they use a high-throughput batch inference via vLLM and a cross-method caching architecture that computes intermediate results once and shares them across methods. |
| Outcome: | The proposed library performs a 5 speedup in inference and a cross-method caching architecture that computes intermediate results once and shares them across methods. |
Copied to clipboard
| Challenge: | Automated Scientific Discovery (ASD) systems rely on parametric knowledge to generate and run code-based experiments. |
| Approach: | They propose a system that distills large collections of scientific Github repositories into a vetted library of working domain-specific code examples. |
| Outcome: | The proposed system produces more accurate, complete, and scientifically sound experiments than an agent with only general materials-science code examples. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are inadequate for extracting structured insights from unstructured documents. |
| Approach: | They propose a framework enabling agentic AI for end-to-end document intelligence with four key components: DocSplit, configurable Extraction Module, and Rule Validation Module. |
| Outcome: | The proposed framework achieves 98% classification accuracy, 80% reduced processing latency, and 77% lower operational costs over legacy baselines. |
Copied to clipboard
| Challenge: | Existing systems rely on proprietary LLMs, which compromise reproducibility and data security. |
| Approach: | They propose a platform that combines controlled question administration of survey software with the flexibility of LLMs. |
| Outcome: | AInterviewer combines controlled question administration of survey software with flexibility of LLMs. |
Copied to clipboard
| Challenge: | Existing systems that generate publication-ready forest plots from biomedical papers are fragmented and time-consuming. |
| Approach: | They propose a system that generates publication-ready forest plots directly from biomedical papers . autoforest automatically suggests ICO elements, extracts outcome data and performs statistical synthesis . authors demonstrate how the system can accelerate evidence synthesis and lower the barrier to conducting meta-analyses . |
| Outcome: | The proposed system accelerates evidence synthesis and lowers the barrier to meta-analyses. |
Copied to clipboard
| Challenge: | Dash-M5H integrates transcript text, audio, and facial behavior with a clinically grounded VLM prediction pipeline that produces DSM-5-aligned depression predictions. |
| Approach: | They propose a dashboard that integrates multimodal behavioral data with multi-model signal outputs of recorded clinical interviews. |
| Outcome: | Dash-M5H is an interactive dashboard for *multi-modal, multi-model mental health assessment that integrates transcript text, audio, and facial behavior with a clinically grounded VLM prediction pipeline. |
Copied to clipboard
| Challenge: | MixtureKit is a modular open-source framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned checkpoints. |
| Approach: | They propose a modular open-source framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned checkpoints. |
| Outcome: | Experiments on multilingual code-switched (Arabic–Latin) show that BTX models built with MixtureKit outperform dense baselines across multiple benchmarks. |
Copied to clipboard
| Challenge: | Existing methods are designed for specific settings, each with its own set of challenges. |
| Approach: | They propose a unified, modular, and extensive Text-to-SQL framework . it proposes a universal execution paradigm and a multi-actor collaboration mechanism . |
| Outcome: | Squrve proposes a unified, modular, and extensive Text-to-SQL framework . the framework outperforms existing methods on widely adopted benchmarks . |
Copied to clipboard
| Challenge: | Recent advances in large language model (LLM) agents have accelerated deployment of multi-agent systems for complex tasks. |
| Approach: | They propose an open-source toolkit for instantiating, probing, and measuring emergent risks in LLM-based multi-agent systems under controlled conditions. |
| Outcome: | The proposed toolkit is based on a structured topology–environment–protocol–agent–task quintuple enabling reproducible studies of how communication structure, coordination mechanisms, and incentives shape system-level risks. |
Copied to clipboard
| Challenge: | Existing frameworks for large language model (LLM) inference on CPUs overlook overhead of cross-NUMA memory access. |
| Approach: | They propose a lightweight LLM inference architecture designed from the ground up for many-core CPUs. |
| Outcome: | Experimental results show that ArcLight surpasses the performance ceiling of mainstream frameworks, achieving up to 46% higher inference throughput. |
Copied to clipboard
| Challenge: | Existing tools do not surface subtler psychosocial harms, nor provide explainable rationales that practitioners need. |
| Approach: | They propose an open-source system that lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
| Outcome: | The open-source DialogGuard system lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
Copied to clipboard
| Challenge: | In 2014 there were 2.3B SNOMED CT codes recorded in English healthcare practices. By 2024, this number had grown almost threefold to 6.1B codes: approximately 100 codes per person each year. |
| Approach: | They introduce a system for translating natural language population health questions into executable database queries using agents with access to both local and external knowledge bases. |
| Outcome: | The proposed system is able to handle a range of health informatics tasks on three datasets and via a beta-testing phase. |
Copied to clipboard
| Challenge: | Rankify unifies retrieval-augmented generation (RAG) and retrieval based question answering systems. |
| Approach: | They propose an open-source Python toolkit that unifies retrieval-augmented generation in a single modular framework. |
| Outcome: | The proposed framework unifies retrieval-augmented generation (RAG) tools in a single modular framework. |
Copied to clipboard
| Challenge: | a new approach to natural-language research questions requires manual effort to generate an annotation schema and label the corpus. |
| Approach: | They propose a natural-language search tool that takes a question and a corpus to produce a schema and db with a web interface that lets steer and revise the extraction. |
| Outcome: | The proposed model yields outputs that support real-world analysis in law and computational biology. |
Copied to clipboard
| Challenge: | Efforts over the past three decades have produced web archives containing billions of webpage snapshots and petabytes of data. |
| Approach: | They propose a public search system that supports multimodal searches across 10,015,993 federal government PDFs from the 2020 End of Term crawl. |
| Outcome: | The proposed system supports multimodal searches across 10,015,993 federal government PDFs from the 2020 End of Term crawl (70,958,487 total PDF pages) significant compute cost for GovScape’s pre-processing pipeline for 10 million PDFs was approximately 1,500, equivalent to 47,000 PDF pages per dollar spent on compute. |
Copied to clipboard
| Challenge: | Linux kernel device drivers are tightly coupled with hardware, making them difficult to execute and test without physical devices. |
| Approach: | They present a tool that generates QEMU-based virtual devices directly from Linux driver source code. |
| Outcome: | The proposed tool generates QEMU-based virtual devices directly from Linux driver source code. |
Copied to clipboard
| Challenge: | Existing document translation pipelines face a tension between linguistic processing and layout preservation. |
| Approach: | They propose a framework for layout-preserving PDF translation that decouples visual layout metadata from semantic content. |
| Outcome: | The proposed framework improves layout fidelity, visual aesthetics, and terminology consistency over representative baselines while maintaining competitive translation precision. |
Copied to clipboard
| Challenge: | Existing automated ICD coding systems face several fundamental challenges due to the limited availability of publicly available Chinese ICD datasets. |
| Approach: | They propose to use a Chinese ICD coding dataset and a multi-agent framework to reformulate ICD as a joint disease-procedure coding task. |
| Outcome: | The proposed system outperforms state-of-the-art methods on real-world Chinese ICD coding datasets and 1.7B-parameter models. |
Copied to clipboard
| Challenge: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Approach: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Outcome: | Hindsight is a working memory system for AI agents that separates facts from beliefs . the system outperforms existing models on LongMemEval and LoCoMo with 83.6% accuracy . |
Copied to clipboard
| Challenge: | Early ConvAI agents rely on an in-context approach that appends the growing conversation history to the model input, but this approach scales poorly under context-window limits. |
| Approach: | They propose a tree-structured memory module to access and update structured conversational memory. |
| Outcome: | The proposed system improves over flat-RAG baselines while using only 9.1% of the tokens required by in-context memory. |
Copied to clipboard
| Challenge: | Scientific AI agents can perform complex research tasks, but these unfolded workflows are difficult for humans to inspect and review, limiting interpretable, controllable and effective human–AI collaboration. |
| Approach: | They propose a monitoring and visualization framework that records fine-grained execution events and organizes them into a directed graph that makes agent workflows explicit as they proceed. |
| Outcome: | The proposed framework records intermediate steps (e.g. tool calls and code executions) and renders them as real-time updated visual traces that expose workflow structure. |
Copied to clipboard
| Challenge: | Existing code sandboxes fail to provide accurate verification and efficiency under high-concurrency workloads. |
| Approach: | They propose a high-fidelity code verification system that provides sandbox feedback for RL training and evaluation. |
| Outcome: | The proposed system outperforms heuristic-matching baselines on LiveCodeBench and training stability on high-concurrency workloads. |
Copied to clipboard
| Challenge: | Existing synthetic data tools are limited by convoluted workflows, fragmented data standards, and limited scalability across modalities. |
| Approach: | They develop an open-source framework that aims to reduce the technical barrier to synthetic data generation and subsequent model training. |
| Outcome: | The proposed framework achieves an optimal balance between generation efficiency and data quality. |
Copied to clipboard
| Challenge: | Mobile input method editors (IMEs) are the primary interface for text input, yet they are constrained to manual typing and struggle to produce personalized text. |
| Approach: | They propose a personalized on-device IME powered by large language models . they endow HUOZIIME with initial human-like prediction ability . |
| Outcome: | The proposed IME has initial human-like prediction ability and is optimized for on-device deployment. |
Copied to clipboard
| Challenge: | Existing systems assume wideband audio and client-side acoustic echo cancellation (AEC) . a system that enables bidirectional speech translation over the PSTN requires app installation or carrier integration. |
| Approach: | They propose a server-side relay system that enables bidirectional LLM-based speech translation over ordinary telephone calls without app installation or carrier integration. |
| Outcome: | The proposed system is deployed over the public switched telephone network (PSTN) it observes 555 ms median latency, 2,684 mS median translation loops, and COMET semantic adequacy of 0.71 (enko) and 0.62 (koen) |
Copied to clipboard
| Challenge: | MASEval provides a framework-agnostic, system-level comparison across any agent framework and benchmark. |
| Approach: | They propose a Python library that treats the entire agentic system as the unit of analysis. |
| Outcome: | The proposed framework treats the entire agentic system as the unit of analysis. |
Copied to clipboard
| Challenge: | Large language model-based multi-agent systems (MAS) are increasingly used to extend agentic problem solving via role specialization and collaboration. |
| Approach: | They propose a graph-centric framework for orchestrating large language model-based multi-agent systems . they compile a user's natural-language intent into an editable workflow specification and then into an executable graph . |
| Outcome: | The proposed framework compiles natural-language intent into an executable graph and then compile and executes it at runtime. |
Copied to clipboard
| Challenge: | Existing tool-augmented verification systems depend on opaque search APIs, introducing uncontrolled variability into factuality evaluation. |
| Approach: | They propose a reproducibility-oriented agentic fact search system for claim-level factuality verification built on a locally aggregated open-source search infrastructure. |
| Outcome: | The proposed system decomposes model outputs into atomic factual claims, generates targeted search queries, retrieves supporting evidence via a self-hosted meta-search engine, and performs modular verification within a fully configurable pipeline. |
Copied to clipboard
| Challenge: | Annotated data is still central to NLP and Generative AI, yet the demands on annotation have grown in both scale and complexity. |
| Approach: | They introduce Potato 2.0, an open source annotation platform for easy deployment and customization. |
| Outcome: | The new version of potato supports 39 different types of annotation tasks and multiple AI-assistance features. |
Copied to clipboard
| Challenge: | mllm-shap is an open-source Python platform for researchers and ML practitioners that extends Shapley value (SV) explainability from text-only large languagemodels to multimodal LLMs that process both text and audio. |
| Approach: | They present a framework that extends Shapley value (SV) explainability from text-only large languagemodels to multimodal LLMs thatjointly process text and audio. |
| Outcome: | mllm-shap is the first publicly available framework for complete, reproducible SV-based explainability of text-audioMLLMs. |
Copied to clipboard
| Challenge: | Emerging AI-powered writing assistants focus on grammar fixes or simulating peer review with final scores, yet they fall short of providing concrete, actionable suggestions that help students improve their papers during drafting. |
| Approach: | They propose a human-centered writing assistant system that delivers actionable suggestions as Overleaf-native inline comments while leaving the actual writing entirely to human authors. |
| Outcome: | The proposed system outperforms a baseline with the skill library and provides actionable suggestions while leaving the actual writing to human authors. |
Copied to clipboard
| Challenge: | Existing large language models encode workflow progress as conversational state and rely on cloud-centric execution, which hinders traceability and auditability. |
| Approach: | Gardener is an open-source desktop application for macOS and windows under the Apache License 2.0. |
| Outcome: | Gardener is released as an open-source desktop application for macOS and Windows under the Apache License 2.0. |
Copied to clipboard
| Challenge: | Tokenization quality varies significantly across languages, leading to disparities in LLM performance and cost for speakers of less-resourced languages. |
| Approach: | They propose a Python-based evaluation framework and a JavaScript visualization interface that evaluates tokenizers in a variety of languages. |
| Outcome: | TokCollate is an evaluation framework for tokenizers with a JavaScript visualization interface. |
Copied to clipboard
| Challenge: | Recent work in social media platforms delegate content moderation decisions to users and communities. |
| Approach: | They propose a software system for the dynamic monitoring of Reddit posts, communities, and moderation actions to enable scalable and reproducible research on decentralized platform governance and content moderation. |
| Outcome: | The proposed system is the only available solution for general-purpose, real-time, policy-compliant longitudinal data collection on Reddit. |
Copied to clipboard
| Challenge: | The AI Steerability 360 toolkit is an extensible, open-source Python library for steering LLMs. |
| Approach: | The AI Steerability 360 toolkit is an extensible, open-source Python library for steering LLMs. |
| Outcome: | The toolkit is available under an Apache 2.0 license and is available on https://github.com/IBM/AISteer360. |
Copied to clipboard
| Challenge: | Existing systems that use keyword-based media monitoring miss semantically relevant articles and are expensive at scale. |
| Approach: | They propose a multi-agent media monitoring system that processes streaming articles through three stages: article matching, batched feature extraction, and report generation with deterministic deduplication and density-based clustering. |
| Outcome: | The proposed system reduces agent invocations by 20% and reduces core feature extraction calls from 7 to 2 per article - a 71% reduction - with bounded quality tradeoffs . |
Copied to clipboard
| Challenge: | a non-linear reading order of academic literature is recognized by authors who make explicit connections between non-adjacent passages. |
| Approach: | They propose an enhanced reading experience which generates questions and searches for answer-bearing passages in academic papers to form intra-document connections when answers are found. |
| Outcome: | The proposed interface makes connections between related but non-adjacent passages even if the author did not make them explicit. |
Copied to clipboard
| Challenge: | a new platform addresses five pain points in legal research in Thailand . the tools available to legal practitioners are fragmented and lack a unified tool for cross-referencing, version tracking or structural navigation. |
| Approach: | They propose a platform that addresses five practitioner pain points through three modules built on a temporal legal knowledge graph covering 552K nodes and 6.3M edges. |
| Outcome: | The proposed platform addresses five practitioner pain points through three modules built on a temporal legal knowledge graph covering 552K nodes and 6.3M edges. |
Copied to clipboard
| Challenge: | Existing LLM–robotic systems are tightly intertwined, making it difficult to switch hardware, add extra capabilities, or expand to multiple devices without bespoke integration. |
| Approach: | They propose an open-source agentic operating system for embodied LLMs . rosaOS integrates agentic tool-calling and ROS for robot interactions . |
| Outcome: | The proposed system integrates with the Reachy Mini robot and supports a multi-device setup with a quadruped robot, wheeled mobile robot, and smart lamp. |
Copied to clipboard
| Challenge: | a recent study found that social media misinformation is reactive and claim-specific, and can degrade under temporal and cross-lingual/domain shift. |
| Approach: | They present a mobile media-literacy app that builds digital literacy skills through short, interactive challenges with instant feedback. |
| Outcome: | The app is the first multilingual and modular platform to improve digital literacy skills. |
Copied to clipboard
| Challenge: | MED-COPILOT is an interactive research prototype for evidence-aware clinical reasoning . large language models (LLMs) are prone to hallucinations and lack verifiable evidence grounding . |
| Approach: | They propose a system that integrates GraphRAG and semantic-keyword similar-patient retrieval to support transparent clinical reasoning. |
| Outcome: | The proposed system outperforms baseline and standard RAGs on clinical note completion and medical question answering. |
Copied to clipboard
| Challenge: | Large language models (LLMs) enable scalable content generation for personalized learning, but reliability and pedagogical alignment remain open challenges. |
| Approach: | They propose a web-based system that integrates expert-validated assessment, retrieval-augmented generation (RAG), and an LLM-as-a-Judge validation loop within a closed instructional pipeline. |
| Outcome: | The proposed system achieves a gain of 37.9 percentage points and a large effect size in a real-world deployment with 179 registered users. |
Copied to clipboard
| Challenge: | AutoChecklist is an open-source library that unifies checklist-based evaluation into composable pipelines. |
| Approach: | They propose an open-source library that unifies checklist-based evaluation into composable pipelines. |
| Outcome: | The open-source library unifies checklist-based evaluation into composable pipelines. |
Copied to clipboard
| Challenge: | Recruiters and job seekers rely on search systems to navigate labor markets . many systems fail to handle skill synonyms and nonlinear careers . |
| Approach: | They propose a production-ready system that integrates Transformer embeddings, skill knowledge graphs, and interpretable reranking. |
| Outcome: | The proposed system optimizes utility across skill fit, experience, location, salary, and company preferences. |
Copied to clipboard
| Challenge: | Existing Arabic writing technologies primarily use a single quality score for essays, but there is limited support for Arabic AES. |
| Approach: | They propose a Web-based platform that integrates Arabic AES workflows with a user-friendly interface. |
| Outcome: | The proposed system integrates with existing Arabic scoring systems and provides a user-friendly interface. |
Copied to clipboard
| Challenge: | Existing claim verification pipelines operate over text, producing ungrounded judgments. |
| Approach: | They propose a physics-grounded material claim verification system that can be used to verify claims with physical evidence. |
| Outcome: | The proposed system reduces MAE and sign-offs with experts over text-only GPT-5.1. |
Copied to clipboard
| Challenge: | pAtChWoRK corrects manual classification errors and extracts complex unstructured fields such as award and solvency criteria and tenders’ objectives. |
| Approach: | pAtChWoRK corrects manual classification errors and extracts complex unstructured fields such as award and solvency criteria and tenders’ objectives. |
| Outcome: | pAtChWoRK corrects manual classification errors and extracts complex unstructured fields such as award and solvency criteria and tenders’ objectives. |
Copied to clipboard
| Challenge: | Existing evaluation frameworks for audio foundation models are heavily reliant on English, making it difficult to objectively assess models’ performance on Chinese. |
| Approach: | They propose a unified framework that supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards. |
| Outcome: | The proposed framework supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards. |
Copied to clipboard
| Challenge: | Current approaches to creating layout-aware, interactive academic webpages are limited. |
| Approach: | They propose a benchmark dataset and evaluation framework for assessing academic webpage generation that incorporates rule-based metrics like Connectivity, Completeness and PaperQuiz. |
| Outcome: | The proposed framework outperforms baselines like template-based webpages and arXiv/alphaXivo versions while maintaining low cost. |
Copied to clipboard
| Challenge: | Existing MAS frameworks lack standardized abstractions, leading to low efficiency and repetitive implementation of core functions. |
| Approach: | They propose an open-source framework that encapsulates agents, tools, and reasoning flows as pluggable atomic components. |
| Outcome: | The OxyGent framework provides a robust and scalable foundation for multi-agent systems in industrial environments. |
Copied to clipboard
| Challenge: | Existing annotation tools do not support team-based workflows or access to instructional context . |
| Approach: | They present an open-source web platform for annotating classroom conversation transcripts . they argue that existing annotation tools do not provide access to instructional context . |
| Outcome: | EduCoder is an open-source web platform designed for annotating classroom conversation transcripts. |
Copied to clipboard
| Challenge: | Clinical procedure coding is an extreme multi-label classification problem . CPTCoder predicts standardized medical procedure codes from clinical text . |
| Approach: | a new human-in-the-loop system predicts standardized medical procedure codes from clinical text. |
| Outcome: | CPTCoder outperforms baseline system by 12 and 5 points in a clinical procedure classification problem. |
Copied to clipboard
| Challenge: | Recent advances in speech-to-text technologies and large language models (LLMs) have the potential to overcome these limitations with automated, real-time jargon support. |
| Approach: | They built an interactive LLM-powered system that provides real-time personalized jargon support tailored to users’ individual backgrounds in online meetings. |
| Outcome: | The proposed system provides more precise jargon identification and enhanced participants’ comprehension, engagement, and appreciation of colleagues’ work. |
Copied to clipboard
| Challenge: | olmOCR is a fully open OCR system developed through iterative public releases and community feedback. |
| Approach: | They propose an open OCR system that combines a 7B vision-language model trained in two stages: finetuning and reinforcement learning with visual unit tests. |
| Outcome: | The proposed system achieves state-of-the-art performance among open systems and proprietary APIs at a fraction of the cost. |
Copied to clipboard
| Challenge: | Recent advances in artificial intelligence (AI) have accelerated the growth of both human-authored and AI-generated research outputs. |
| Approach: | They propose an AI-driven open-access platform built on open preprints, AI-augmented analysis and review, and reader feedback. |
| Outcome: | The proposed platform supports human scientists through an interactive UI and AI scientists through Model Context Protocol (MCP)-based interactions. |
Copied to clipboard
| Challenge: | Existing argumentation tools analyze properties of the argument, but leave perspective-specific background knowledge implicit. |
| Approach: | They propose a system that analyzes arguments from multiple perspectives using a perspective-dependent analysis approach. |
| Outcome: | The proposed system analyzes arguments from multiple perspectives and generates interpretations, value conflicts and assumption gaps. |
Copied to clipboard
| Challenge: | Existing NLP tools are fragmented, closed-source, or difficult to use . a single sentence can convey emotion, social dynamics, cognitive states, and implicit attitudes . |
| Approach: | They propose an open-source python TOolkit for NLP in clinical psychology. |
| Outcome: | The TOolkit bridges traditional psycholinguistic analysis and modern NLP . it integrates interpretable lexical features with state-of-the-art lightweight transformer models . the toolkit is released under an open-source license and is evaluated through multiple MH–related datasets. |
Copied to clipboard
| Challenge: | Formal verification typically requires developers to write detailed formal specifications . a formal verification system that generates candidate specifications is costly and error-prone . |
| Approach: | They propose an LLM-driven neuro-symbolic demonstration system that reframes specification writing as constrained structured synthesis. |
| Outcome: | The proposed system reduces hallucinations and produces proof-ready annotations. |
Copied to clipboard
| Challenge: | Social media platforms are a popular way to communicate with medical experts and improve health literacy. |
| Approach: | They introduce a semi-automated annotation framework for medical texts in low-resource languages . they use large language models for pre-annotation and human validation to support efficient annotation . |
| Outcome: | The proposed framework is applied to medical social media texts in Bahasa Indonesia . it yields higher inter-annotator agreement and human review improves output . future work focuses on mitigating pre-annotation bias and reducing annotation overhead . |
Copied to clipboard
| Challenge: | Deployed in a university software engineering course, RECAP captured 2,034 prompts and 8,239 code edits from 41 students across a multi-week project. |
| Approach: | They propose an open-source platform that passively records AI chat sessions and fine-grained code edits inside VS Code without disrupting the developer’s workflow. |
| Outcome: | The open-source platform captures 2,034 prompts and 8,239 code edits from 41 students across a multi-week project. |
Copied to clipboard
| Challenge: | High-quality information extractions often require domain-specific accuracy, up-to-date understanding of specialized taxonomies, and the ability to incorporate emerging jargon and rare outliers. |
| Approach: | They propose a Dynamic Self-Evolving Extraction and Curation Toolkit which continuously improves as it is used to extract structured information from raw text. |
| Outcome: | The proposed toolkit continuously improves as it is used in medical, legal, and HR domains. |
Copied to clipboard
| Challenge: | Existing TTC scaling strategies and reasoning scorers are fragmented and evaluated under inconsistent protocols. |
| Approach: | They propose a framework for seamless test-time compute scaling of large language model reasoning . they use a modular Python library to implement state-of-the-art scaling strategy and scorer families . |
| Outcome: | The proposed framework evaluates performance and computational efficiency on mathematical and coding tasks. |
Copied to clipboard
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
Copied to clipboard
| Challenge: | Expert pruning is a practical deployment technique for Mixture-of-Experts models . but its success depends heavily on the calibration set used for pruning . |
| Approach: | They propose a calibration tool that compares expert activations across datasets to predict calibration perturbations without running expensive pruning procedures. |
| Outcome: | The proposed system compares expert activations across datasets to predict calibration perturbations without running expensive pruning procedures. |
Copied to clipboard
| Challenge: | Existing post-training pipelines that generate QA pairs require costly expert annotation and synthetic data that drops evidence structure. |
| Approach: | They propose a system that converts raw biomedical papers into evidence-enriched training sets and a domain-specialized VLM. |
| Outcome: | Ryze synthesizes QA pairs with complete supporting evidence, reduces layout and OCR errors . the system outperforms the base model on LAB-Bench and surpasses GPT-5.2 by +3.8%. |
Copied to clipboard
| Challenge: | Agentic systems are becoming more capable of defining strategies, taking actions, and solving complex, multi-step tasks. |
| Approach: | They propose an automatic, dynamic, and easy-to-use evaluation framework that provides textual insights into agent behavior on three levels of granularity: system, trace, and node. |
| Outcome: | The proposed framework produces high-quality, data-driven, insightful feedback on system, trace, and node. |
Copied to clipboard
| Challenge: | Clinical trial databases are central to modern oncology research and drug development. |
| Approach: | FD-NL2SQL is a schema-aware clinical NL2sql assistant for SQLite-based oncology databases . it decomposes a natural-language question into predicate-level sub-questions and synthesizes executable SQL . |
| Outcome: | FD-NL2SQL synthesizes SQL based on decomposition, retrieved exemplars, and schema . clinical trial databases are central to modern oncology research and drug development . |
Copied to clipboard
| Challenge: | Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer. |
| Approach: | They propose a diagram question-answer review framework that decouples interface logic from dataset-specific JSON structures through an internal meta-schema and dataset adapters. |
| Outcome: | The proposed framework achieves 85.39% precision and 75.30% recall across six diagram QA datasets. |
Copied to clipboard
| Challenge: | Effective communication is a core objective of graduate education in AI and machine learning (ML). |
| Approach: | They propose an evaluation agent that assesses slide decks against a framework of expert-defined criteria using a visual language model. |
| Outcome: | The evaluation agent detects the majority of expert-identified issues on a dataset of 150 annotated slide decks and shows strong results on structural and visual criteria and known limitations on subjective dimensions such as research quality. |
Copied to clipboard
| Challenge: | Existing interpretability tools for visionlanguage models are limited to activation probing and saving. |
| Approach: | They propose a library specifically designed for mechanistic interpretability of visionlanguage models that provides unified abstractions for activation patching, attention pattern analysis, and meta-functions across diverse VLM architectures. |
| Outcome: | The proposed library handles architecture-specific complexities while maintaining a simple, high-level interface. |
Copied to clipboard
| Challenge: | Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs. |
| Approach: | They propose a human-agent collaborative system that generates interactive educational documents from a single topic input. |
| Outcome: | The proposed system generates documents comparable in quality to human-authored ones. |
Copied to clipboard
| Challenge: | High-quality time-aligned annotation is fundamental to speech processing and animal vocalization research, yet precise boundary localization and consistent labeling remain challenging in collaborative settings. |
| Approach: | They propose a web-based multimedia annotation system for collaborative, video-informed, and AI-assisted timeline labeling of audio and video data. |
| Outcome: | The proposed system improves time-aligned labeling and accuracy in speech and animal vocalization annotations. |
Copied to clipboard
| Challenge: | Existing frameworks for building LLM-based agents treat agent behavior as static-knowledge gained during execution is not preserved for future use. |
| Approach: | They propose a new paradigm that preserves successful task solutions as executable subagent code rather than textual experience. |
| Outcome: | The proposed agent-based agent-driven paradigm preserves successful tasks as executable subagent code rather than textual experience. |
Copied to clipboard
| Challenge: | OpenGlass is an open-source, privacy-oriented, local-first system for low-latency multimodal visual assistance . cloud MLLM assistants offer strong visual understanding but often require uploading first-person visual data . |
| Approach: | They propose an open-source system for low-latency multimodal visual assistance . they use an ESP32-based glasses-side unit to capture visual context . |
| Outcome: | The proposed system captures visual context while a nearby device performs local MLLM inference and speech output. |
Copied to clipboard
| Challenge: | Academic advising is a critical yet resourceintensive component of higher education . monolithic model must simultaneously maintain awareness of heterogeneous institutional constraints . |
| Approach: | They propose a multi-agent academic advising system that handles the full complexity spectrum of student queries. |
| Outcome: | The proposed system handles the full complexity spectrum of student queries . it also provides a real-time transparency interface streaming agent reasoning and negotiation rounds to users . the system is released open-source and has been rated highly by users based on their results . |
Copied to clipboard
| Challenge: | GameDAI is a hierarchical multi-agent framework that transforms instructor-provided questions into fully playable, pedagogically grounded educational games validated through formal mechanic contracts. |
| Approach: | They propose a hierarchical multi-agent framework that transforms instructor-provided questions into fully playable, pedagogically grounded educational games validated through formal mechanic contracts. |
| Outcome: | The proposed framework achieves 90% validation pass rate, 98.3% schema compliance, and 73% token reduction over ReAct agents at 0.46 per game. |
Copied to clipboard
| Challenge: | Existing approaches to scientific hypothesis discovery operate autonomously with little to no human guidance. |
| Approach: | They propose a framework that empowers scientists to steer the generative process via explicit signals. |
| Outcome: | The proposed framework outperforms existing models in terms of quality and accuracy of the input signals. |