Papers by Bernard Ghanem
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
AVA: Attentive VLM Agent for Mastering StarCraft II (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing StarCraft II benchmarks rely on abstract state representations that deviate from human perception . Existing systems rely only on abstract representations, creating an artificial gap between how humans process battlefield information and limiting ecological validity of learned behaviors. |
| Approach: | They introduce AVACraft, the first multimodal benchmark environment for complex decision-making in StarCraft II. |
| Outcome: | The AVACraft benchmark supports both traditional and modern multi-agent reinforcement learning paradigms. |
Relation-aware Video Reading Comprehension for Temporal Language Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for temporal language grounding in videos are boundary regression and span extraction tasks. |
| Approach: | They propose a Relation-aware Network to localize a temporal span relevant to a given query sentence. |
| Outcome: | The proposed framework selects a video moment choice from the predefined answer set with the aid of coarse-and-fine choice-query interaction and choice-choice relation construction. |
Multimodal Safety Evaluation in Generative Agent Social Simulations (2026.acl-long)
Copied to clipboard
Alhim Adonai Vera Gonzalez, Carlos Hinojosa, Karen Sanchez, Haidar Bin Hamid, Donghoon Kim, Bernard Ghanem
| Challenge: | Recent advances in large language models have enabled generative agents that simulate be-like behavior through natural language interactions. |
| Approach: | They propose a reproducible simulation framework to evaluate generative agents in multimodal scenarios . they use metrics that quantify plan revisions and unsafe-to-safe conversions to evaluate their effectiveness . |
| Outcome: | The proposed framework evaluates generative agents in three aspects: safety improvement over time, detection of unsafe activities across social contexts, social dynamics and acceptance rates. |
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for merging large language models often overlook safety alignment during merging, leading to misaligned models. |
| Approach: | They propose to combine safety and domain-specific data to optimize model merging techniques . they propose to use this data to maximize model alignment . |
| Outcome: | The proposed method allows for models that excel in both domain expertise and alignment. |
MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Metadata extraction relies heavily on manual annotation of documents. |
| Approach: | They propose a framework that leverages Large Language Models to automatically extract metadata attributes from scientific papers covering datasets of languages other than Arabic. |
| Outcome: | The proposed framework automates the extraction of metadata attributes from Arabic scientific papers using large language models. |