Papers by Iacer Calixto
What Does Infect Mean to Cardio? Investigating the Role of Clinical Specialty Data in Medical LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | S-MedQA is an English question-answering dataset designed for benchmarking large language models in fine-grained clinical specialties. |
| Approach: | They propose to use an English medical question-answering dataset to benchmark large language models in clinical specialties. |
| Outcome: | The proposed dataset is designed to benchmark large language models in medical specialties. |
Are Scene Graphs Good Enough to Improve Image Captioning? (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing image captioning models rely on object detection features to generate image descriptions, but they are noisy. |
| Approach: | They propose to use scene graphs to introduce information about object relations into captioning to improve image descriptions. |
| Outcome: | The proposed model improves image caption quality by 3.3 CIDEr compared to a strong Bottom-Up Top-Down baseline. |
Mind the Gap: Benchmarking LLM Uncertainty and Calibration with Specialty-Aware Clinical QA and Reasoning-Based Behavioural Features (2026.eacl-long)
Copied to clipboard
| Challenge: | Reliable uncertainty quantification (UQ) is essential when employing large language models in high-risk domains such as clinical question answering (QA). |
| Approach: | They evaluate uncertainty estimation methods for clinical question answering using eleven clinical specialties and six question types. |
| Outcome: | The proposed method is based on behavioral features derived from reasoning-oriented models and examines conformal prediction as a complementary set-based approach. |
LLM aided semi-supervision for efficient Extractive Dialog Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a method to extract dialog summarization data from unlabeled data is currently expensive to build. |
| Approach: | They propose a method to extract dialog summarization using unlabeled data . they frame summarizing as a question-answering problem and use pseudo-labels to fine-tune a chat summarisation model . |
| Outcome: | The proposed method achieves 65.9/57.0/61.0 ROUGE-1/-2/-L on a TWEETSUMM dataset compared with current state-of-the-art methods on the entire training dataset. |
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena (2022.acl-long)
Copied to clipboard
| Challenge: | Current models have difficulty addressing most phenomena, despite performance gains on many V&L tasks. |
| Approach: | They propose a benchmark for testing general-purpose pretrained vision and language models for their visio-linguistic grounding capabilities on specific linguistic phenomena. |
| Outcome: | The proposed model can be used to evaluate models on linguistic phenomena in the visual modality. |
Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks (2021.naacl-main)
Copied to clipboard
| Challenge: | Masked language models have become the de facto standard when processing text . however, these models are evaluated in a monolingual setting only . |
| Approach: | They propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap between different languages. |
| Outcome: | The proposed approach bridges the gap between word representations and knowledge graphs by using a shared vocabulary of entities. |
DeVisE: Towards the Behavioral Testing of Medical Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing evaluations of large language models do not reveal whether their outputs reflect genuine medical reasoning or superficial correlations. |
| Approach: | They propose a framework that probes fine-grained clinical understanding through controlled counterfactuals. |
| Outcome: | The proposed framework is based on demographic and vital signs data from the ICU discharge notes of patients in the intensive care unit (MIMIC-IV). |
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too (2020.aacl-main)
Copied to clipboard
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, Samuel R. Bowman
| Challenge: | a study of intermediate-task training in monolingual English shows that it improves model performance on non-English language understanding tasks. |
| Approach: | They evaluate whether English intermediate-task training is still helpful on non-English target tasks . BUCC and Tatoeba sentence retrieval tasks see large improvements . |
| Outcome: | The proposed model outperforms existing models on non-English language understanding tasks. |
Latent Variable Model for Multi-modal Translation (P19-1)
Copied to clipboard
| Challenge: | Libovick and Helcl (2017) show improvements due to imposing a constraint on the KL term to promote models with non-negligible mutual information between inputs and latent variable and training on additional target-language image descriptions. |
| Approach: | They propose to model interaction between visual and textual features for multi-modal neural machine translation (MMT) using a latent variable model. |
| Outcome: | The proposed model improves over baselines including a multi-task learning approach and a conditional variational auto-encoder approach. |
Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA (2026.acl-short)
Copied to clipboard
| Challenge: | a growing body of research suggests that social descriptors can influence LLM-generated clinical recommendations. |
| Approach: | They examine whether social descriptors of a patient distort uncertainty signals and model accuracy. |
| Outcome: | The presence of social identity cues affects the reliability of confidence signals, the authors show . incorporating sociodemographic attributes alters outputs in clinical trial matching and QA . |