Papers by Bruno Martins
Retrieval-augmented Image Captioning (2023.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that image captioning can be effectively formulated from this new perspective. |
| Approach: | They propose a pretrained visual and language decoders for image captioning that generate sentences from the input image and a set of captions retrieved from a datastore. |
| Outcome: | The proposed model generates sentences given the input image and retrieved captions, while the decoder attends to the multimodal encoder representations. |
Accurate and Well-Calibrated ICD Code Assignment Through Attention Over Diverse Label Embeddings (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to assigning ICD codes to clinical text are time-consuming, labor intensive, and error-prone. |
| Approach: | They propose to adapt a Transformer-based model to a longformer model and use it to encode clinical narratives. |
| Outcome: | The proposed approach outperforms current state-of-the-art models in ICD coding with the label embeddings contributing to the good performance. |
Leveraging LLMs to Streamline the Review of Public Funding Applications (2025.emnlp-industry)
Copied to clipboard
João DS Marques, Andre Vicente Duarte, André Mendes Marques de Carvalho, Gil Rocha, Bruno Martins, Arlindo L. Oliveira
| Challenge: | Large Language Models (LLMs) have been used to improve evaluation processes, but there are concerns over reliability and potential misapplications. |
| Approach: | They propose to deploy AI-assisted evaluation in two government initiatives . they found that the solution increased reviewer productivity by 20.1% . |
| Outcome: | The proposed solution reduced reviewer productivity by 20.1% while keeping a negligible false-positive rate . the proposed solution reduces evaluation time by more than 2 months . |
Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval (2024.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that Transformer-based language models lose information in the middle of input sequences, especially in the context of web document retrieval. |
| Approach: | They examine position biases at multiple stages of the training pipeline for an encoder-decoder neural retrieval model, namely language model pre-training, contrastive pre- training, and contrastive fine-tuning. |
| Outcome: | The proposed model generates embeddings that better capture the beginning of the input content, with fine-tuning further aggravating this effect. |
Dense Template Retrieval for Customer Support (2022.coling-1)
Copied to clipboard
| Challenge: | Templated answers are used to cover a wide range of topics, but the number of templates is often too high for an agent to manually search. |
| Approach: | They propose a dense retrieval framework that adapts a standard in-batch negatives technique to support unpaired sampling of queries and templates. |
| Outcome: | The proposed approach improves performance and training speed over more standard methods. |
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing captioning models are mostly available on English or are specialised for other high-resource languages. |
| Approach: | They propose an image-blind few-shot multilingual captioning model that prompts a language model with retrieved captions. |
| Outcome: | The proposed model is competitive with fully-supervised multilingual captioning models without requiring any supervised training on captioning data. |
From Tower to Spire: Adding the Speech Modality to a Translation-Specialist LLM (2025.findings-emnlp)
Copied to clipboard
Kshitij Ambilduke, Ben Peters, Sonal Sannigrahi, Anil Keshwani, Tsz Kin Lam, Bruno Martins, Andre Martins, Marcely Zanon Boito
| Challenge: | Spire is a speech-augmented language model capable of translating speech input into 10 languages and transcribing text input in both directions. |
| Approach: | They introduce a speech-augmented language model capable of translating speech input into 10 languages . they integrate the model into existing multilingual LMs via speech discretization . |
| Outcome: | Spire integrates speech-augmented language model into existing multilingual model using speech discretization and pre-training using only 42.5 K hours of speech. |
PAELLA: Parameter-Efficient Lightweight Language-Agnostic Captioning Model (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing models that only generate English captions are expensive due to the trend of scaling both data and model size. |
| Approach: | They propose a parameter-efficient lightweight language-agnostic captioning model that uses retrieval enhancement to train parameters between a visual model and a multilingual language model. |
| Outcome: | The proposed model outperforms models with more parameters and data and shows strong zero-shot abilities in low-resource languages. |
Annotating Arguments in a Corpus of Opinion Articles (2022.lrec-1)
Copied to clipboard
Gil Rocha, Luís Trigo, Henrique Lopes Cardoso, Rui Sousa-Silva, Paula Carvalho, Bruno Martins, Miguel Won
| Challenge: | Argument annotation is the process of exposing and justifying one's points of view, with the aim of conveying a logical reasoning through a set of semantically related propositions. |
| Approach: | They propose to use argumentative discourse units to annotate arguments in Portuguese using a multi-layered process to analyze the annotations produced. |
| Outcome: | The proposed model exploits the best practices identified in previous studies while fostering the potential use of the resulting annotated corpus for new purposes. |
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate image captions are English-centric, despite improvements in the CLIPScore metric . however, there are no available benchmarks for multilingual captioning evaluation . |
| Approach: | They propose to use machine-translated and machine-repurposed datasets to evaluate CLIPScore variants in multilingual settings. |
| Outcome: | The proposed evaluation strategies are based on machine-translated and human judgements. |
A Conformal Risk Control Framework for Granular Word Assessment and Uncertainty Calibration of CLIPScore Quality Estimates (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing caption evaluation metrics lack granular assessments for errors within captions . lack of uncertainty quantification can give way to misleading scores, reducing user trust . |
| Approach: | They propose a conformal risk control framework to generate and calibrate CLIPScore distributions . they use a model-agnostic conformal framework to detect erroneous words . |
| Outcome: | The proposed method detects erroneous words while providing formal guarantees aligned with desired risk levels. |
Efficient Architectures for High Resolution Vision-Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in vision-Language Models (VLMs) have limited accuracy of fine details within high resolution images, which limits performance in multiple tasks. |
| Approach: | They propose a new architecture that efficiently processes high-resolution images while training fewer parameters than similarly sized VLMs. |
| Outcome: | The proposed architecture achieves high efficiency while maintaining strong performance in tasks that require fine-grained image understanding and/or handling of scene-text. |