Adjusting Image Attributes of Localized Regions with Low-level Dialogue (2020.lrec-1)
Copied to clipboard
| Challenge: | Image editing is time-consuming and requires a wide assortment of features and combinations of these features to achieve a desired effect. |
| Approach: | They propose a task-oriented dialogue system to investigate low-level instructions for NLIE . 25% of users found the system easy-to-use, resonating with their motivation . |
| Outcome: | The proposed system is easy-to-use and user-friendly. |
Similar Papers
Learning to Follow Object-Centric Image Editing Instructions Faithfully (2023.findings-emnlp)
Copied to clipboard
| Challenge: | avrahami et al., 2022b,a): natural language instructions are often underspecified, requiring models to uncover their implicit meaning. |
| Approach: | They propose to use paired data to model the implicit meaning of instructions . they also propose to ground the model to localize where the edit has to be performed . |
| Outcome: | The proposed model performs better than state-of-the-art baselines on paired data, showing improvements in quality and faithfulness. |
Edit me: A Corpus and a Framework for Understanding Natural Language Image Editing (L18-1)
Copied to clipboard
Ramesh Manuvinakurike, Jacqueline Brixey, Trung Bui, Walter Chang, Doo Soon Kim, Ron Artstein, Kallirroi Georgila
| Challenge: | a corpus of image edit requests is elicited for real world images, and an annotation framework is developed . evaluators evaluate crowd-sourced annotation as a means of efficiently creating a sizable corpus at a reasonable cost. |
| Approach: | They propose a natural language interface for interacting with an image editing program . they propose an annotation framework for understanding natural language requests . |
| Outcome: | The proposed tool interprets image edit requests and maps them to actionable commands. |
Hands-off Image Editing: Language-guided Editing without any Task-specific Labeling, Masking or even Training (2025.coling-main)
Copied to clipboard
| Challenge: | State-of-the-art approaches to this task resort to supervised training and labelling, masking or training. |
| Approach: | They propose an approach that does without any task-specific supervision and offers thus a better potential for improvement. |
| Outcome: | The proposed approach achieves very competitive performance and scales up in a way that requires no task-specific supervision. |
DELOC: Document Element Localizer (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to edit PDFs using natural language are ineffective at grounding the edit location effectively. |
| Approach: | They propose a system to ground PDF edit request spatially using a model to predict the edit location in the PDF. |
| Outcome: | The proposed system outperforms existing Multimodal Large Language Models and specialized models on DocEdit. |
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021.naacl-main)
Copied to clipboard
Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Hassan Awadallah
| Challenge: | Existing systems frame semantic parsing as a one-shot translation from a natural language question to the logical form. |
| Approach: | They propose a model that uses natural language feedback to correct parsers . they show that NL-EDIT can boost the accuracy of existing parser by 20% . |
| Outcome: | The proposed model can boost parsers' accuracy by 20% with just one turn of correction. |
Bayesian Optimization for Controlled Image Editing via LLMs (2025.findings-acl)
Copied to clipboard
Chengkun Cai, Haoliang Liu, Xu Zhao, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, John Lee, Jenq-Neng Hwang, Lei Li
| Challenge: | achieving precise control over generated content and maintaining semantic consistency remain significant limitations, particularly concerning grounding techniques and the necessity for model fine-tuning. |
| Approach: | They propose an off-the-shelf approach that integrates Large Language Models with Bayesian Optimization to facilitate precise and user-friendly image editing. |
| Outcome: | The proposed approach outperforms existing methods in editing accuracy and semantic preservation, as validated using different LLMs including Claude3 and GPT-4. |
Toward Interactive Regional Understanding in Vision-Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing image-text pairs capture only coarse and global information of an image, leading to a limitation in their regional understanding ability. |
| Approach: | They propose a model with explicit regional modeling capabilities that allows VLP models to understand user-indicated image regions. |
| Outcome: | The proposed model performs better on zero-shot region understanding tasks without compromising its ability for global image understanding. |
HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks (2026.findings-acl)
Copied to clipboard
Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting Yu
| Challenge: | Existing approaches treat instruction-based text editing as a generic text generation problem. Existing methods either over-edit or fail to apply modifications consistently. |
| Approach: | They propose a framework that processes each editing request to best align with it. |
| Outcome: | The proposed framework achieves 9% improvement over the state-of-the-art model. |
Understanding Pre-Editing for Black-Box Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | a study has demonstrated the effectiveness of pre-editing for black-box neural MT, but a deep understanding of what it is and how it works for black box NMT is lacking. |
| Approach: | They investigated 6,652 instances of pre-editing across three translation directions, two MT systems and four text domains. |
| Outcome: | The proposed method can be used in MT systems with black-box neural MT (NMT) but it is not yet fully understood in the literature. |
XplaiNLI: Explainable Natural Language Inference through Visual Analytics (2020.coling-demos)
Copied to clipboard
| Challenge: | Recent research has revealed some heuristics and biases of natural language inference models. |
| Approach: | They propose an interactive visualization interface that computes Natural Language Inference with different methods and provides explanations for the decisions made by each approach. |
| Outcome: | The proposed interface computes Natural Language Inference (NLI) with three different approaches and provides explanations for the decisions made by each approach. |