Edit me: A Corpus and a Framework for Understanding Natural Language Image Editing (L18-1)
Copied to clipboard
Ramesh Manuvinakurike, Jacqueline Brixey, Trung Bui, Walter Chang, Doo Soon Kim, Ron Artstein, Kallirroi Georgila
| Challenge: | a corpus of image edit requests is elicited for real world images, and an annotation framework is developed . evaluators evaluate crowd-sourced annotation as a means of efficiently creating a sizable corpus at a reasonable cost. |
| Approach: | They propose a natural language interface for interacting with an image editing program . they propose an annotation framework for understanding natural language requests . |
| Outcome: | The proposed tool interprets image edit requests and maps them to actionable commands. |
Similar Papers
Learning to Follow Object-Centric Image Editing Instructions Faithfully (2023.findings-emnlp)
Copied to clipboard
| Challenge: | avrahami et al., 2022b,a): natural language instructions are often underspecified, requiring models to uncover their implicit meaning. |
| Approach: | They propose to use paired data to model the implicit meaning of instructions . they also propose to ground the model to localize where the edit has to be performed . |
| Outcome: | The proposed model performs better than state-of-the-art baselines on paired data, showing improvements in quality and faithfulness. |
Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions (L18-1)
Copied to clipboard
Albert Gatt, Marc Tanti, Adrian Muscat, Patrizia Paggio, Reuben A Farrugia, Claudia Borg, Kenneth P Camilleri, Michael Rosner, Lonneke van der Plas
| Challenge: | a crowdsourcing study has been conducted to generate rich textual descriptions of human faces . the aim is to investigate how users describe images of human face images . |
| Approach: | They propose to extend the problem of automatically generating text from images to face description . they conducted an annotation study on a subset of the corpus to gain a better understanding of the variation they find in face descriptions . |
| Outcome: | The proposed corpus is based on images taken in the wild and is expected to be large enough to support non-trivial machine learning work on the automated description of faces. |
WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse (D18-1)
Copied to clipboard
| Challenge: | a corpus of 43 million atomic edits is available for Wikipedia edit history . edits are instances in which a human editor has inserted a single contiguous phrase into, or deleted a contigous phrase from, an existing sentence. |
| Approach: | They use Wikipedia edit history to mine atomic edits across 8 languages . they find edits contain instances in which a human editor has inserted a single phrase into, or deleted a contiguous phrase from, an existing sentence. |
| Outcome: | The data show that edits differ from the language observed in standard corpora and that models trained on edits encode different aspects of semantics and discourse than models trained in raw text. |
Building The First English-Brazilian Portuguese Corpus for Automatic Post-Editing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing corpus for automatic post-editing of English and Brazilian Portuguese is limited. |
| Approach: | They introduce a corpus for Automatic Post-Editing of English and Brazilian Portuguese. |
| Outcome: | The proposed corpus improves on the English and Brazilian Portuguese languages. |
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
Adjusting Image Attributes of Localized Regions with Low-level Dialogue (2020.lrec-1)
Copied to clipboard
| Challenge: | Image editing is time-consuming and requires a wide assortment of features and combinations of these features to achieve a desired effect. |
| Approach: | They propose a task-oriented dialogue system to investigate low-level instructions for NLIE . 25% of users found the system easy-to-use, resonating with their motivation . |
| Outcome: | The proposed system is easy-to-use and user-friendly. |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |
Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation (2021.acl-long)
Copied to clipboard
| Challenge: | Edited media frames are structured annotations with respect to intents, emotional reactions, attacks on individuals, and the implications of disinformation. |
| Approach: | They propose a new formalism to understand visual media manipulation as structured annotations with respect to intents, emotional reactions, attacks on individuals, and the implications of disinformation. |
| Outcome: | The proposed model obtains promising results on a dataset with 56k question-answer pairs written in rich natural language. |
Corpus Considerations for Annotator Modeling and Scaling (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent trends in natural language processing and annotation tasks emphasize individual perspectives . annotator models that rely on a single ground truth may disregard valuable minority perspectives omissions . |
| Approach: | They propose a composite embedding approach to investigate annotator modeling techniques . they show that the commonly used user token model consistently outperforms more complex models . |
| Outcome: | The proposed model outperforms more complex models on a given dataset. |
Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)
Copied to clipboard
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in model editing for LLMs have created challenges and opportunities for the community. |
| Approach: | They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
| Outcome: | The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |