| Challenge: | a new annotation tool is designed to fill the niche of a lightweight interface for terminal users . current tools are built with direct manipulation via a Graphical User Interface (GUI) this approach is time-consuming and difficult to modify . |
| Approach: | They propose a terminal-based annotation tool that supports multiple annotations . they use a text-based interface that uses almost the entire screen to display documents . |
| Outcome: | The proposed tool is designed to fill the niche of a lightweight interface for users with a terminal-based workflow. |
Similar Papers
YEDDA: A Lightweight Collaborative Text Span Annotation Tool (P18-4)
Copied to clipboard
| Challenge: | Existing annotation tools do not consider post-annotation quality analysis due to inter-annotator disagreement. |
| Approach: | They propose a lightweight but efficient open-source tool for text span annotation that can be used for collaborative user annotation and administrator evaluation and analysis. |
| Outcome: | The proposed system reduces the annotation time by half compared with existing tools and the time can be compressed by 16.47% through intelligent recommendation. |
Tools Impact on the Quality of Annotations for Chat Untangling (2021.acl-srw)
Copied to clipboard
| Challenge: | Creating annotated datasets is time-consuming and expensive . a tool that minimizes task load and facilitates the annotation process is lacking in studies on how it influences annotation quality. |
| Approach: | They propose to use SLATE and Parlay to improve annotation quality for a task of chat-untangling. |
| Outcome: | The proposed tool improves the user experience for the task of chatuntangling. |
POTATO: The Portable Text Annotation Tool (2022.emnlp-demos)
Copied to clipboard
Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, David Jurgens
| Challenge: | POTATO is a free, fully open-sourced annotation system that supports labeling many types of text and multimodal data. |
| Approach: | They propose to use POTATO to design and deploy complex annotation tasks. |
| Outcome: | The proposed annotation system improves labeling speed and productivity over two tasks. |
Lightweight Grammatical Annotation in the TEI: New Perspectives (L18-1)
Copied to clipboard
| Challenge: | a small set of descriptive devices have been made available for lightweight linguistic annotation . merit of a predefined TEI tagset is the homogeneity of tagging and better interoperability of simple linguistic resources encoded in the TE. |
| Approach: | They propose a new attribute class that would gather token-level attributes facilitating simple linguistic annotation. |
| Outcome: | The proposed attribute class addresses community feedback on the lack of a specific tagset for lightweight linguistic annotation within the TEI. |
LightTag: Text Annotation Platform (2021.emnlp-demo)
Copied to clipboard
| Challenge: | LightTag is a text annotation tool built on the premise of global optimization by addressing annotator as well as project managers and data scientists who manage the work and enforce production quality. |
| Approach: | They propose to use LightTag to optimize the global NLP process by addressing annotators as well as project managers and data scientists who manage the work and enforce production quality. |
| Outcome: | The proposed tool is based on the theory of constraints and is available for free for academic use. |
SLATE: A Sequence Labeling Approach for Task Extraction from Free-form Inked Content (2022.emnlp-industry)
Copied to clipboard
Apurva Gandhi, Ryan Serrao, Biyi Fang, Gilbert Antonius, Jenna Hong, Tra My Nguyen, Sheng Yi, Ehi Nosakhare, Irene Shaffer, Soundararajan Srinivasan
| Challenge: | SLATE is a sequence labeling approach for extracting tasks from free-form content . past approaches for task extraction from typed content focus on building separate sentence-level task classification models. |
| Approach: | They propose a sequence labeling approach for extracting tasks from free-form content . they use a single, low-latency sequence labelling approach to perform sentence segmentation and classification . |
| Outcome: | The proposed model outperforms a baseline model and achieves 84.4% task F1 score and 88.4% boundary similarity score. |
Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains (P19-1)
Copied to clipboard
Claudia Schulz, Christian M. Meyer, Jan Kiesewetter, Michael Sailer, Elisabeth Bauer, Martin R. Fischer, Frank Fischer, Iryna Gurevych
| Challenge: | Existing deep learning methods require large amounts of training data to achieve reasonable performance. |
| Approach: | They propose to generate automatic annotation suggestions for a discourse-level sequence labelling task that requires extensive domain expertise. |
| Outcome: | The proposed model improves with newly annotated texts while introducing no biases. |
A Web-based Collaborative Annotation and Consolidation Tool (2020.lrec-1)
Copied to clipboard
| Challenge: | Annotation tools have a rigid structure, closed back-end and front-end, and are built in a non-user-friendly way rendering them unusable for a large cohort. |
| Approach: | They propose a web-based collaborative annotation and consolidation tool (AWOCATo) that supports varied textual formats and allows users to easily adapt to the annotation task. |
| Outcome: | AWOCATo supports a range of tasks and domains, filling the gap left by the lack of tools that can be used by people with and without programming knowledge. |
Corpus Considerations for Annotator Modeling and Scaling (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent trends in natural language processing and annotation tasks emphasize individual perspectives . annotator models that rely on a single ground truth may disregard valuable minority perspectives omissions . |
| Approach: | They propose a composite embedding approach to investigate annotator modeling techniques . they show that the commonly used user token model consistently outperforms more complex models . |
| Outcome: | The proposed model outperforms more complex models on a given dataset. |
CoRefi: A Crowd Sourcing Suite for Coreference Annotation (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Using a web-based coreference annotation suite, we demonstrate that non-expert annotators can be trained to perform and review coreference resolution tasks. |
| Approach: | They propose a web-based coreference annotation suite oriented for crowdsourcing that provides guided onboarding and a novel algorithm for a reviewing phase. |
| Outcome: | The proposed tool provides guided onboarding and a novel algorithm for a review phase. |