A Simple Log-based Loss Function for Ordinal Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for ordinal text classification do not incorporate ordinal character into their feedback. |
| Approach: | They propose a new ordinal log-loss loss function that incorporates ordinal character into its feedback. |
| Outcome: | The proposed loss function outperforms state-of-the-art methods on four benchmark text classification datasets. |
Similar Papers
Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques (2024.findings-acl)
Copied to clipboard
Siva Rajesh Kasa, Aniket Goel, Karan Gupta, Sumegh Roychowdhury, Pattisapu Priyatam, Anish Bhanushali, Prasanna Srinivasa Murthy
| Challenge: | Ordinal classification (OC) is a key task in natural language processing with applications in various domains such as sentiment analysis, rating prediction, and more. |
| Approach: | They propose to tackle ordinal classification (OC) through the implicit semantics of the labels . they propose to use a classical explicit approach and an implicit approach that organically engages the semantics. |
| Outcome: | The proposed methods are based on pre-trained language models and offer strategic recommendations based upon specific settings. |
Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods do not provide a fairly accurate comparison of the two loss functions. |
| Approach: | They propose to use the Bregman divergence to provide a unified interpretation of the softmax cross-entropy and negative sampling loss functions. |
| Outcome: | The proposed model can be used to predict missing relational links between entities using a scoring method. |
Ranking-Constrained Learning with Rationales for Text Classification (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification use labels and rationales as ranking constraints. |
| Approach: | They propose a ranking-constrained loss function that combines cross-entropy loss with ranking losses as rationale constraints to speed up deep learning models with limited training data. |
| Outcome: | The proposed approach outperforms baselines on three human-annotated datasets and shows that it is more efficient than existing approaches. |
Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models for multi-label classification ignore complexity and dependencies among labels . Experimental results show that our method can obtain more accurate multi-lab classification results. |
| Approach: | They propose a meta-learning method to capture complex label dependencies . they use a Meta-learner to jointly learn the training policies and prediction policies for different labels. |
| Outcome: | The proposed method can capture complex label dependencies on fine-grained entity typing and text classification tasks. |
Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-Tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive performance across various tasks, but current training approaches combine cross-entropy loss with extensive data, human feedback, or ad hoc methods to enhance performance. |
| Approach: | They evaluate the use of established semantic segmentation loss functions in natural language generation to create a versatile, practical, and scalable solution for fine-tuning different architectures. |
| Outcome: | The proposed models improve on Math Word Problems and question answering tasks without additional data or human feedback. |
An Effective Deployment of Contrastive Learning in Multi-label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning in natural language processing tasks have not explored the effectiveness of the technology. |
| Approach: | They propose five novel contrastive losses for multi-label text classification tasks that exploit the complexity of the input logic and the semantic representation space. |
| Outcome: | The proposed contrastive losses improve multi-label text classification tasks and can be adapted for multi-task tasks. |
Similarity-Based Reconstruction Loss for Meaning Representation (D18-1)
Copied to clipboard
| Challenge: | a new paper proposes and evaluates a set of loss functions that can be used to train models for representation learning . cross-entropy loss penalizes models when they fail to generate the exact word from ground truth data . |
| Approach: | They propose and evaluate loss functions that can be used to train any neural model for representation learning. |
| Outcome: | The proposed loss functions amplify semantic diversity while preserving original meaning . they show performance improvement on paraphrase detection and language inference tasks . |
Hierarchical Label Generation for Text Classification (2023.findings-eacl)
Copied to clipboard
| Challenge: | None Hierarchical text classification (HTC) aims to assign the most relevant labels with their structure for a given document. |
| Approach: | They propose a method that captures the label hierarchy for real-world classification applications by using a taxonomic hierarchy. |
| Outcome: | The proposed method can generate unseen labels in subword level. |
Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-domain sentiment classification are difficult and costly . domain adaptation is difficult because data in source and target domains are drawn from different distributions. |
| Approach: | They propose a semi-supervised learning approach that minimizes the distance between source and target instances in embedded feature space. |
| Outcome: | The proposed approach can improve on baseline methods in various settings. |
Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Cross-domain sentiment analysis is a hot topic in research and industry . domain-invariant representation learning (DIRL) is used to learn a feature representation across domains . but, when label distribution P(Y) shifts across domain, it degrades performance . |
| Approach: | They propose a domain-invariant representation learning framework to improve cross-domain sentiment analysis performance. |
| Outcome: | The proposed model is easy to transfer existing models to the proposed model. |