Improving Personalized Sentiment Representation with Knowledge-enhanced and Parameter-efficient Layer Normalization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on personalized sentiment classification consider document reviews as overall text unit and incorporate backgrounds (i.e., user and product information) Existing methods for personalized sentiment modeling have quadratic costs that increase with text length and heterogeneous mixes of background information and textual information. |
| Approach: | They propose a knowledge-enhanced and parameter-efficient layer normalization model that leverages pretrained checkpoints and background information into transformer structures. |
| Outcome: | The proposed model can be used to improve pretrained language models in document reviews and incorporate background information with parameter-efficient fine-tuning and knowledge injecting. |
Similar Papers
Improving Document-Level Sentiment Analysis with User and Product Context (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work that improves document-level sentiment analysis by encoding user and product information has been limited to considering only the text of the current review. |
| Approach: | They propose to incorporate all available historical review text belonging to the author of the review in question and investigate the inclusion of his- torical reviews associated with the current product. |
| Outcome: | The proposed model improves on IMDB, Yelp 2013 and Yelpan 2014 datasets by more than 2 percentage points in the best case. |
Semantic Simplification for Sentiment Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on document-level sentiment classification has shown that the sentiment in the original text is often hard to capture . previous studies focus on predicting the overall sentiment from original text using statistical or neural models, but these methods either heavily rely on human knowledge or suffer from the complex structure of the text. |
| Approach: | They propose a document-level sentiment classification model that enhances the original text with a simplified clause to intensify its sentiment. |
| Outcome: | Empirical studies show that the proposed model over strong baselines is effective over several strong baseline models. |
XDoc: Unified Pre-training for Cross-Format Document Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained models target one document format at a time, making it difficult to combine knowledge from multiple document formats. |
| Approach: | They propose a unified pre-trained model which deals with different document formats in a single model. |
| Outcome: | The proposed model achieves comparable or even better performance on a variety of downstream tasks compared with the individual pre-trained models. |
UserAdapter: Few-Shot User Learning in Sentiment Analysis (2021.findings-acl)
Copied to clipboard
| Challenge: | Adapting a model to a handful of personalized data is challenging, authors say . standard fine-tuning requires hundreds of millions of parameters for each user . |
| Approach: | They propose a lightweight method that clamps millions of parameters of a Transformer model and optimizes a tiny user-specific vector. |
| Outcome: | The proposed method improves accuracy on Yelp and IMDB datasets and reduces the number of parameters added for each user. |
Linear Layer Extrapolation for Fine-Grained Emotion Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that Transformer-based language models are more factual accurate in later layers . |
| Approach: | They propose a method that optimizes contrast based on the selected intermediate layer . they observe a similar pattern for fine-grained emotion classification in text . |
| Outcome: | Experiments show that the proposed method outperforms standard methods in fine-grained emotion classification tasks. |
Pre-trained Personalized Review Summarization with Effective Salience Estimation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) are a new paradigm in text generation for the strong ability of natural language comprehension. |
| Approach: | They propose a pre-trained personalized review summarization method that incorporates personalized information into the salience estimation of input reviews. |
| Outcome: | The proposed method performs better than the state-of-the-art methods on real-world datasets. |
Feature Structure Matching for Multi-source Sentiment Analysis with Efficient Adaptive Tuning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing domain matching methods tend to pull all feature instances close, but they are expensive and expensive to update. |
| Approach: | They propose to extract multi-layer features from a large pre-trained model and propose a dynamic parameter fusion module to exploit them for efficient and adaptive tuning. |
| Outcome: | The proposed framework is more robust and generalizable in the multi-source scenario. |
Using the Past Knowledge to Improve Sentiment Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing model retains knowledge learned from past tasks and selectively transfers it to new task to help it learn better. |
| Approach: | They propose a lifelong learning model that can retain and selectively transfer the knowledge learned in the past to help learn the new task. |
| Outcome: | The proposed model outperforms strong baselines, including even multiple task learning. |
Exploiting Rich Textual User-Product Context for Improving Personalized Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Typical approaches do not exploit the potential of historical reviews or do not make full use of user/product associations. |
| Approach: | They propose to use historical reviews to initialize user and product representations and incorporate textual associations via a user-product cross-context module. |
| Outcome: | The proposed method outperforms existing state-of-the-art models on IMDb, Yelp and Longformer benchmarks. |
Metadata Shaping: A Simple Approach for Knowledge-Enhanced Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to capture entity knowledge with factual knowledge are limited . despite its simplicity, metadata shaping is quite effective . |
| Approach: | They propose a method which inserts substrings corresponding to readily available entity metadata into examples at train and inference time based on mutual information. |
| Outcome: | The proposed method exceeds the baseline model by 4.3 F1 points and achieves state-of-the-art results. |