Papers with ICA
Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed Embeddings (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Embedding is an important component in natural language processing, but interpreting high-dimensional embeddings remains challenging. |
| Approach: | They propose a method which optimizes the order of axes in word embedding space by maximizing semantic continuity. |
| Outcome: | The proposed method improves the clarity of the word embedding space by maximizing the semantic continuity of the axes. |
LLM-Friendly Knowledge Representation for Customer Support (2025.coling-industry)
Copied to clipboard
| Challenge: | a new approach to customer support is proposed to integrate large language models with a framework designed to navigate the complexities of Airbnb customer support operations. |
| Approach: | They propose a method for integrating Large Language Models with a framework designed to navigate the complexities of Airbnb customer support operations. |
| Outcome: | The proposed approach is cost-effective and improves customer support performance . it also allows human agents to focus on more complex issues, the authors show . |
Investigating the Contextualised Word Embedding Dimensions Specified for Contextual and Temporal Semantic Changes (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on the meaning of contextualised word embeddings (SCWEs) have not shown how meaning changes are encoded in the embeddable space. |
| Approach: | They compare pre-trained and fine-tuned contextualised word embeddings on contextual and temporal semantic change detection benchmarks. |
| Outcome: | The pre-trained and fine-tuned versions of (SCWE) and their fine- tuned versions on contextual and temporal semantic change detection benchmarks show that they represent semantic changes across all dimensions when fine--and that they are more efficient than ICA. |
Understanding Higher-Order Correlations Among Semantic Components in Embeddings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Independent Component Analysis (ICA) is an effective method for visualizing and interpreting the geometric structure of embeddings. |
| Approach: | They quantified embeddings' non-independencies using higher-order correlations and a maximum spanning tree of semantic components. |
| Outcome: | The results provide deeper insights into embeddings through ICA. |
Discovering Universal Geometry in Embeddings with ICA (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on achieving sparse embeddings or acquiring semantic axes, but this study focuses on the intrinsic independence present within embeddables. |
| Approach: | They propose to use independent component analysis to extract independent semantic components from pre-trained embeddings by leveraging anisotropic information that remains after the whitening process in Principal Component Analysis. |
| Outcome: | The proposed method reveals that embeddings can be expressed as a composition of a few interpretable axes and that these axe axe are consistent across languages, algorithms, and modalities. |
Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on cosine similarity focus on the angle or correlation coefficient, but this study proposes a novel interpretation of the term word similarity. |
| Approach: | They propose a method for selecting statistically significant axes by deriving the probability distributions that govern each component and the product of components. |
| Outcome: | The proposed interpretation of cosine similarity is demonstrated through intuitive numerical examples and thorough numerical experiments. |
How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that In-Context Learning (ICA) can align Large Language Models (LLMs) with human preferences without requiring parameter adjustments. |
| Approach: | They investigate the effectiveness of each part in enabling ICA to function effectively and examine how variants in these parts impact alignment performance. |
| Outcome: | The proposed model can comprehend human instructions without parameter adjustments. |
Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for conversational AI are user-driven, but in many real-world situations, they do not extract information to achieve its own objectives. |
| Approach: | They propose an inquisitive conversational agent that learns when and how to ask probing questions . they also propose a framework for a conversational ICA specifically tailored to the court . |
| Outcome: | The proposed method outperforms single-agent RL baselines on a U.S. Supreme Court dataset. |
Exploring Interpretability of Independent Components of Word Embeddings with Automated Word Intruder Test (2024.lrec-main)
Copied to clipboard
| Challenge: | Independent Component Analysis (ICA) is an algorithm for finding separate sources in a mixed signal. |
| Approach: | They propose to use ICA to analyze word embeddings to quantify interpretability . they propose to automate word intruder test to quantify the components . |
| Outcome: | The proposed algorithm can be used to find semantic features of words . it can be combined to find words that have features associated with the components . |
Exploring Intra and Inter-language Consistency in Embeddings with ICA (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that ICA can reveal universal semantic axes across languages but lack verification of consistency of independent components within and across languages. |
| Approach: | They propose to use independent component analysis to identify independent components that are more interpretable than PCA to find universal semantic axes. |
| Outcome: | The proposed framework ensures the reliability and universality of semantic axes. |