AutoRC: Improving BERT Based Relation Classification Models via Architecture Search (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing literature on relation classification models shows no consensus on optimal architecture . |
| Approach: | They propose a search space for BERT based relation classification models and employ an ENAS method to find better architectures. |
| Outcome: | The proposed method can find better architectures than baseline models on eight benchmark RC tasks. |
Similar Papers
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to enhance BERT with factual knowledge about entities require no additional pretraining and no changes to the encoder itself. |
| Approach: | They propose a way to inject factual knowledge into the pretrained BERT model by aligning Wikipedia2Vec entity vectors with BERT's native wordpiece vector space and feeding the aligned entity vector into BERT as if they were wordpieces. |
| Outcome: | The proposed version outperforms baseline models on unsupervised question answering, supervised relation classification and entity linking tasks. |
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition (D19-1)
Copied to clipboard
| Challenge: | Neural architecture search (NAS) is a popular approach for finding new models and freeing researchers from the hard work of designing network architectures. |
| Approach: | They propose differentiable neural architecture search methods for natural language processing . they remove the softmax-local constraint and apply it to named entity recognition . |
| Outcome: | The proposed method outperforms strong baselines on the language modeling task. |
RCL: Relation Contrastive Learning for Zero-Shot Relation Extraction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to extract relations require large-scale labeled data. |
| Approach: | They propose a Relation Contrastive Learning framework to mitigate similar relations and similar entities problems by optimizing a contrastive instance loss with a relation classification loss on seen relations. |
| Outcome: | The proposed framework can learn subtle difference between instances and achieve better separation between different relation categories in the representation space simultaneously. |
schuBERT: Optimizing Elements of BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent Transformer based models have achieved state-of-the-art performance for many natural language processing tasks including machine translation, question-answering tasks and semantic role labeling. |
| Approach: | They propose to reduce the number of parameters of BERT to obtain a much efficient light model. |
| Outcome: | The proposed model achieves 6.6% higher average accuracy on GLUE and SQuAD datasets than the previous model with three encoder layers while having the same number of parameters. |
NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a widely adopted NLP task . authors present three variants of NER task, with dataset to support them . |
| Approach: | They propose three variants of the NER task, together with a dataset to support them . they propose a move towards more fine-grained entities and zero-shot recognition . |
| Outcome: | The proposed model matches or surpasses existing models in NER tasks . the proposed model is based on a large, silver-annotated corpus of 4 million paragraphs . |
Best Practices for Distilling Large Language Models into BERT for Web Search Ranking (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of Large Language Models (LLMs) as zero-shot relevance rankers. |
| Approach: | They propose to use a ranking loss to transfer ranking knowledge from LLMs to smaller models like BERT. |
| Outcome: | The proposed model has been successfully integrated into a commercial web search engine as of February 2024. |
Simple Yet Powerful: An Overlooked Architecture for Nested Named Entity Recognition (2022.coling-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is an important task in Natural Language Processing that aims to identify text spans belonging to predefined categories. |
| Approach: | They propose to revisit the Multiple LSTM-CRF (MLC) model, a simple, overlooked, yet powerful approach based on training independent sequence labeling models for each entity type. |
| Outcome: | The proposed model achieves state-of-the-art results in the Chilean Waiting List corpus by including pre-trained language models. |
Large-scale Exploration of Neural Relation Classification Architectures (D18-1)
Copied to clipboard
| Challenge: | Existing studies on relation classification have been limited to a very narrow range of datasets, making comparisons between approaches difficult. |
| Approach: | They propose a multi-channel LSTM model combined with a CNN that takes advantage of all currently popular linguistic and architectural features. |
| Outcome: | The proposed model achieves state-of-the-art on two datasets and provides direct insights into the challenges faced by language models on relation classification. |
Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Rapid adoption of LLMs has overshadowed the potential advantages of traditional BERT-like models in text classification. |
| Approach: | They compare BERT-like models fine-tuning, LLM internal state utilization, and LLM zero-shot inference across six datasets. |
| Outcome: | The proposed method outperforms LLMs on six challenging datasets. |
Named Entity Recognition for Entity Linking: What Works and What’s Next (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Entity Linking (EL) systems have achieved impressive results on standard benchmarks thanks to the contextualized representations provided by recent pretrained language models. |
| Approach: | They propose to exploit Named Entity Recognition (NER) to narrow the gap between EL systems trained on high and low amounts of labeled data. |
| Outcome: | The proposed model can be exploited to narrow the gap between EL systems trained on high and low amounts of labeled data. |