On an Intermediate Task for Classifying URL Citations on Scholarly Papers (2024.lrec-main)
Copied to clipboard
| Challenge: | Citations using URLs can be used as information source for research resource search engines. |
| Approach: | They propose a method to classify URL citations using a simple fine-tuning strategy. |
| Outcome: | The proposed method outperforms methods using a simple fine-tuning strategy with higher macro F-scores for different model sizes and architectures. |
Similar Papers
Fine-Tuning Language Models on Multiple Datasets for Citation Intention Classification (2024.findings-emnlp)
Copied to clipboard
Zeren Shui, Petros Karypis, Daniel Karls, Mingjian Wen, Saurav Manchanda, Ellad Tadmor, George Karypis
| Challenge: | Prior research has shown that pretrained language models (PLMs) can achieve state-of-the-art performance on CIC benchmarks. |
| Approach: | They propose a multi-task learning framework that fine-tunes pretrained language models on a dataset of primary interest together with multiple auxiliary CIC datasets to take advantage of additional supervision signals. |
| Outcome: | The proposed framework outperforms current state-of-the-art models on small datasets while aligning with the best-performing model on a large dataset. |
Cluster & Tune: Boost Cold Start Performance in Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained models for text classification are poor in practice. |
| Approach: | They propose to add an intermediate unsupervised classification task between pre-training and fine-tuning phases to boost performance of pre-trained models. |
| Outcome: | The proposed method improves performance on topical classification tasks when labeled data is scarce. |
What to Pre-Train on? Efficient Intermediate Task Selection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning intermediate tasks are inefficient and expensive. |
| Approach: | They propose to use a set of 42 intermediate and 11 target English classification, multiple choice, question answering, and sequence tagging tasks to identify the best settings for intermediate transfer learning. |
| Outcome: | The proposed methods achieve an average Regret@3 of 1% across all target tasks. |
Rethinking Why Intermediate-Task Fine-Tuning Works (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models can be fine-tuned on intermediate labeled-data tasks before fine- tuning the models on the target task of interest. |
| Approach: | They conduct extensive experiments to study the impact of different factors on STILT . they find that the improvement from an intermediate task could be orthogonal to it containing reasoning or other complex skills. |
| Outcome: | The proposed method improves the performance of pretrained language models on various target tasks. |
A Context-based Framework for Modeling the Role and Function of On-line Resource Citations in Scientific Literature (D19-1)
Copied to clipboard
| Challenge: | Existing academic search engines cannot detect relevant papers where a resource is mentioned. |
| Approach: | They propose a framework to model the role and function of on-line resource citations . they construct a dataset SciRes, which includes 3,088 manually annotated resource contexts based on a multi-task framework . |
| Outcome: | The proposed model achieves the best results on both the classification task and recommendation task. |
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on automatic generation of citation texts in scholarly papers have not investigated this problem. |
| Approach: | They propose to train an implicit citation extraction model based on BERT and a multi-source pointer-generator network with cross attention mechanism for citation text generation. |
| Outcome: | The proposed model can generate short texts to describe cited papers in scholarly papers with training data. |
Self-Supervised Intermediate Fine-Tuning of Biomedical Language Models for Interpreting Patient Case Descriptions (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has found that biomedical language models lack the knowledge needed for such tasks. |
| Approach: | They propose to fine-tune biomedical language models on the task of predicting masked medical concepts from PubMed abstracts to improve their performance. |
| Outcome: | The proposed strategy improves the performance of biomedical language models on the task of predicting masked medical concepts from patient case descriptions. |
CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding (2021.findings-acl)
Copied to clipboard
| Challenge: | Scientific document understanding is challenging due to the highly domain specific nature of scientific language. |
| Approach: | They propose a large, contextualized, rigorously cleaned labelled dataset for cite-worthiness detection built from extracted scientific documents. |
| Outcome: | The proposed model improves on a paragraphlevel contextualized sentence labelling model based on Longformer . the model shows a 5 F1 point improvement over SciBERT which considers only individual sentences . |
ACT2: A multi-disciplinary semi-structured dataset for importance and purpose classification of citations (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for classifying citations rely on bibliometric measures to consider the semantics of citation. |
| Approach: | They propose to use a Citation Context Classification (3C) shared task dataset to classify citations according to their purpose and importance. |
| Outcome: | The proposed model can be used to link research works to graphs and enable efficient knowledge discovery. |
Extreme Fine-tuning: A Novel and Fast Fine-tuning Approach for Text Classification (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models require massive computational resources and time. |
| Approach: | They propose a novel approach for fine-tuning a pre-trained model using backpropagation and an iterative extreme learning machine for training a classifier. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in training-time measurement and performance with comparable model performance. |