Papers by Jarca Andrei
Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Text (2025.findings-acl)
Copied to clipboard
| Challenge: | Masked language modeling is widely adopted, but the process of selecting tokens for masking is random and the percentage of masked tokens is typically fixed for the entire training process. |
| Approach: | They propose to adjust the masking ratio based on a task-informed anti-curriculum learning scheme to mask useful and harmful tokens. |
| Outcome: | The proposed approach improves the ability of the model to focus on key task-relevant features, contributing to statistically significant performance gains across tasks. |