Papers with variability
Using J-K-fold Cross Validation To Reduce Variance When Tuning NLP Models (C18-1)
Copied to clipboard
| Challenge: | a recent study shows that performance estimations are unstable and variable . this makes it difficult to use parameter tuning and model selection . |
| Approach: | They propose to use a less variable CV method to evaluate performance . they propose lower choices of K than are typically seen in the NLP literature . |
| Outcome: | The proposed method can be used for parameter tuning and performance estimation, but it is unstable and unstable. |
Reproduction and Replication: A Case Study with Automatic Essay Scoring (2020.lrec-1)
Copied to clipboard
| Challenge: | reproducibility of experiments has gained more attention in the NLP community . recent negative reproduction results indicate that published results are not verifiable . |
| Approach: | They propose to reproduce an earlier study of automatic essay scoring for determining the proficiency of second language learners in a multilingual setting. |
| Outcome: | The proposed reproduction of an AES system for determining the proficiency of second language learners in a multilingual setting is compared with the original. |
Do LLM Agents Mirror Socio-Cognitive Effects in Power-Asymmetric Conversations? (2026.acl-long)
Copied to clipboard
| Challenge: | Power differences shape human communication through well-documented socio-cognitive effects . asymmetric relationships or power differentials give rise to well-known socio-computational effects - lianelli, 1976 . |
| Approach: | They simulate multi-turn, power-asymmetric dialogues with personas from diverse professions . they find that LLMs show key socio-cognitive effects of power, albeit with nuances and variability . |
| Outcome: | The results show that large language models exhibit socio-cognitive effects of power . the results are consistent with previous studies on LLMs . |