Papers by Maksim Eremeev
Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling (2022.aacl-main)
Copied to clipboard
| Challenge: | Neural autoregressive sequence models assign high probability to unreasonably short sequences . authors propose to minimize oversmoothing rate during training . |
| Approach: | They propose to minimize the oversmoothing rate during training by tuning the regularization strength. |
| Outcome: | The proposed regularization can control the oversmoothing rate and improve decoding performance. |
Injecting knowledge into language generation: a case study in auto-charting after-visit care instructions from medical dialogue (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language modeling have enabled applications across multiple domains such as education, jurisprudence, and healthcare. |
| Approach: | They propose a method to use knowledge to identify which rare words are important and uplift their conditional probability. |
| Outcome: | The proposed approach reduces the uncertainty of the model and improves factuality and coherence without negatively impacting fluency. |