Papers by Éric Clergerie
PatentEval: Understanding Errors in Patent Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | a patent is a legal instrument that grants inventors or entities exclusive rights over their invention for a designated period. |
| Approach: | They propose a typology specifically designed for evaluating two distinct tasks in machine-generated patent texts. |
| Outcome: | The proposed approach provides valuable insights into the capabilities and limitations of current language models in the specialized field of patent text generation. |
Anisotropy Is Inherent to Self-Attention in Transformers (2024.eacl-long)
Copied to clipboard
| Challenge: | despite their success, Transformers models suffer from a representation degeneration problem . anisotropy is a property of hidden representations that makes them unexpectedly close to each other . |
| Approach: | They investigate the representation degeneration problem in a self-supervised learning model based on Transformers . anisotropy is a property of hidden representations which makes them unexpectedly close to each other . |
| Outcome: | The representation degeneration problem is a phenomenon widely observed among self-supervised learning methods based on Transformers. |