Papers by Gian Wiher
On the probability–quality paradox in language generation (2022.acl-short)
Copied to clipboard
| Challenge: | a recent study suggests that mode-seeking decoding methods produce unnatural language . a higher probability text is perceived as more human-like, but this is not the case with stochastic decoding . |
| Approach: | They propose to explain why higher-probability text is perceived as more human-like . they argue that human-type language should contain an amount of information close to entropy . |
| Outcome: | The proposed model produces lower-probability text, which is perceived as more human-like . elucidation of this hypothesis supports the hypothesis . |
Locally Typical Sampling (2023.tacl-1)
Copied to clipboard
| Challenge: | a discrepancy in probabilistic language generators has puzzled the language generation community for years . |
| Approach: | They propose a method for local typical sampling to enforce a conditional entropy criterion for probabilistic models. |
| Outcome: | The proposed method can provide new insights into why high-probability texts can be dull or repetitive. |
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |