Papers by Rishi Bommasani
Evaluation for Change (2023.findings-acl)
Copied to clipboard
| Challenge: | Evaluation is the central means for assessing, understanding, and communicating about NLP models. |
| Approach: | They argue that evaluation should be more than that: it is a force for driving change and has a sociological and political character beyond its technical dimensions. |
| Outcome: | The proposed analysis concludes that evaluation’s power is waning despite its potential for realizing more pluralistic ambitions in the field. |
Intrinsic Evaluation of Summarization Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Almost all popular summarization datasets do not come with inherent quality assurance guarantees. |
| Approach: | They propose to use 5 metrics to evaluate quality of summarization datasets . they find that data usage in recent summarizing research is inconsistent with the properties of the data. |
| Outcome: | The proposed metrics can be inexpensive heuristics for detecting generically low quality examples. |
Long-Distance Dependencies Don’t Have to Be Long: Simplifying through Provably (Approximately) Optimal Permutations (P19-2)
Copied to clipboard
| Challenge: | Neural models at the sentence level often need to model the interaction between words . however, there is no guarantee that the standard ordering of words is computationally efficient or optimal . |
| Approach: | They propose to use a dependency parse as a proxy for inter-word dependencies in a sentence to simplify the sentence with combinatorial objectives imposed on the sentence-parse pair. |
| Outcome: | The proposed model improves classification accuracy and reduces classification error by 2.0% over the previous state of the art. |
Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Contextualized representations have become the default for downstream NLP applications. |
| Approach: | They propose a method for converting from contextualized representations to static lookup-table embeddings and apply it to 5 popular pretrained models and 9 sets of pretrained weights. |
| Outcome: | The proposed methods show that pooling over many contexts significantly improves representational quality under intrinsic evaluation. |