Papers by Jerry Ngo
What Do Language Models Hear? Probing for Auditory Representations in Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a linear probe is used to retrieve the correct text representation of an object given a snippet of audio related to that object. |
| Approach: | They develop a linear probe that retrieves the correct text representation of an object . they then test the probe's generalization to objects that were not seen during training . |
| Outcome: | The proposed model generalizes to objects that were not seen during training, the study finds . the model can learn representations of perceptual concepts that plausibly mirror the grounded representations . |