Papers by Mitja Nikolaus
Do Vision-and-Language Transformers Learn Grounded Predicate-Noun Dependencies? (2022.emnlp-main)
Copied to clipboard
| Challenge: | a recent study examines whether vision-and-language models learn syntactic dependencies . a controlled evaluation of the models is crucial for a precise and rigorous test of their knowledge . |
| Approach: | They propose a task to evaluate understanding of predicate-noun dependencies in a controlled setup. |
| Outcome: | This study compares state-of-the-art models with a case study on predicate-noun dependencies. |
CHICA: A Developmental Corpus of Child-Caregiver’s Face-to-face vs. Video Call Conversations in Middle Childhood (2024.lrec-main)
Copied to clipboard
Dhia Elhak Goumri, Abhishek Agrawal, Mitja Nikolaus, Hong Duc Thang Vu, Kübra Bodur, Elias Emmar, Cassandre Armand, Chiara Mazzocconi, Shreejata Gupta, Laurent Prévot, Benoit Favre, Leonor Becerra-Bonache, Abdellah Fourtassi
| Challenge: | Existing studies of language-in-interaction focus on the two ends of the developmental spectrum, i.e., early childhood and adulthood, leaving a gap in our knowledge about how development unfolds, especially across middle childhood. |
| Approach: | They propose to use CHICA to analyze child-caregiver conversations at home . they use mobile, lightweight eye-tracking and head motion detection to optimize the naturalness of the recordings. |
| Outcome: | The proposed corpus of child-caregiver conversations at home was compared with a previous corpus based on a set of conversations between children aged 7, 9, and 11 years old. |
Automatic Coding of Contingency in Child-Caregiver Conversations (2024.lrec-main)
Copied to clipboard
| Challenge: | Current research on children's language development relies on manual annotation of a small sample of children, which limits our ability to draw general conclusions about development. |
| Approach: | They propose to use automatic tools to assess contingency in children's natural interactions with caregivers by annotating a small set of data with a Transformer-based model. |
| Outcome: | The proposed model replicates existing results and generates new data-driven hypotheses. |
Learning English with Peppa Pig (2022.tacl-1)
Copied to clipboard
| Challenge: | Current approaches to model or simulate the acquisition of spoken language via grounding in perception are not generalizable to real-life situations that humans or adaptive artificial agents experience. |
| Approach: | They propose to use a dataset based on the children’s cartoon Peppa Pig to train a bi-modal architecture that learns aspects of the visual semantics of spoken language. |
| Outcome: | The proposed model learns to represent speech and visual data in a joint vector space. |
Automatic Annotation of Grammaticality in Child-Caregiver Conversations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing child language acquisition have been tedious and inconsistent. |
| Approach: | They propose a coding scheme for context-dependent grammaticality in child-caregiver conversations and annotate 4,000 utterances from a large corpus of transcribed conversations. |
| Outcome: | The proposed method achieves human inter-annotation agreement levels and is faster and reproducible than manual methods. |