At this week’s MCQLL meeting, Benjamin LeBrun will be presenting Inferring meaning from visually grounded language via symbolic generative programs.” An abstract follows.
We will be meeting this Tuesday, March 21 at 3:00 PM. Meetings will be held both in person in room 117 of the McGill Linguistics department at 1085 Dr Penfield and on Zoom here.
Title: Inferring meaning from visually grounded language via symbolic generative programs
Abstract: Humans can infer precise meanings from natural language statements about the physical world in real time, and can learn new lexical concepts from just a single positive example. In contrast, large multi-modal neural models struggle to infer precise meanings for language making reference to the external world. This talk will present ongoing work which integrates insights from cognitive science and probabilistic programming to ground the meaning of natural language expressions in symbolic representations of 3D scenes. I will present prototype models which can infer precise meanings for spatial language where large multi-modal models fail and predict the time-course of grounded online language processing, as well as preliminary work towards a model which can learn meanings for nouns given a single ambiguous example.