Aviv Netanyahu

I am a PhD student in EECS at MIT CSAIL, advised by Prof. Pulkit Agrawal, as part of the Embodied Intelligence Community of Research.

I earned my MSc from the Faculty of Mathematics and Computer Science at the Weizmann Institute of Science, advised by Prof. Shimon Ullman, where I worked on Computer Vision. Before that, I received my BSc in Mathematics and Computer Science from the Hebrew University of Jerusalem.

Email  /  Twitter  /  LinkedIn

profile photo

I am interested in studying how to deploy agents in new environments with minimum to no supervision. My research focuses, in this context, on inferring how agents interact with the environment and with other agents in ways that are generalizable to new settings. Issues related to this research include combinatorial and out-of-distribution generalization, learning from demonstrations, human in the loop learning and object-centric learning.

PontTuset Learning to Extrapolate: A Transductive Approach
Aviv Netanyahu*, Abhishek Gupta*, Max Simchowitz, Kaiqing Zhang, Pulkit Agrawal
ICLR 2023 (In Submission)
NeurIPS Workshop on Distribution Shifts, 2022

Out-of-support generalization via transductive reparameteraization to within-support combinatorial generalization and bilinear embeddings.
PontTuset Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
Aviv Netanyahu*, Tianmin Shu*, Joshua B. Tenenbaum, Pulkit Agrawal
ICML 2022
RLDM 2022
ICLR Elements of Reasoning: Objects, Structure and Causality Workshop 2022
project page / arXiv / talk / code

Graph-based one-shot reward learning via active learning for object rearrangement tasks.

PontTuset PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Aviv Netanyahu*, Tianmin Shu*, Boris Katz, Andrei Barbu, Joshua B. Tenenbaum
AAAI 2021
NeurIPS Shared Visual Representations in Human & Machine Intelligence Workshop 2020 (Oral Presentation, Best Paper Award)
project page / arXiv / talk / code

A dataset for inferring physically grounded social interactions between two agents.

PontTuset Image Interpretation by Iterative Bottom-Up Top-Down Processing
Shimon Ullman, Liav Assif*, Alona Strugatski*, Ben-Zion Vatashsky, Hila Levi, Aviv Netanyahu, Adam Yaari
under review
arXiv / code

Extraction and representation of scene components, such as objects and their parts, people, and places, individual properties, and relations and interactions between them by combining bottom-up and top-down networks, interacting through a symmetric bi-directional communication between them.

PontTuset Cyclical Bottom-Up Top-Down Neural Networks for Relational Reasoning
MSc thesis 2018

A system for selective visual relationship detection that achieves out-of-distribution and combinatorial generalization.

TA, Machine Learning (6.867), MIT Fall 2021
Co-instructor, Deep Learning for Control (6.S090), MIT Winter 2021
Organizer Computational Sensorimotor Learning (CSL) Seminar, MIT 2021
Outreach Officer GW6 2020, Lecturer 'Rishonot BaMada' 2017-2019, Coding instructor 'Project Mehamemet' 2016-2019