Aviv Netanyahu

I am a PhD candidate in EECS at MIT CSAIL, advised by Prof. Pulkit Agrawal, as part of the Embodied Intelligence Community of Research.

I earned my MSc from the Faculty of Mathematics and Computer Science at the Weizmann Institute of Science, advised by Prof. Shimon Ullman, where I worked on Computer Vision. Before that, I received my BSc in Mathematics and Computer Science from the Hebrew University of Jerusalem.

Email  /  Twitter  /  LinkedIn  /  Google Scholar

profile photo
Research

I am interested in studying how to deploy agents in new environments with minimum to no supervision. My research focuses, in this context, on inferring how agents interact with the environment and with other agents in ways that are generalizable to new settings. Issues related to this research include combinatorial and out-of-distribution generalization, learning from demonstrations, human in the loop learning and object-centric learning.

PontTuset Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules
Nofit Segal*, Aviv Netanyahu*, Kevin Greenman, Pulkit Agrawal, Rafael Gómez-Bombarelli
paper
NeurIPS Workshop on AI for Accelerated Materials Design, 2024 (Oral Presentation)
NeurIPS Workshop for Women in Machine Learning, 2024
Materials Research Society Fall Meeting, 2024
Molecular Machine Learning Conference, 2024

Extrapolating property prediction in materials science.
PontTuset Few-Shot Task Learning through Inverse Generative Modeling
Aviv Netanyahu, Yilun Du, Antonia Bronars, Jyothish Pari, Joshua B. Tenenbaum, Tianmin Shu, Pulkit Agrawal
project page / arXiv
NeurIPS 2024
RSS Workshop on Generative Modeling meets HRI, 2024
RSS Workshop on Task Specification for General-Purpose Intelligent Robots, 2024

Few-shot continual learning via generative modeling.
PontTuset Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-time Policy Adaptation
Andi Peng, Aviv Netanyahu, Mark Ho, Tianmin Shu, Andreea Bobu, Julie Shah, Pulkit Agrawal
project page / arXiv / video / MIT News (front page story)
ICML 2023
NeurIPS Workshop on Human in the Loop Learning, 2022

Counterfactual demonstrations for personalized policy adaptation.
PontTuset Learning to Extrapolate: A Transductive Approach
Aviv Netanyahu*, Abhishek Gupta*, Max Simchowitz, Kaiqing Zhang, Pulkit Agrawal
arXiv / video / code / MIT CSAIL Alliances (Improbable AI Lab Tour)
ICLR 2023
NeurIPS Workshop on Distribution Shifts, 2022

Out-of-support generalization via a transductive reparameterization.
PontTuset Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
Aviv Netanyahu*, Tianmin Shu*, Joshua B. Tenenbaum, Pulkit Agrawal
project page / arXiv / talk / code
ICML 2022
RLDM 2022
ICLR Elements of Reasoning: Objects, Structure and Causality Workshop, 2022

Graph-based reward learning via active learning for object rearrangement.

PontTuset PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Aviv Netanyahu*, Tianmin Shu*, Boris Katz, Andrei Barbu, Joshua B. Tenenbaum
project page / arXiv / talk / code
AAAI 2021
NeurIPS Shared Visual Representations in Human & Machine Intelligence Workshop, 2020 (Oral Presentation, Best Paper Award)

A benchmark for inferring physically grounded social interactions between agents.

PontTuset Human-Like Scene Interpretation by a Guided Counterstream Processing
Shimon Ullman, Liav Assif*, Alona Strugatski*, Ben-Zion Vatashsky, Hila Levi, Aviv Netanyahu, Adam Yaari
arXiv / code
PNAS 2023

Scene graph extraction with bottom-up top-down networks.

PontTuset Cyclical Bottom-Up Top-Down Neural Networks for Relational Reasoning
MSc thesis

A system for selective visual relationship detection that achieves out-of-distribution and combinatorial generalization.

Teaching
TA, Machine Learning (6.867), MIT Fall 2021
Co-instructor, Deep Learning for Control (6.S090), MIT Winter 2021
Project Mentor, Computational Cognitive Science (9.66), MIT Fall 2020
Grader, Logic and Set Theory (69), Reichman University Spring 2017
Grader and Mathematics Help Center Instructor, Linear Algebra 2 (80135), Hebrew University of Jerusalem Spring 2016
Service
Reviewer ICML 2024, NeurIPS 2023 (Top Reviewer) and workshops at RSS, ICRA and NeurIPS 2021-2023
Organizer Social Intelligence in Humans and Robots RSS 2024 Workshop, Computational Sensorimotor Learning Seminar, MIT 2021
Outreach Officer GW6 2020, Lecturer 'Rishonot BaMada' 2017-2019, Coding instructor 'Project Mehamemet' 2016-2019, Mentor for undergrad freshmen 'Shalom Shana Alef' 2014-2015

template