Tyler Zhu

I am a 5th year Master's student in EECS at UC Berkeley advised by Jitendra Malik. I received my B.S. in EECS from Berkeley in 2022. I am broadly interested in computer vision, especially in drawing from human cognition to create visual systems whiich are effective and robust.

While at Berkeley, I've had the great fortune to have collaborated with and been mentored by a number of wonderful people, including Karttikeya Mangalam, Alvin Wan, and Dan Hendrycks. I was also heavily involved in teaching and outreach, serving on CS 70 course staff multiple times and previously leading Machine Learning @ Berkeley. You can find out more from my main website here.

CV  /  Google Scholar  /  Twitter  /  Github

profile photo
Research

I'm broadly interested in computer vision, especially in drawing from human cognition to create visual systems whiich are effective and robust.

Humans have incredibly proficient visual systems. As examples, they are highly adaptive to new information and settings and can accurately track objects even through occlusions. I am interested in understanding how we can replicate such capabilities in machines to teach them to see as we do, especially drawing from psychology and cognition for inspiration.

My goal is to be able to develop flexible and general learners which can learn efficiently from data. Some trends in this direction are scalable methods of self-supervised learning, robustness to distribution shift in real-world deployment, and utilizing rich visual priors in data (in some happy accordance with the Bitter Lesson).

Permutation Modeling for Pretraining Vision Transformers
Tyler Zhu
Technical Report, 2022
code

We investigate the use of causal and permutation modeling as pretraining objectives for vision transformers, finding that they can performance on par with masked image modeling (MIM).

Parallelized Reversible Vision Transformers
Tyler Zhu
Technical Report, 2022
code

A simple extension of Reversible Vision Transformers that parallelizes the backward pass using CUDA streams with a study into when these benefits are tangible.

The many faces of robustness; A critical analysis of out-of-distribution generalization.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer
ICCV 2021
code

Four new datasets measuring real-world distribution shifts, as well as a new state-of-the-art data augmentation method that outperforms models pretrained with 1000x more labeled data.

Misc
Preference Learning for Text-to-Image Prompt Tuning with RL
Arnav Gudibande*, Tyler Zhu*
Fall 2022 CS 285 Deep RL Final Project

We propose a framework towards automating prompt tuning for learning preferences iin text-to-image synthesis using reinforcement learning with human feedback.


Source code taken from Jon Barron's lovely website.