Yi Li

Biography

I am a Member of Technical Staff at xAI on the pretraining team. My work focuses on web-scale data processing, quality filtering, interleaved web corpora, token efficiency, and training-time robustness for frontier LLM pretraining.

Before joining xAI, I earned my Ph.D. at the University of Washington under the guidance of Prof. Dieter Fox. My doctoral research focused on developing multi-modal foundation models capable of understanding and interacting with the real world.

During my Ph.D., I interned at the NVIDIA AI Robotics Research Lab, where I developed HAMSTER, a hierarchical vision-language-action model for open-world robot manipulation, under the mentorship of Dr. Ankit Goyal.

Previously, I received my M.S. from Tsinghua University, advised by Prof. Xiangyang Ji, and my B.S. from Tsinghua University. I also collaborated with Dr. Jifeng Dai and Dr. Kaiming He at Microsoft Research Asia on deep learning and computer vision research.

News

Joined xAI Pretraining as a Member of Technical Staff, working on trillion-token-scale web data pipelines.

Invited to give a talk at OpenAI, Google DeepMind, and ByteDance Seed for our project HAMSTER.

Invited speaker on humanoid robot cognition, hosted by Bernstein.

Our paper HAMSTER has been accepted by ICLR 2025! Check our paper, demo, and more details on the Project Page.

Our paper STOW has been accepted by CoRL 2023! Code and dataset are released on GitHub.

Our paper DeepIM has been accepted by ECCV 2018 as oral! Code is released on GitHub. DeepIM was also selected as one of the 12 best papers at ECCV 2018, with an extended version published in IJCV.

Our paper Deformable ConvNets has been accepted by ICCV 2017. Code is released on GitHub. There is also a cool third-party improvement showing the best single-model performance on COCO dataset.

Awarded Summa Cum Laude in Beijing (top 2% at Tsinghua University).

Our paper FCIS has been accepted by the CVPR 2017 as spotlight! Check out the code on GitHub

We won first prize in MS COCO segmentation challenge 2016.

Our paper R-FCN has been accepted by the NeurIPS 2016 conference, and code is released on GitHub.

Our paper Inst-FCN has been accepted by the ECCV 2016 conference.

Awarded the National Scholarship (top 0.5% nationwide).

Selected Publications

(⁺ indicates co-first author)

HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation
Yi Li*, Yuquan Deng*, Jesse Zhang*, Joel Jang, Marius Memmel, Caelan Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal
International Conference on Learning Representations (ICLR), 2025
PDF, Bibtex, Project Page, Demo

@article{li2025hamster,
  title={HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation},
  author={Li, Yi and Deng, Yuquan and Zhang, Jesse and Jang, Joel and Memme, Marius and Yu, Raymond and Garrett, Caelan Reed and Ramos, Fabio and Fox, Dieter and Li, Anqi and others},
  journal={arXiv preprint arXiv:2502.05485},
  year={2025}
}

STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots
Yi Li, Muru Zhang, Markus Grotz, Kaichun Mo, Dieter Fox
In Conference on Robot Learning (CoRL), 2023
PDF, Bibtex, Poster, Spotlight Video, Project Page

@inproceedings{
  li2023stow,
  title={{STOW}: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots},
  author={Yi Li and Muru Zhang and Markus Grotz and Kaichun Mo and Dieter Fox},
  booktitle={7th Annual Conference on Robot Learning},
  year={2023},
  url={https://openreview.net/forum?id=48qUHKUEdBf}
  }

DeepIM: Deep Iterative Matching for 6D Pose Estimation
Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, Dieter Fox
In International Journal of Computer Vision (IJCV), 2020
In European Conference on Computer Vision (ECCV), 2018 (oral).
PDF, Bibtex, Code Poster Slides Project Page IJCV PDF

@article{li2020deepim,
  title={DeepIM: Deep Iterative Matching for 6D Pose Estimation},
  author={Li, Yi and Wang, Gu and Ji, Xiangyang and Xiang, Yu and Fox, Dieter},
  journal={International Journal of Computer Vision},
  volume={128},
  number={3},
  pages={657--678},
  year={2020},
  publisher={Springer}
}

Deformable Convolutional Networks
Jifeng Dai⁺, Haozhi Qi⁺, Yuwen Xiong⁺, Yi Li⁺, Guodong Zhang⁺, Han Hu, and Yichen Wei
In International Conference on Computer Vision (ICCV), 2017 (oral).
PDF, Bibtex, Code

@inproceedings{dai2017deformable,
  title={Deformable convolutional networks},
  author={Dai, Jifeng and Qi, Haozhi and Xiong, Yuwen and Li, Yi and Zhang, Guodong and Hu, Han and Wei, Yichen},
  booktitle={Proceedings of the IEEE international conference on computer vision},
  pages={764--773},
  year={2017}
}

Fully Convolutional Instance-aware Semantic Segmentation
Yi Li⁺, Haozhi Qi⁺, Jifeng Dai, Xiangyang Ji, and Yichen Wei
In Computer Vision and Patter Recognition (CVPR), 2017 (spotlight)
PDF, Bibtex, Code Visualize Results, Slides

@inproceedings{li2017fully,
  title={Fully convolutional instance-aware semantic segmentation},
  author={Li, Yi and Qi, Haozhi and Dai, Jifeng and Ji, Xiangyang and Wei, Yichen},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={2359--2367},
  year={2017}
}

R-fcn: Object detection via region-based fully convolutional networks
Jifeng Dai, Yi Li, Kaiming He, Jian Sun
In Advances in Neural Information Processing Systems (NeurIPS), 2016
PDF, Bibtex, Code, Poster

@inproceedings{dai2016r,
  title={R-fcn: Object detection via region-based fully convolutional networks},
  author={Dai, Jifeng and Li, Yi and He, Kaiming and Sun, Jian},
  booktitle={Advances in Neural Information Processing Systems},
  pages={379--387},

year={2016}
}

Instance-sensitive fully convolutional networks
Jifeng Dai, Kaiming He, Yi Li, Shaoqing Ren, Jian Sun
In European Conference on Computer Vision (ECCV), 2016
PDF, Bibtex, Poster

@inproceedings{dai2016instance,
title={Instance-sensitive fully convolutional networks},
author={Dai, Jifeng and He, Kaiming and Li, Yi and Ren, Shaoqing and Sun, Jian},
booktitle={European Conference on Computer Vision},
pages={534--549},
year={2016},
organization={Springer International Publishing}
}

Honor and Awards

Reviewer for AAAI, ICCV, CVPR, ECCV, ICRA, IROS, ICLR, and RA-L since 2017

Outstanding 2017 Master's Thesis by the Chinese Institute of Electronics (top 10 in China)

2017 Summa Cum Laude in Beijing (top 2% at Tsinghua University)

1st Prize in MS COCO 2016 Object Detection Challenge

Outstanding 2016 Intern in MSRA

2013 National Scholarship (top 0.5% nationwide)

2012 Tsinghua Alumni Zheng Geru Scholarship