Weijie Su

Short Bio

Weijie Su is a fourth-year PhD student at University of Science and Technology of China (USTC), under the supervision of Prof. Bin Li. Before that, he received his Bachelor’s degree from USTC in June 2019, majoring in Information and Computing Science (Computational Mathematics). Currently, he is also a research intern at SenseTime Research, mentored by Dr. Jifeng Dai and Dr. Xizhou Zhu.
Research Interests General Capable Agents; Representation Learning; Multimodal Learning; Visual Recognition.

He is currently seeking a research position to start in the summer of 2024, please feel free to drop him an email if you are interested in his research.

News

2023.05 Introducing GITM, a Generally Capable Agent that Fully Masters Minecraft. 2023.02 Our M3I Pre-training and SiameseIM are accepted by CVPR 2023. 2021.06 Presentation slides and video of our Deformable DETR on ICLR 2021 are available now. 2021.01 Deformable DETR is accepted for oral presentation by ICLR 2021. 2020.11 Code of Deformable DETR is now available at this github repository. 2020.05 Presentation slides and video of our VL-BERT on ICLR 2020 are available now. 2019.12 Our paper VL-BERT got accepted by ICLR 2020. 2019.11 Code for our paper VL-BERT has been made public available at github.

Experiences

2020.1 - Present Research Intern, SenseTime Research (Mentor: Dr. Jifeng Dai and Dr. Xizhou Zhu).

2018.7 - 2019.9 Research Intern, Microsoft Research Asia (Mentor: Dr. Jifeng Dai).

2017.7 - 2018.6 Research Intern, GCL Lab at USTC (Mentor: Prof. Ligang Liu).

Honors

National Scholarship, University of Science and Technology of China, 2020.

Excellent Bachelor Graduate, University of Science and Technology of China, 2019.

Bronze Medal, The 30th Chinese Mathematical Olympiad (CMO), 2014.

Publications

(^† Equal Contribution)

Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory [Project Page] [Blog] [Demo] Xizhou Zhu^†, Yuntao Chen^†, Hao Tian^†, Chenxin Tao^†, Weijie Su^†, Chenyu Yang^†, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, Jifeng Dai Arxiv Preprint, 2023.
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information [Code] [Slides] [Presentation] Weijie Su^†, Xizhou Zhu^†, Chenxin Tao^†, Lewei Lu, Bin Li, Gao Huang, Yu Qiao, Xiaogang Wang, Jie Zhou, Jifeng Dai IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Siamese Image Modeling for Self-Supervised Vision Representation Learning Chenxin Tao^†, Xizhou Zhu^†, Weijie Su^†, Gao Huang, Bin Li, Jie Zhou, Yu Qiao, Xiaogang Wang, Jifeng Dai IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Deformable DETR: Deformable Transformers for End-to-End Object Detection [Code] [Blog] [Slides] [Presentation] Xizhou Zhu^†, Weijie Su^†, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai International Conference on Learning Representations (ICLR), 2021. (Oral)
VL-BERT: Pre-training of Generic Visual-Linguistic Representations [Code] [Slides] [Presentation] Weijie Su^†, Xizhou Zhu^†, Yue Cao, Bin Li, Lewei Lu, Furu Wei, Jifeng Dai International Conference on Learning Representations (ICLR), 2020.