About Me
Hi 👋, I am a final-year PhD student at Monash University, supervised by Asst. Prof. Bohan Zhuang and Prof. Jianfei Cai. I am a member of ZIP Lab. Previously I was a Master student at the University of Adelaide. Prior to that, I received my bachelors’ degree from Harbin Institute of Technology, Weihai, a beautiful coastal campus 🏖️, which left me with cherished memories. Here is my CV.
My research is all about efficency in deep neural networks, including training, inference, and deployment. Some topics that I currently focus on:
- Flexible model deployment: SN-Net, SN-Netv2
- Transformer architcture optimization: LIT, LITv2
- Efficient attention mechansims: HiLo, EcoFormer
- Token pruning/merging for inference speedup: HVT
- Memory-efficient training: Mesa
I will join DeepSeek as a full-time researcher in July 2024.
News
- 2023.12.29 One paper is accepted by CVPR 2024!
- 2023.12.29 One paper is accepted by TPAMI!
- 2023.10.25 I gave an online talk at University of Massachusetts Amherst.
- 2023.04.20 One paper is accepted by IJCAI 2023!
- 2023.04.11 I will be interning at NVIDIA Research this summer.
- 2023.03.22 SN-Net was selected as a highlight at CVPR 2023!🔥
- 2023.02.28 Two papers are accepted by CVPR 2023!
- 2022.11.11 Both LITv2 and Ecoformer will be presented as Spotlight!
- 2022.09.15 Our LITv2 and EcoFormer are accepted by NeurIPS 2022.
- 2022.07.04 Our paper STPT is accepted by ECCV 2022.
- 2021.12.01 Our paper LIT is accepted by AAAI 2022.
- 2021.07.23 Our paper HVT and ORIST are accepted by ICCV 2021.
Education
- Ph.D in Computer Science, Monash University, 2021 - Now.
- M.S. in Computer Science, The University of Adelaide, 2020.
- B.E. in Software Engineering, Harbin Institute of Technology, Weihai, 2019.
Work Experience
- Intern, NVIDIA, AI Algorithm Group, July, 2023 - Oct, 2023
Research
- T-Stitch: Accelerating Sampling in Pre-trained Diffusion Models with Trajectory Stitching
- Zizheng Pan, Bohan Zhuang, De-An Huang, Weili Nie, Zhiding Yu, Chaowei Xiao, Jianfei Cai, Anima Anandkumar
- ArXiv, 2024. [Paper] [Code] [Project Page]
- Stitched ViTs are Flexible Vision Backbones
- Zizheng Pan, Jing Liu, Haoyu He, Jianfei Cai, Bohan Zhuang
- ArXiv, 2023.
- [Paper] [Code] [Project Page]
- Efficient Stitchable Task Adaptation
- Haoyu He, Zizheng Pan, Jing Liu, Jianfei Cai, Bohan Zhuang
- Conference on Computer Vision and Pattern Recognition (CVPR), 2024
- [Paper] [Code]
- Pruning Self-attentions into Convolutional Layers in Single Path
- Haoyu He, Jing Liu, Zizheng Pan, Jianfei Cai, Jing Zhang, Dacheng Tao, Bohan Zhuang
- IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.
- [Code], [Paper]
- A Survey on Efficient Training of Transformers
- Bohan Zhuang, Jing Liu, Zizheng Pan, Haoyu He, Yuetian Weng, Chunhua Shen
- International Joint Conference on Artificial Intelligence (IJCAI), 2023.
- [Paper]
- Stitchable Neural Networks
- Zizheng Pan, Jianfei Cai, Bohan Zhuang
- Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
- [Code], [Paper], [Project Page]
- Dynamic Focus-aware Positional Queries for Semantic Segmentation
- Haoyu He, Jianfei Cai, Zizheng Pan, Jing liu, Jing Zhang, Dacheng Tao and Bohan Zhuang.
- Conference on Computer Vision and Pattern Recognition (CVPR), 2023
- [Code], [Paper]
- Fast Vision Transformers with HiLo Attention
- Zizheng Pan, Jianfei Cai, Bohan Zhuang
- Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)
- [Code], [Paper], [OpenReview]
- EcoFormer: Energy-Saving Attention with Linear Complexity
- Jing Liu*, Zizheng Pan*, Haoyu He, Jianfei Cai, Bohan Zhuang (*equal contribution)
- Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)
- [Code], [Paper], [OpenReview]
- An Efficient Spatio-Temporal Pyramid Transformer for Action Detection
- Yuetian Weng, Zizheng Pan, Mingfei Han, Xiaojun Chang, Bohan Zhuang
- European Conference on Computer Vision (ECCV), 2022
- [Code], [Paper]
- Mesa: A Memory-saving Training Framework for Transformers
- Zizheng Pan, Peng Chen, Haoyu He, Jing Liu, Jianfei Cai, Bohan Zhuang
- Arxiv, 2021
- [Code], [Paper]
- Less is More: Pay Less Attention in Vision Transformers
- Zizheng Pan, Bohan Zhuang, Haoyu He, Jing Liu, Jianfei Cai
- AAAI Conference on Artificial Intelligence (AAAI), 2022
- [Code], [Paper]
- Scalable Visual Transformers with Hierarchical Pooling
- Zizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He, Jianfei Cai
- International Conference on Computer Vision (ICCV), 2021
- [Code], [Paper]
- The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
- Yuankai Qi, Zizheng Pan, Yicong Hong, Ming-Hsuan Yang, Anton van den Hengel, Qi Wu
- International Conference on Computer Vision (ICCV), 2021
- [Code], [Paper]
- Object-and-Action Aware Model for Visual Language Navigation
- Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, Qi Wu
- European Conference on Computer Vision (ECCV), 2020
- [Paper]
Teaching
- FIT5201 - Machine learning, 2022, TA
Talk
- 2023.10.25 “Optimizing Vision Transformers for Efficient Training, Inference and Deployment”, invited online talk at University of Massachusetts Amherst.
Professional Activities
Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, IJCV
Awards
- Google Travel and Conference Grants, 2023
- Monash Graduate Scholarship, 2020
- Adelaide Summer Research Scholarship, 2019
- Outstanding Graduate in Harbin Institute of Technology, 2019