About Me

Hi 👋, I am a final-year PhD student at Monash University, supervised by Asst. Prof. Bohan Zhuang and Prof. Jianfei Cai. I am a member of ZIP Lab. Previously I was a Master student at the University of Adelaide. Prior to that, I received my bachelors’ degree from Harbin Institute of Technology, Weihai, a beautiful coastal campus 🏖️, which left me with cherished memories. Here is my CV.

My research is all about efficency in deep neural networks, including training, inference, and deployment. Some topics that I currently focus on:

• Flexible model deployment: SN-Net
• Transformer architcture optimization: LIT, LITv2
• Efficient attention mechansims: HiLo, EcoFormer
• Token pruning/merging for inference speedup: HVT
• Memory-efficient training: Mesa

News

• 2023.10.25   I gave an online talk at University of Massachusetts Amherst.
• 2023.04.20   One paper is accepted by IJCAI 2023!
• 2023.04.11   I will be interning at NVIDIA Research this summer.
• 2023.03.22   SN-Net was selected as a highlight at CVPR 2023!🔥
• 2023.02.28   Two papers are accepted by CVPR 2023!
• 2022.11.11   Both LITv2 and Ecoformer will be presented as Spotlight!
• 2022.09.15   Our LITv2 and EcoFormer are accepted by NeurIPS 2022.
• 2022.07.04   Our paper STPT is accepted by ECCV 2022.
• 2021.12.01   Our paper LIT is accepted by AAAI 2022.
• 2021.07.23   Our paper HVT and ORIST are accepted by ICCV 2021.

Education

• Ph.D in Computer Science, Monash University, 2021 - Now.
• M.S. in Computer Science, The University of Adelaide, 2020.
• B.E. in Software Engineering, Harbin Institute of Technology, Weihai, 2019.

Work Experience

• Intern, NVIDIA, AI Algorithm Group, July, 2023 - Oct, 2023

Publications

A Survey on Efficient Training of Transformers

Bohan Zhuang, Jing Liu, Zizheng Pan, Haoyu He, Yuetian Weng, Chunhua Shen

International Joint Conference on Artificial Intelligence (IJCAI), 2023.

[Paper]

Stitchable Neural Networks

Zizheng Pan, Jianfei Cai, Bohan Zhuang

Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)

[Code], [Paper], [Project Page]

Dynamic Focus-aware Positional Queries for Semantic Segmentation

Haoyu He, Jianfei Cai, Zizheng Pan, Jing liu, Jing Zhang, Dacheng Tao and Bohan Zhuang.

Conference on Computer Vision and Pattern Recognition (CVPR), 2023

[Code], [Paper]

Fast Vision Transformers with HiLo Attention

Zizheng Pan, Jianfei Cai, Bohan Zhuang

Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)

[Code], [Paper], [OpenReview]

EcoFormer: Energy-Saving Attention with Linear Complexity

Jing Liu*, Zizheng Pan*, Haoyu He, Jianfei Cai, Bohan Zhuang (*equal contribution)

Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)

[Code], [Paper], [OpenReview]

An Efficient Spatio-Temporal Pyramid Transformer for Action Detection

Yuetian Weng, Zizheng Pan, Mingfei Han, Xiaojun Chang, Bohan Zhuang

European Conference on Computer Vision (ECCV), 2022

[Code], [Paper]

Less is More: Pay Less Attention in Vision Transformers

Zizheng Pan, Bohan Zhuang, Haoyu He, Jing Liu, Jianfei Cai

AAAI Conference on Artificial Intelligence (AAAI), 2022

[Code], [Paper]

Scalable Visual Transformers with Hierarchical Pooling

Zizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He, Jianfei Cai

International Conference on Computer Vision (ICCV), 2021

[Code], [Paper]

The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation

Yuankai Qi, Zizheng Pan, Yicong Hong, Ming-Hsuan Yang, Anton van den Hengel, Qi Wu

International Conference on Computer Vision (ICCV), 2021

[Code], [Paper]

Object-and-Action Aware Model for Visual Language Navigation

Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, Qi Wu

European Conference on Computer Vision (ECCV), 2020

[Paper]

Preprints

Mesa: A Memory-saving Training Framework for Transformers

Zizheng Pan, Peng Chen, Haoyu He, Jing Liu, Jianfei Cai, Bohan Zhuang

[Code], [Paper]

Pruning Self-attentions into Convolutional Layers in Single Path

Haoyu He, Jing Liu, Zizheng Pan, Jianfei Cai, Jing Zhang, Dacheng Tao, Bohan Zhuang

[Code], [Paper]

Teaching

• FIT5201 - Machine learning, 2022, TA

Talk

• 2023.10.25   “Optimizing Vision Transformers for Efficient Training, Inference and Deployment”, invited online talk at University of Massachusetts Amherst.

Professional Activities

Reviewer: CVPR/ICCV/ECCV/NeurIPS

Awards

• Google Travel and Conference Grants, 2023
• Monash Graduate Scholarship, 2020
• Adelaide Summer Research Scholarship, 2019
• Outstanding Graduate in Harbin Institute of Technology, 2019