About Me

Hi 👋, I am a 2nd-year PhD student at Monash University, supervised by Asst. Prof. Bohan Zhuang and Prof. Jianfei Cai. I am a member of ZIP Lab. My research focuses on model compression and efficiency.

News

  • [2022.11.11] Both LITv2 and Ecoformer will be presented as Spotlight!
  • [2022.09.15] Our LITv2 and EcoFormer are accepted by NeurIPS 2022.
  • [2022.07.04] Our paper STPT is accepted by ECCV 2022.
  • [2021.12.01] Our paper LIT is accepted by AAAI 2022.
  • [2021.07.23] Our paper HVT and ORIST are accepted by ICCV 2021.

Education

  • Ph.D in Computer Science, Monash University, 2021 - Now.
  • M.S. in Computer Science, The University of Adelaide, 2020.
  • B.E. in Software Engineering, Harbin Institute of Technology, Weihai, 2019.

Publications

Fast Vision Transformers with HiLo Attention
Zizheng Pan, Jianfei Cai, Bohan Zhuang
Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)
[Code], [Paper], [OpenReview]

EcoFormer: Energy-Saving Attention with Linear Complexity
Jing Liu*, Zizheng Pan*, Haoyu He, Jianfei Cai, Bohan Zhuang (*equal contribution)
Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)
[Code], [Paper], [OpenReview]

An Efficient Spatio-Temporal Pyramid Transformer for Action Detection
Yuetian Weng, Zizheng Pan, Mingfei Han, Xiaojun Chang, Bohan Zhuang
European Conference on Computer Vision (ECCV), 2022
[Code], [Paper]

Less is More: Pay Less Attention in Vision Transformers
Zizheng Pan, Bohan Zhuang, Haoyu He, Jing Liu, Jianfei Cai
AAAI Conference on Artificial Intelligence (AAAI), 2022
[Code], [Paper]

Scalable Visual Transformers with Hierarchical Pooling
Zizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He, Jianfei Cai
International Conference on Computer Vision (ICCV), 2021
[Code], [Paper]

The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
Yuankai Qi, Zizheng Pan, Yicong Hong, Ming-Hsuan Yang, Anton van den Hengel, Qi Wu
International Conference on Computer Vision (ICCV), 2021
[Code], [Paper]

Object-and-Action Aware Model for Visual Language Navigation
Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, Qi Wu
European Conference on Computer Vision (ECCV), 2020
[Paper]

Preprints

Pruning Self-attentions into Convolutional Layers in Single Path
Haoyu He, Jing Liu, Zizheng Pan, Jianfei Cai, Jing Zhang, Dacheng Tao, Bohan Zhuang
[Code], [Paper]

Mesa: A Memory-saving Training Framework for Transformers
Zizheng Pan, Peng Chen, Haoyu He, Jing Liu, Jianfei Cai, Bohan Zhuang
[Code], [Paper]

Teaching

  • FIT5201 - Machine learning, 2022, TA

Professional Activities

Reviewer: CVPR/ICCV/ECCV/NeurIPS

Awards

  • Monash Graduate Scholarship, 2020
  • Adelaide Summer Research Scholarship, 2019
  • Outstanding Graduate in Harbin Institute of Technology, 2019