About Me

Hi 👋, I am a researcher at DeepSeek. I studied my PhD at ZIP Lab, Monash University, supervised by Prof. Bohan Zhuang and Prof. Jianfei Cai. Previously I was a master student at the University of Adelaide. Prior to that, I received my bachelors’ degree from Harbin Institute of Technology, Weihai, a beautiful coastal campus 🏖️, which left me with cherished memories. Here is my CV (usually outdated).

My research during PhD is all about efficiency in deep neural networks, including training, inference, and deployment. Some topics that I previously focused on:

• Flexible model deployment: SN-Net, SN-Netv2
• Transformer architcture optimization: LIT, LITv2
• Efficient attention mechansims: HiLo, EcoFormer
• Token pruning/merging for inference speedup: HVT
• Memory-efficient training: Mesa

If you are working on the following topics and interested in internship/full-time opportunities at DeepSeek, please feel free to contact me via email. We are always looking for talents!

• Multimodal LLMs
• Visual Generative Model
• Math/Code/LLM Alignment
• Agent

News

• 2024.09.26   One paper is accepted by NeurIPS 2024!
• 2024.07.01   First day at DeepSeek & One paper is accepted by ECCV 2024!
• 2023.12.29   One paper is accepted by CVPR 2024!
• 2023.12.29   One paper is accepted by TPAMI!
• 2023.10.25   I gave an online talk at University of Massachusetts Amherst.
• 2023.04.20   One paper is accepted by IJCAI 2023!
• 2023.04.11   I will be interning at NVIDIA Research this summer.
• 2023.03.22   SN-Net was selected as a highlight at CVPR 2023!🔥
• 2023.02.28   Two papers are accepted by CVPR 2023!
• 2022.11.11   Both LITv2 and Ecoformer will be presented as Spotlight!
• 2022.09.15   Our LITv2 and EcoFormer are accepted by NeurIPS 2022.
• 2022.07.04   Our paper STPT is accepted by ECCV 2022.
• 2021.12.01   Our paper LIT is accepted by AAAI 2022.
• 2021.07.23   Our paper HVT and ORIST are accepted by ICCV 2021.

Education

• Ph.D in Computer Science, Monash University, 2021 - 2024.
• M.S. in Computer Science, The University of Adelaide, 2018 - 2020.
• B.E. in Software Engineering, Harbin Institute of Technology, Weihai, 2015 - 2019.

Work Experience

• Full-time researcher at DeepSeek, July, 2024 - Now.
• Research intern at NVIDIA, AI Algorithm Group, July, 2023 - Oct, 2023

Research

Stitched ViTs are Flexible Vision Backbones

Zizheng Pan, Jing Liu, Haoyu He, Jianfei Cai, Bohan Zhuang

European Conference on Computer Vision (ECCV), 2024

[Paper] [Code] [Project Page]

PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reduction

Shangyu Chen, Zizheng Pan, Jianfei Cai, Dinh Phung

Arxiv, 2024

[Paper]

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models

Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, Bohan Zhuang

Conference on Neural Information Processing Systems (NeurIPS), 2024

[Paper]

T-Stitch: Accelerating Sampling in Pre-trained Diffusion Models with Trajectory Stitching

Zizheng Pan, Bohan Zhuang, De-An Huang, Weili Nie, Zhiding Yu, Chaowei Xiao, Jianfei Cai, Anima Anandkumar

ArXiv, 2024. [Paper] [Code] [Project Page]

Efficient Stitchable Task Adaptation

Haoyu He, Zizheng Pan, Jing Liu, Jianfei Cai, Bohan Zhuang

Conference on Computer Vision and Pattern Recognition (CVPR), 2024

[Paper] [Code]

Pruning Self-attentions into Convolutional Layers in Single Path

Haoyu He, Jing Liu, Zizheng Pan, Jianfei Cai, Jing Zhang, Dacheng Tao, Bohan Zhuang

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.

[Code], [Paper]

A Survey on Efficient Training of Transformers

Bohan Zhuang, Jing Liu, Zizheng Pan, Haoyu He, Yuetian Weng, Chunhua Shen

International Joint Conference on Artificial Intelligence (IJCAI), 2023.

[Paper]

Stitchable Neural Networks

Zizheng Pan, Jianfei Cai, Bohan Zhuang

Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)

[Code], [Paper], [Project Page]

Dynamic Focus-aware Positional Queries for Semantic Segmentation

Haoyu He, Jianfei Cai, Zizheng Pan, Jing liu, Jing Zhang, Dacheng Tao and Bohan Zhuang.

Conference on Computer Vision and Pattern Recognition (CVPR), 2023

[Code], [Paper]

Fast Vision Transformers with HiLo Attention

Zizheng Pan, Jianfei Cai, Bohan Zhuang

Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)

[Code], [Paper], [OpenReview]

EcoFormer: Energy-Saving Attention with Linear Complexity

Jing Liu*, Zizheng Pan*, Haoyu He, Jianfei Cai, Bohan Zhuang (*equal contribution)

Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight)

[Code], [Paper], [OpenReview]

An Efficient Spatio-Temporal Pyramid Transformer for Action Detection

Yuetian Weng, Zizheng Pan, Mingfei Han, Xiaojun Chang, Bohan Zhuang

European Conference on Computer Vision (ECCV), 2022

[Code], [Paper]

Mesa: A Memory-saving Training Framework for Transformers

Zizheng Pan, Peng Chen, Haoyu He, Jing Liu, Jianfei Cai, Bohan Zhuang

Arxiv, 2021

[Code], [Paper]

Less is More: Pay Less Attention in Vision Transformers

Zizheng Pan, Bohan Zhuang, Haoyu He, Jing Liu, Jianfei Cai

AAAI Conference on Artificial Intelligence (AAAI), 2022

[Code], [Paper]

Scalable Visual Transformers with Hierarchical Pooling

Zizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He, Jianfei Cai

International Conference on Computer Vision (ICCV), 2021

[Code], [Paper]

The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation

Yuankai Qi, Zizheng Pan, Yicong Hong, Ming-Hsuan Yang, Anton van den Hengel, Qi Wu

International Conference on Computer Vision (ICCV), 2021

[Code], [Paper]

Object-and-Action Aware Model for Visual Language Navigation

Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, Qi Wu

European Conference on Computer Vision (ECCV), 2020

[Paper]

Teaching

• FIT5201 - Machine learning, 2022, TA

Talk

• 2023.10.25   “Optimizing Vision Transformers for Efficient Training, Inference and Deployment”, invited online talk at University of Massachusetts Amherst.

Professional Activities

Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, IJCV

Awards

• Google Travel and Conference Grants, 2023
• Monash Graduate Scholarship, 2020
• Adelaide Summer Research Scholarship, 2019
• Outstanding Graduate in Harbin Institute of Technology, 2019