Qi Zhang

About

I am a fourth-year PhD student at Peking University, advised by Yisen Wang. My research centers on self-supervised learning 🧩, interpretability 🔍 , and post-training 🚀 of foundation models. I am expected to graduate in 2027 and am seeking opportunities starting Summer 2027. Feel free to reach out if you would like to chat, discuss research, or explore potential collaborations.

Publications

An Augmentation Overlap Theory of Contrastive Learning
Q Zhang*, Y Wang*, Y Wang
JMLR 2025
Proposes a theoretical framework based on augmentation overlap for contrastive learning. A preliminary work was published at ICLR 2022 and Cited over 150 times.
PDF | Code 🧩

Proposes a new theoretical framework based on augmentation overlap for contrastive learning.

Chaos is a ladder: A new theoretical understanding of contrastive learning via augmentation overlap
Y Wang*, Q Zhang*, Y Wang, J Yang, Z Lin
ICLR 2022
We derived tight generalization bounds for contrastive learning with a new realistic theoretical framework. Cited over 150 times.
PDF | Code

A new theoretical understanding of contrastive learning using augmentation overlap.

🧩

How mask matters: Towards theoretical understandings of masked autoencoders
Q Zhang*, Y Wang*, Y Wang
NeurIPS 2022 (Spotlight)
Theoretical explanation of masked autoencoders and connections to joint embedding methods. Cited over 120 times
PDF | Code

Theoretical explanation of masked autoencoders and connections to joint embedding methods.

🧩

On the generalization of multi-modal contrastive learning
Q Zhang*, Y Wang*, Y Wang
ICML 2023
Establishes theoretical generalization guarantees for multi-modal contrastive learning.
PDF | Code 🧩

A message passing perspective on learning dynamics of contrastive learning
Y Wang*, Q Zhang*, T Du, J Yang, Z Lin, Y Wang
ICLR 2023
Analyzes contrastive learning dynamics from a message passing viewpoint
PDF | Code

Analyzes contrastive learning dynamics from a message passing viewpoint.

🧩

Identifiable contrastive learning with automatic feature importance discovery
Q Zhang*, Y Wang*, Y Wang
NeurIPS 2023
Proposes identifiable contrastive learning with automatic discovery of important features.
PDF | Code

Proposes identifiable contrastive learning with automatic discovery of important features.

🧩 🔍

Non-negative Contrastive Learning
Y Wang*, Q Zhang*, Y Guo, Y Wang
ICLR 2024
Introduces non-negative constraints to contrastive learning
PDF | Code

Introduces non-negative constraints to contrastive learning.

🧩 🔍

Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
Q Zhang*, T Du*, H Huang, Y Wang, Y Wang
ICML 2024
Theoretical comparison of GPT-style autoregressive and BERT-style masked pretraining.
PDF | Code

Theoretical comparison of GPT-style autoregressive and BERT-style masked pretraining.

🧩

Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
Q Zhang*, Y Wang*, J Cui, X Pan, Q Lei, S Jegelka, Y Wang
ICLR 2025
Shows the benefits of monosemantic features on robustness beyond interpretability.
PDF | Code

Shows the benefits of monosemantic features on robustness beyond interpretability.

🔍 🚀

Projection Head is Secretly an Information Bottleneck
Z Ouyang, K Hu, Q Zhang, Y Wang, Y Wang
ICLR 2025
Reveals that projection heads act as information bottlenecks in contrastive learning
PDF | Code

Reveals that projection heads act as information bottlenecks in contrastive learning.

🧩

On the Theoretical Understanding of Identifiable Sparse Autoencoders and Beyond
J Cui*, Q Zhang*, Y Wang, Y Wang
ICLR 2026
Theoretical analysis of sparse autoencoders.
PDF | Code

Theoretical analysis of identifiable sparse autoencoders and extensions.

🔍

SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Q Zhang*, Y Wang*, X Wang, J Chai, G Yin, W Lin, Y Wang
ICLR 2026
Uses SAE features to predict cross-domain transferability of large language models without training.
PDF | Code

Uses SAE features to predict cross-domain transferability of large language models without further training.

🔍 🚀

SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
X Guo, R Zhou, Y Wang, Q Zhang, C Zhang, S Jegelka, X Wang, J Chai, ...
arXiv preprint arXiv:2510.16416, 2025
Revisits self-supervised learning as an intrinsic reward for visual-language reasoning tasks
PDF | Code

Revisits self-supervised learning as an intrinsic reward for visual-language reasoning tasks.

🧩 🚀

Education

Sun Yat-sen University – School of Computer Science
Bachelor of Science in Information and Computational Science
Sep 2018 – Jun 2022
GPA: 91.72/100, Rank: 1/57
Honors & Awards:

National Scholarship (2019)
Sun Yat-sen University First-class Scholarship (2019)
Sun Yat-sen University Second-class Scholarship (2020)

Peking University – School of Intelligence
Ph.D. in Computer Science and Technology (Intelligent Science and Technology)
Sep 2022 – Present