Quantong Qiu | Homepage

📝 Biography

- I am a 2nd-year Master student at the Institute of Artificial Intelligence, Soochow University, under the supervision of Assoc. Prof. Juntao Li.

- Before that, I obtained the Bachelor's degree in Software Engineering from Jiangxi University of Finance and Economics. During my undergraduate studies, I won more than ten national-level awards in Computer Science Category A competitions recognized by the Ministry of Education, and received honors including Outstanding Undergraduate Thesis and Outstanding Student Pacesetter.

- Currently, my research focuses on Efficient Inference for Large Language Models, including but not limited to KV Cache Optimization, Sparse Attention, and Model Optimization.

Check my publications and CV (中文 PDF) for more information!

Last Update: 2026-09-30 (github.io) 2026-09-30 (online CV)

📚 Research Interests

KV Cache Optimization

Quantization: OTT (ACL 2025 Long Main)
KV Cache Eviction: CaliDrop (AAAI 2025)

AI Infrastructure

Long Sequence Infrastructure: LOOM-Scope
Inference Engine: Meta-Infer (MetaStone, 2026)

Efficient Model Architectures

Sparse Attention: Elastic Attention (ICML 2026)
Hybrid Attention: Flux Attention (NeurIPS 2026)

📖 Education & Internship

Ms Student, Software Engineering, 2025-current
School of Computer Science and Technology, Soochow University

B.Eng., Software Engineering, 2021-2025
School of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics

Deployment Optimization Intern & Team Lead, 2026
MetaStone (是石科技)
Worked on Meta-Infer, an inference engine for heterogeneous accelerators; it lifted DeepSeek-V4.1-Flash input throughput from 1,932 to 13,274 tok/s (6.87×) on 8× PCIe-only GPUs.
Meta-Infer repo (coming soon) · QbitAI coverage (Sep 2026)

📒 Selected Publications

Check my Google Scholar for more information! (* stands for equal contribution)

[NeurIPS 2026 (CCF A)] Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
Quantong Qiu, Zhiyi Hong, Yi Yang, Haitian Wang, Kebin Liu, Qingqing Dang, Juntao Li, Min Zhang

arXiv

Code
[ICML 2026 Main (CCF A)] Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
Zecheng Tang*, Quantong Qiu*, Yi Yang, Zhiyi Hong, Haiya Xiang, Kebin Liu, Qingqing Dang, Juntao Li, Min Zhang

arXiv

Code
[EMNLP 2026 Main (CCF B)] MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios
Haitian Wang, Ruoxi Sun, Quantong Qiu, Juntao Li, Junhui Li, Hua Chen, Jinxiong Chang, Min Zhang

arXiv

Code
[ACL 2025 Long (CCF A)] OTT: Accurate KV Cache Quantization with Outlier Tokens Tracing
Yi Su*, Yuechi Zhou*, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, Min Zhang

arXiv

Code

📜 Selected Preprints

LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation framework
Zecheng Tang, Haitian Wang, Quantong Qiu, Baibei Ji, Ruoxi Sun, Keyan Zhou, Juntao Li, Min Zhang

arXiv

Code
Recovering Semantic Loss in Extreme KV Cache Compression via Query-Based Calibration
Quantong Qiu, Yi Su, Yuechi Zhou, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, Juntao Li, Min Zhang

arXiv

Code
LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling
Zecheng Tang, Baibei Ji, Quantong Qiu, Haitian Wang, Xiaobo Liang, Juntao Li, Min Zhang

arXiv

Code
Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads
Ruoxi Sun, Quantong Qiu, Juntao Li, Zecheng Tang, Yihang Lou, Min Zhang

arXiv

🎖️ Scholarship & Honors

The 25th China Robot and Artificial Intelligence Competition First Prize (National Level, Team Leader)

June 2023

Chinese Collegiate Computing Competition Second Prize (National Level, Team Leader)

The 17th (August 2024) & the 16th (August 2023)

Outstanding Student Pacesetter, JUFE University Honor (Only 10 Recipients University-wide)

2025

Outstanding Undergraduate Thesis, JUFE University Honor (Only 10 Recipients University-wide)