Megh Thakkar, Quentin Fournier, et al.
ACL 2025
Part 1: Introduction of ZO-ML
Preliminary Concepts and Mathematical Foundations Basic mathematical tools and formulations Why ZO over FO: Limitations of Traditional Gradient-Based Optimization Emerging challenges and drawbacks of relying solely on FO gradientbased methods Survey of Practical Applications and Use Cases Overview of applications that benefit from ZOML
Part 2: Foundations of ZO-ML
Algorithmic Landscape of ZO-ML A rundown of primary algorithms and methods in ZOML Convergence and Query Complexity Understanding the provable properties of ZOML Scaling ZO-ML: Practical Techniques and Implementations Tips and tricks for ZOML algorithms at scale Extending ZO-ML across Learning Paradigms How does ZOML adapt to various ML paradigms? Break
Part 3: Applications of ZO-ML
Prompt Learning in FMs Fine-tuning and Personalization in FMs via ZO-ML ZO-ML in the Context of AI Robustness, Efficiency, and Automation
Part 4: Demo Expo
Introducing the ZO-ML Toolbox A guided tour of our specialized toolbox for ZOML Benchmarking with ZO algorithms An introduction to ZO performance metrics and benchmark applications Practical Demos: Utilizing ZOT for Parameter-Efficient Fine-Tuning (PEFT), and Adversarial Defense Live demonstrations showcasing the utility of ZOML
Part 5: Conclusion and Q&A
Wrap-Up: Key Takeaways from the Tutorial Future Horizons: SP and ML Opportunities and Challenges Resources for Deeper Exploration A curated list of essential ZOML resources
Megh Thakkar, Quentin Fournier, et al.
ACL 2025
Thomas Bohnstingl, Ayush Garg, et al.
ICASSP 2022
Jiaqi Han, Wenbing Huang, et al.
NeurIPS 2022
Wojciech Ozga, Do Le Quoc , et al.
IFIP DBSec 2021