Samyam Rajbhandari
Bellevue Hospital Center(US)
Publications by Year
Research Areas
Advanced Neural Network Applications, Topic Modeling, Parallel Computing and Optimization Techniques, Natural Language Processing Techniques, Stochastic Gradient Optimization Techniques
Most-Cited Works
- → ZeRO: Memory optimizations Toward Training Trillion Parameter Models(2020)697 cited
- → DeepSpeed(2020)672 cited
- → Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model(2022)299 cited
- → DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale(2022)208 cited
- → ZeRO-infinity(2021)205 cited
- International Conference on Computational Science, ICCS 2012.(2012)
- → Learning Intrinsic Sparse Structures within Long Short-Term Memory(2017)103 cited
- ZeRO: Memory Optimization Towards Training A Trillion Parameter Models.(2019)
- → ZeRO-Offload: Democratizing Billion-Scale Model Training(2021)61 cited