DISC: A Dynamic Shape Compiler for Machine Learning Workloads
Citations Over Time
Abstract
Many recent machine learning models show dynamic shape characteristics. However, existing AI compiler optimization systems suffer a lot from problems brought by dynamic shape models, including compilation overhead, memory usage, optimization pipeline and deployment complexity. This paper provides a compiler system to natively support optimization for dynamic shape workloads, named DISC. DISC enriches a set of IR to form a fully dynamic shape representation. It generates the runtime flow at compile time to support processing dynamic shape based logic, which avoids the interpretation overhead at runtime and enlarges the opportunity of host-device co-optimization. It addresses the kernel fusion problem of dynamic shapes with shape propagation and constraints collecting methods. This is the first work to demonstrate how to build an end-to-end dynamic shape compiler based on MLIR infrastructure. Experiments show that DISC achieves up to 3.3x speedup than TensorFlow/PyTorch, and 1.8x than Nimble.
Related Papers
- → Use of Previously Acquired Positioning of Optimizations for Phase Ordering Exploration(2015)21 cited
- → The use of compiler optimizations for embedded systems software(2008)7 cited
- Application Performance Evaluation on Different Compiler Optimizations(2013)
- → Effectiveness of an optimizing compiler for arithmetic expressions(1970)1 cited
- → Effectiveness of an optimizing compiler for arithmetic expressions(1970)