Artificial Neural Network Training on an Optical Processor via Direct Feedback Alignment
Citations Over Time
Abstract
Artificial Neural Networks (ANN) are habitually trained via the back-propagation (BP) algorithm. This approach has been extremely successful: Current models like GPT-3 have O(1011) parameters, are trained on O(1011) words and produce awe-inspiring results. However, there are good reasons to look for alternative training methods: With current algorithms and hardware constraints sometimes only half the available computing power is actually used. This is due to a complicated interplay between the size of the ANN, the available memory, throughput limitations of interconnects, the architecture of the network of computers, and the training algorithm. Training a model like the aforementioned GPT-3 takes months and costs millions. A different training paradigm, which could make clever use of specialized hardware, may train large ANNs more efficiently.
Related Papers
- → GENETIC ALGORITHM AND NEURAL NETWORK FOR OPTICAL CHARACTER RECOGNITION(2013)12 cited
- → Standard and Advanced Backpropagation Models for Image Processing Application in Traffic Engineering(2002)3 cited
- → Standard and Advanced Backpropagation Models for Image Processing Application in Traffic Engineering(2002)9 cited
- → Adaptability of the backpropagation procedure(2003)5 cited
- AN ADVANCED BACKPROPAGATION MODEL FOR APPLICATION IN TRAFFIC ENGINEERING(1998)