rwightman/pytorch-image-models: v0.1-attn-weights
Zenodo (CERN European Organization for Nuclear Research)2021
Ross Wightman, Aman Arora, Chris Ha, Alexander Soare, Mike Mike, contrastive, Mohammed Rizin, MrT23, Csaba Kertész, Dushyant Mehta, Guillem Cucurull, Kushajveer Singh, Andrew Lavin, Juntang Zhuang, Matthijs Hollemans, Sepehr Sameni, Vyacheslav Shults, Yusuke Uchida, Zhun Zhong, Comar, Taehoon Kim, Alexey Chernov, Greg Dongyoon Han, Eli Uriegas, Jasha10, Marcin Kardas, Michael Monashev, , Mike, Min‐Qin Chen, Mohamed Rashad
Abstract
A collection of weights I've trained comparing various types of SE-like (SE, ECA, GC, etc), self-attention (bottleneck, halo, lambda) blocks, and related non-attn baselines.