A Data-Based Perspective on Transfer Learning
Citations Over TimeTop 10% of 2023 papers
Abstract
It is commonly believed that in transfer learning including more pre-training data translates into better performance. However, recent evidence suggests that removing data from the source dataset can actually help too. In this work, we take a closer look at the role of the source dataset's composition in transfer learning and present a framework for probing its impact on downstream performance. Our framework gives rise to new capabilities such as pinpointing transfer learning brittleness as well as detecting pathologies such as data-leakage and the presence of misleading examples in the source dataset. In particular, we demonstrate that removing detrimental datapoints identified by our framework indeed improves transfer learning performance from ImageNet on a variety of target tasks. 1 1 Code is available at https://github.com/MadryLab/data-transfer
Related Papers
- → Multi-Source Distilling Domain Adaptation(2020)208 cited
- → Towards demystifying dimensions of source code embeddings(2020)3 cited
- Source Code Plagiarism Detection 'SCPDet': A Review(2014)
- → Multi-source Distilling Domain Adaptation(2019)7 cited
- → Evaluating the Values of Sources in Transfer Learning(2021)1 cited