Benchmarking AlphaFold for protein complex modeling reveals accuracy determinants

Citations Over TimeTop 1% of 2022 papers

Abstract

High-resolution experimental structural determination of protein-protein interactions has led to valuable mechanistic insights, yet due to the massive number of interactions and experimental limitations there is a need for computational methods that can accurately model their structures. Here we explore the use of the recently developed deep learning method, AlphaFold, to predict structures of protein complexes from sequence. With a benchmark of 152 diverse heterodimeric protein complexes, multiple implementations and parameters of AlphaFold were tested for accuracy. Remarkably, many cases (43%) had near-native models (medium or high critical assessment of predicted interactions accuracy) generated as top-ranked predictions by AlphaFold, greatly surpassing the performance of unbound protein-protein docking (9% success rate for near-native top-ranked models), however AlphaFold modeling of antibody-antigen complexes within our set was unsuccessful. We identified sequence and structural features associated with lack of AlphaFold success, and we also investigated the impact of multiple sequence alignment input. Benchmarking of a multimer-optimized version of AlphaFold (AlphaFold-Multimer) with a set of recently released antibody-antigen structures confirmed a low rate of success for antibody-antigen complexes (11% success), and we found that T cell receptor-antigen complexes are likewise not accurately modeled by that algorithm, showing that adaptive immune recognition poses a challenge for the current AlphaFold algorithm and model. Overall, our study demonstrates that end-to-end deep learning can accurately model many transient protein complexes, and highlights areas of improvement for future developments to reliably model any protein-protein interaction of interest.

Related Papers

• Strategic Benchmarking: How to Rate Your Company's Performance against the World's Best(1993)
• → AI as a Sport: On the Competitive Epistemologies of Benchmarking(2024)9 cited
• → COMPARISON OF BEST PRACTICE BENCHMARKING MODELS(2011)11 cited
• → A guide for mental health clinicians to develop and undertake benchmarking activities(2010)2 cited
• Theoretical Aspects of Benchmarking Theory(2004)