Continuous Automated Model EvaluatiOn (CAMEO) complementing the critical assessment of structure prediction in CASP12

Citations Over TimeTop 10% of 2017 papers

Abstract

Every second year, the community experiment "Critical Assessment of Techniques for Structure Prediction" (CASP) is conducting an independent blind assessment of structure prediction methods, providing a framework for comparing the performance of different approaches and discussing the latest developments in the field. Yet, developers of automated computational modeling methods clearly benefit from more frequent evaluations based on larger sets of data. The "Continuous Automated Model EvaluatiOn (CAMEO)" platform complements the CASP experiment by conducting fully automated blind prediction assessments based on the weekly pre-release of sequences of those structures, which are going to be published in the next release of the PDB Protein Data Bank. CAMEO publishes weekly benchmarking results based on models collected during a 4-day prediction window, on average assessing ca. 100 targets during a time frame of 5 weeks. CAMEO benchmarking data is generated consistently for all participating methods at the same point in time, enabling developers to benchmark and cross-validate their method's performance, and directly refer to the benchmarking results in publications. In order to facilitate server development and promote shorter release cycles, CAMEO sends weekly email with submission statistics and low performance warnings. Many participants of CASP have successfully employed CAMEO when preparing their methods for upcoming community experiments. CAMEO offers a variety of scores to allow benchmarking diverse aspects of structure prediction methods. By introducing new scoring schemes, CAMEO facilitates new development in areas of active research, for example, modeling quaternary structure, complexes, or ligand binding sites.

Related Papers

• → Critical assessment of methods of protein structure prediction (CASP)—Round XIII(2019)538 cited
• → Critical assessment of methods of protein structure prediction (CASP)—Round XIV(2021)521 cited
• → Critical assessment of methods of protein structure prediction (CASP)—round IX(2011)206 cited
• → A Historical Perspective of Template-Based Protein Structure Prediction(2008)28 cited
• → Benchmarking establishment for green building based on DEA and individual advantage identification(2017)3 cited