Menu
Menu
This blog post will be focused on AlphaFold3 (Abramson et al. 2024) use for biologists. Here we cover practical considerations for people with a biological background: strong and weak sides of all AlphaFold models, types of structures AlphaFold3 fails to predict, things to be cautious about, and why we still need experimental structures even with all AlphaFold models.
A brief overview to be at the same page
Structure prediction is often used in the context of protein structure prediction in particular (even though you can also predict structure of RNA or DNA or small molecules). It’s a computational method used to determine the 3D structure of a protein based on its amino acid sequence. Understanding the structure of proteins is crucial for deciphering their functions and roles in biological processes. Experimental methods like X-ray crystallography, nuclear magnetic resonance (NMR) spectroscopy, and Cryo-EM are traditional approaches for determining protein structures, but they can be time- and resource-consuming, taking months and sometimes years of effort. Computational methods provide an alternative or complementary approach to predict protein structures more efficiently, from hours on the high end down to sub-seconds.
AlphaFold2 (Jumper et al. 2021), developed by DeepMind, is a state-of-the-art deep learning-based method for protein structure prediction. It combines a deep neural network architecture and training procedures based on the evolutionary, physical and geometric constraints of protein structures. It mainly relies on multiple sequence alignments (MSAs), pairwise features, specific output representation and associated loss, and transformer-based equivariant attention architecture. AlphaFold was designed to predict the 3D coordinates of amino acid residues in a protein monomer, providing accurate and reliable structural models. It participated in the Critical Assessment of Structure Prediction (CASP) competition, demonstrating groundbreaking performance and significantly advancing the field.
AlphaFold-Multimer (Evans et al. 2021) is an extension of the AlphaFold2 framework that is specifically trained to predict multi-chain proteins. Protein complexes are formed when multiple proteins interact with each other to form a larger functional unit. AlphaFold-Multimer is designed to capture these interactions and predict the structure of the entire complex. AlphaFold-Multimer utilizes the same deep learning architecture as AlphaFold, but introduces a new way of selecting subsets of residues for training such that the model is learning more about how chains interact with each other, and makes various small adjustments to the structure losses and the model architecture. These features allow predicted multimeric interfaces while maintaining high intra-chain accuracy.
Building upon the successes of its predecessors, AlphaFold3 represents a significant leap forward in non-protein structure prediction. Unlike previous versions, AlphaFold3 is able to accurately model not only protein structures but also those of DNA, RNA, ligands, and ions, along with the interactions of them. Its new diffusion architecture, together with extensive training across almost all classes of biomolecules in the world, enables AlphaFold3 to transcend the limitations of its predecessors, significantly improving accuracy over previous models. Building on the success of AlphaFold2, AlphaFold3 represents a significant leap forward in biomolecular understanding, with the potential to revolutionize drug discovery and scientific research.
Exploring each AlphaFold model limitations
AlphaFold2 struggles with structure prediction of proteins that lack evolutionary information (e.g. antibodies), proteins whose structure is dependent on environmental conditions (e.g. membrane proteins), and proteins with disordered regions (e.g. IDPs). Although, this last category is by definition, not expected to have structure. Additionally, it's restricted to the 20 canonical amino acids and predicts only rigid protein structures, excluding other biomolecular classes like DNA or RNA and their dynamics. AlphaFold-Multimer shares the same limitations as AlphaFold2, but works with protein complexes (no options for protein-DNA or protein-RNA predictions) instead of monomers.
Similar to AlphaFold2 and AlphaFold-Multimer, AlphaFold3 struggles with proteins lacking evolutionary information, those dependent on the environment, or containing disordered regions. Additionally, it does not support arbitrary chemical modifications, ligands, or ions, and it may generate non-existent secondary structures when uncertain, introducing a risk of hallucination (alpha helices instead of unpredicted regions). This issue can be remediated by always checking the per residues confidence, the pLDDT, of the predictions. While it predicts only static structures and lacks modeling for conformational changes upon binding, it also faces challenges with clashing atoms and predicting chirality accurately. From the legal side, the tool is restricted to non-commercial scientific use only, as of the writing of this article. Moreover, it lacks open-source availability and code accessibility, with access limited to the AlphaFold3 web server, and it imposes constraints on job numbers per day and input sequence size. Such restrictions hamper the ability of using AlphaFold3 in real-world scientific and industrial applications.
Let’s take a test drive to see how really good (or bad) the new version of AlphaFold is.
Test drive
In all the examples below experimental structures are colored in green (protein) and gray (nucleic acid) while AlphaFold3 models are blue (protein) and red (nucleic acid). One run (job) generates 5 AlphaFold3 models.
AlphaFold3 can predict biomolecules with stable structure that relies a lot on the evolutionary information that is the basis of all AlphaFold models. These examples include:
An additional observation deserves a separate note. It’s a protein-RNA complex (7UXA) but the experimental structure has missing loops (failed during the experiment). However, AlphaFold3 predicted them similarly to AlphaFold2 – no hallucination, but long noodle-looking unstructured loops. It gives a hint that AlphaFold3 was so good in the previous examples of nucleic acids in complex with proteins because they were in the training dataset.
AlphaFold3 can’t predict biomolecules with relatively evolutionary independent, flexible, or unstable structures (conditions and environment dependent which AlphaFold3 architecture doesn't catch). These examples include:
First I tried to model both conformations without ions (K+ are functionally important for this protein but I intentionally omitted them to test AlphaFold3 ability to capture the change)
Next I modeled both conformations with the correct amount of ions (biologically accurate condition).
In both cases AlpaFold3 doesn’t capture the membrane protein confirmation or its change. The quality of predictions is worse when you do not submit the full biological system (e.g., forgetting to add ions when the system requires them). To improve accuracy, try to always keep the system as close as possible to real-world conditions (maintain the correct amount of all ions, ligands, and chemical modifications).
One of the systems that combine several tricky proteins for AlphaFold3 is the SARS-CoV RBD/ACE2-B0AT1 complex. First, I tried just the ACE2-B0AT1 part (different subunits are colored in different shades: experimental ACE2 is default green, experimental B0AT1 is dull green, modeled ACE2 is default blue, modeled B0AT1 is dull blue). Already here, AlphaFold3 starts to struggle.
Next I looked at the open conformation of the same system. The difference between models and the experimental structure has become bigger.
At the end, I added the receptor-binding domain (RBD) with many loops in this structure (colored dark green in the experimental structure and dark blue in the predictions). The binding site is completely off in AlphaFold3 models. Loops at the binding interface resemble antibodies (where loops are also responsible for binding), leading to poor AlphaFold3 modeling of the antibody binding site as well.
Summary of the results
| System | Result | Type | Comment | 2MXU | Bad | Protein with disordered regions | Beta-amyloid fibril (structured oligomer formed by disordered monomers) |
|---|---|---|---|
| 5TBA | Good | Protein and nucleic acid complex | Human DNA polymerase with DNA substrate (DNA-protein complex) |
| 3DD2 | Good | Protein and nucleic acid complex | RNA aptamer bound to human thrombin (RNA-protein complex) |
| 4OO8 | Good | Protein and nucleic acid complex | Cas9 in complex with guide RNA and target DNA (part of CRISPR-Cas system) |
| 1IGT | Bad | Protein with disordered regions | Mouse IgG2a monoclonal antibody (full antibody structure with all domains) |
| 1P32 | Good | Protein with ordered structure | Human p32 eukaryotic acidic mitochondrial matrix protein (doughnut-shaped protein) |
| 2QZE | Good | Protein with ordered structure | Mimivirus mRNA capping enzyme (protein with catalytic activity) |
| 7MN1 | Good | Protein-switch (conformational changes) | Sa1 protein fold switches (shape-shifting protein) |
| 7MQ4 | Bad | Protein-switch (conformational changes) | Sb1 protein fold switches (shape-shifting protein) |
| 7MN2 | Good | Protein-switch (conformational changes) | Sb2 protein fold switches (shape-shifting protein) |
| 7UXA | Good | Protein and nucleic acid complex (conformational changes) | Human tRNA splicing endonuclease complex bound to pre-tRNA-ARG (RNA-protein complex) |
| 1J8I | Bad | Metamorphic protein (conformational changes) | Chemokine lymphotactin (immunological protein with several conformations at the same conditions) |
| 1K0M | Good | Protein with ordered structure | Crystal structure of a soluble form of the intracellular chloride ion channel CLIC1 (soluble form of a membrane protein) |
| 4WFF | Bad | Membrane protein | Human TRAAK K+ channel in a K+ bound nonconductive conformation (ion channel) |
| 4WFE | Bad | Membrane protein | Human TRAAK K+ channel in a K+ bound conductive conformation (ion channel) |
| 6M18 | Bad | Protein complex | ACE2-B0AT1 complex (molecular complex of several proteins) |
| 6M1D | Bad | Protein complex | Open conformation of ACE2-B0AT1 complex (molecular complex of several proteins) |
| 6M17 | Bad | Protein complex | The 2019-nCoV RBD/ACE2-B0AT1 complex (molecular complex of several proteins) |
The benchmarking conclusion remains the same as with previous AlphaFold models: the tool is useful, but you need to be aware that it makes mistakes (e.g., Retron-Eco1 or collagen examples). Now, instead of just showing these mistakes, AlphaFold3 sometimes covers them with diffusion-generated alpha-helices (check the pLDDT scores to notice these tricks). In contrast, AlphaFold2 and AlphaFold-Multimer leave the parts they can't predict unstructured, resulting in noodle-like loops in those regions. Similar to its predecessors, AlphaFold3 works poorly with intrinsically disordered proteins (IDPs), proteins with intrinsically disordered regions (IDRs), or dynamic proteins that change their conformation based on internal and external factors, such as protein loops, membrane proteins, protein switchers, and apo/holo conformations. AlphaFold3 does not capture the dynamic nature of proteins, especially those with IDRs, which is extremely important for accurate structure prediction of this class of biomolecules.
To address and validate these limitations, we still need experimentally determined structures for a sanity check. The machine learning field thrives on data quality – the better the data, the better the models, even with the same architecture. Without good data, there are no good models. Experimental structures serve as benchmarks for evaluating and improving computational methods. Their integration drives innovation within structural biology. Overall, with the transformative potential of computational models like AlphaFold2 and AlphaFold3, we must also acknowledge the invaluable contribution of experimental structures. A collaborative approach that combines both computational and structural data is key to structural biology advancement.
One more interesting thing to mention about the new AlphaFold3 paper is there is no direct comparison to the previous models in terms of protein structure prediction accuracy. Where is the Global Distance Test (GDT) for AlphaFold3 on this barplot (replicated from “Drug design for beginners”)? The absence of a direct comparison between AF3 and AF2 highlights the need for comprehensive benchmarking studies to provide clarity and confidence in the use of these advanced protein prediction models. From the limited observations we covered in this blog, AlphaFold2 can better manage only protein structure prediction, while AlphaFold3 does well with protein-DNA and protein-RNA but could perform worse than AlphaFold2 with some solely protein cases. For now, it's just an assumption based on the observation shown above, but you can follow our updates to find out more about the ability of AlphaFold2, AlphaFold3, and other tools to predict (and design) protein structures.
AlphaFold2 became a breakthrough in computational protein structure prediction, advancing a solution to a 50-year-old life science grand challenge. AlphaFold-Multimer expanded the capabilities of AlphaFold2 on protein-protein complexes, boosting even more the field of computational biology. AlphaFold3 advanced the drug discovery field by modeling cross-biomolecular systems like proteins, DNA, and RNA with ligands and ions. With a substantial improvement in predicting molecular interactions compared to existing methods, AlphaFold3 is poised to accelerate scientific research. Its free availability through the AlphaFold Server democratizes access to cutting-edge computational biology tools, enabling researchers worldwide to explore novel hypotheses and accelerate discoveries. Even though the web server has only a limited amount of jobs per day, it still enables many scientists worldwide with an opportunity to see what molecules look like. It’s especially important in those places that can’t afford high-throughput calculations or experimental identifications of the molecules on their own. With all its limitations, AlphaFold3 brings more benefit than harm to the world, and I’m looking forward to its further development.
Additional materials
While we wait for AlphaFold3 code to be released, you can check out other cool models like ESMFold on 310 copilot: https://310.ai/copilot/book
References