In 2018, the Bert model with nearly 300 million parameters came out of nowhere, pushing the NLP field to new heights. In recent years, the development of the artificial intelligence field has increasingly tended to the study of large models, and all major AI giants have released their large models with hundreds of billions of parameters, giving birth to many new AI application scenarios. On the other hand, a variety of factors continue to drive the significant development of big models: 1) society is experiencing a deep digital transformation, and a large amount of data is gradually merging, giving rise to many AI application scenarios and needs; 2) hardware technology continues to advance: the NVIDIA A100 GPU, Go

If there is one of the most exciting achievements in AI academia in 2021, then Alphafold deserves the title.Alphafold2 achieved far greater accuracy than comparable models on the CASP14 protein prediction challenge, and for the first time ever improved the accuracy of protein structure prediction to the atomic level-which already close to the level of experimental measurements. The Phantom AI team successfully trained and ran Alphafold2 on the Firefly 2 platform shortly after the launch of Alphafold2, as described in our previous article, "Firefly Running Models | Alphafold".

As mentioned in the previous article, Phantom AI has improved the overall training performance of Alphafold by optimizing data processing, using both feature preprocessing and feature cropping. As we all know, Mirage AI has many parallel training gas pedals, such as hfreduce, 3FS, hfai.nn arithmetic library, etc., can they further accelerate the overall training of Alphafold? In this issue, we will experiment with these questions. hfreduce As mentioned in the previous article "Phantom Power | Model Parallel Training Tool: hfreduce", due to the Phantom AI architecture

The previous two issues of the article showed the optimization of Alphafold by Phantom AI, using feature preprocessing and feature cropping to improve the performance of Alphafold data processing, further improving the training speed of the model through parallel training acceleration artifacts, and deeply integrating Alphafold into the characteristics of Phantom AI's clusters, to play the maximum computational efficiency. From an overall point of view, what else do we need to pay attention to when training Alphafold on Phantom Firefly II, and how to optimize the same type of deep learning model in the future? On these topics, this article will talk to you about Phantom Cube AI's