Keynote: Large-Scale Optimization Strategies for Typical HPC Workloads
Liu, Yu
Ensuring performance of applications running on large-scale clusters is one of the primary focuses in HPC research. In this talk, we will show our strategies for performance analysis and optimization of applications in various fields of research using large-scale HPC clusters. Our strategies are designed to comprehensively analyze runtime features of applications, parallelisation strategies of physical models, algorithmic implementations, and other technical details. These three levels of strategy cover platform optimization, technological innovation, and model innovation, and targeted optimization based on these features. State-of-the-art CPU instructions, network communication patterns, and innovative parallel strategies have been optimized for various applications.
 Good morning, ladies and gentlemen. So first of all, thank you for making it on time for our Thursdays program, especially after I heard you had a pretty good party last night. Most of you left at maybe after 10 p.m., something like that. So I do appreciate that you can come and join our presentation today. So we will start today's program with the keynote presentation given by Dr. Liu Yu from InSpar China. You probably heard about InSpar, the name of InSpar. In Chinese, actually the name means the wave or the tide. The topic that Professor Chao was talking about yesterday. So it's really making some wave. I think the most recent number, your company has maybe over 80 systems in the top 500 list. So it's quite some accomplishment they have made in the recent years. So Dr. Liu, I have known him for quite a while. He's coming from a background of computational physics. His role is a little bit similar to me at the Supercomputing Center. So he's in charge of the HPC application support team in InSpar, which means he gets the honor or say the role to deal with scientists from all these different domains. He needs to optimize until all different kinds of code. The last time when I met with him, we were in Saudi Arabia trying to optimize their climate modeling code. So you can see, he will tell you some more stories in his slides. So now I should give the time to Dr. Liu to share with you his fantastic experience at InSpar. Let's welcome him. Thank you. Thank you. Thank you, Professor Fu. Yeah. Good morning, everyone. I come from InSpar and it is very good honor for me to show our work to the world. I think before Professor Fu has introduced our company, I think most of you do not have heard InSpar before. So what's where are we come from? Who are we? Professor Fu just said we are an international software manufacturer all over the world and we have shared number three server market around the world and number one in China. And also we are the famous HPC and AI software provider and solution provider in China and all over the world. And also we have a great participant in the open computing organizations such as OCP, ODCC and Open19. Besides, I want to say that we are the only one company who can both provide power chips and x86 chips around the world. And InSpar has a lot of research and design centers around the world and the fund creates around the world. In Europe, our base is located in Frankfurt in Germany and our front-line in Europe is in Hungary. So we can provide European customers with local service. Because of many years of experience in the high-performance computing, InSpar also found that there are a lot of challenges in this world of computing. You know that we make supercomputers to help our scientific research. However, you know that today's supercomputers are very big and large and large. However, most of our scientists think that our calculation work has not been accelerated as the computer grows. The big problem is that complex architecture. You can see that the number one computer architecture has improved from the single CPU processor to a lot of accelerators and even complex architectures. So there are big problems is that the hardware utilization will be very low because the new hardware or software is old and cannot use that. So the application scalability would also be very bad. So how can we solve this problem? First of all, our supercomputers is built just for computing. So the main component of our supercomputers is the compute component, like the CPUs, GPUs, and other accelerators. So if we want our compute component run as fast as possible with no bottlenecks, so the data input or output must be no bottlenecks, like the storage IO bandwidth, the memory bandwidth, PCIe, and the link, and the other bandwidth, and so on. Because our today's work is very large, we are among a lot of computing nodes, so the communication between the nodes should not be bottlenecked. So how can we solve this problem? For many years, the appearance of HPC innovation is for summarizing our three strategies. That is, we first build a powerful profiling tool and then use the new list or the start-of-the-art hardware, and then we have to move our latest algorithm on physical models to that hardware. So somebody may ask, what is building a powerful profiling tool? And what is profiling tool? Why how we to build that? I want to say that there is a very old Chinese saying. I don't know whether you have seen that before. It's in Chinese, we call it the 公寓善其事,必先利其其. In English, it means that if you want to make good work, you have to sharpen your tools. You can see, oh, sorry, you can see this interesting picture. If you want your work much good, all the efficiency, so you must sharpen your tools. So from the previous slides, we see that we want to help our computing run faster. We have to make our computer components, data throughput components and communication components know bottleneck as fast as possible. So first of all, we have to see whether it's fast enough or whether there are bottlenecks. So for the computer components, we are not only how to see the utilization rate, like the user says, and so on, but we also have to see the actual floating pound speed while our application is running. And which component or which instruction size has been provided that floating pound. Besides, for the memory bandwidth or the PCI bandwidth or the IO bandwidth, although the great index to see whether there are bottlenecks for our data input into the compute component. And also, the infinite band are the opa bandwidth. Here we list the three figures here. You can see that here is the floating pound speed while our application is running. And here is a memory bandwidth. We can see that some application that you are nearly 200 gigabytes in a single node. So maybe that is some kind of bottleneck. So somebody may ask, we can use some open source or Intel's Weetium to do that. But I want to emphasize is that because our supercomputer is a big or huge machine, so it's not desktop service, desktop machine. So your work had to just as the forest, you just not have to see the single trees, but you have to see the whole behavior of what the forest have to do. So the perfect tool is very important. In the following, I will show several cases to see why the perfect tool is very important. Here I list the first example to show the powerful tools real power. Okay, here I list the CIDR outer atmospheric modeling system that is WALF. We know that this is a very complex model and some of the algorithm may be old. So and it's also a computing intensive application. So if you are going to make CIDR run much faster, we must do some computing or communication optimization. Our work for this optimization is doing for one of our customers in China. They want to do a simulation of the North China's weather forecast, which covers about 60 hours of weather forecasting. And they want to do that is the total computing process would be less than two hours, which includes the WPS process, WALF computing process, and also the post-processing process. However, and the first, our customer claims that the fastest time, the fastest time of only the WALF computing would last more than 150 minutes. So how can we solve this problem? First of all, we have to use a tool to see what's happened while the WALF is running. There we list the floating pound speed while the WALF is running. And we can see that there are a lot of gaps between the floating pound, which means that there is no actual computing. Since we analyzed into details, we found that these are a lot of IO process. So first of all, maybe we have to optimize our IO bandwidth. Besides, we have CZAN and more CZAN, the floating pound speed is very low. And the inflection size may be not in efficiency, but this is a little work. So first of all, we have to optimize our IO bandwidth. And we did three things. Three parts. The first one is electrical performance, and we use a quick IO in our name list. And also the P9CF, you can see that after optimization, and the gaps are nearly disappeared. But there is a big problem in that. You can see that maximum of the network flux has been increased. And there is a continuous feature of the network flux. This may affect our application's scalability. So the next one, what we have to do is to slow down or to let you down the maximum of the network flux. What we do is to make the MPI process fewer, because we know that if there are a lot of MPI process with the communication, the communication is very complex. So if we can use some open P mode, the communication complexity may be lower down. That is what we have done. And we can see that after optimization, the network flux has been down very efficiently, and the performance has been improved. And also after the MPI optimization, the memory has also been dropped, which is very useful for our performance improvement. And we can see that we just did a very simple optimization. It's very simple, just by seeing what's happened for the application running. You can see that we have a very great performance improvement. So since we show that the real power of the providing tool. But I have already shown that, you know that from the application features by the providing tool, we also have the running feature is that the floating points are not very high, and some of the kind of the interaction sites have not been efficiently used. So the next thing of our optimization strategy is that you have to make the new hardware or new architectures much more efficiently. So that is the hardness of the state of architecture. Okay. Here, I show why this is very important. Because if we are, application can use the new architecture and the new instruction site, it means that the machine's performance will be released fully. Here we show three cases that what we have done in the Tianhe 2 supercomputer in China. You know that Tianhe 2 is an architecture with Intel Xeon, CPUs, and also the mic processors. So we optimize these applications on this machine and with the lamps, we can scale it to more than 24,000 cars and the quantum space with more than 28,000 cars. And also, this is the GTC's plus simulation application, which we both use the mic and the Xeon processors, and we can use more than 400,000 cars on this machine. So it is very useful for the state of architecture optimization. For details, I will show you a case to show how can we do this work. This is another global scale atmospheric modeling system, which is developed by one of our customers. And this simulates the global weather forecast with a resolution of 0.125 degrees. And they want to make the simulation less than three hours. And first, they use more than 8,000 cars, but the simulation time is very long, more than one day. So we have to analyze what's wrong with the application. First of all, we list the most time-causing consuming functions and also the program parts of what happened while the program is running. We can see that the most of the time-consuming part is model integration, which we refer to the Hamhoff's equation. It's nearly 50%. And here we list the features while the application is running. We also can see that the total floating pound is not very high for today's hardware architectures. And we can see that most of the interaction sites are provided by SSE. However, the most powerful interaction sites has not been used. So this is the very important thing for us to optimize. We know that the original code, say, so the Hamhoff's equation is to decrease into this form. And the this form is as AX equals B. We know that this is to solve a sparse matrix. We know that this form is a common one. There are a lot of commercial or open source mathematics libraries that have been optimized for this kind of computing. So if we can use highly optimized code for this computing, maybe we can accelerate the Hamhoff's equations computing. That is what we have done. For more things, if we can lower the number of iterations, because we know that some of the, usually we solve the Hamhoff's equation by interaction method. So if we can lower the number of iterations, the total running time would be shorted. OK? And the original one, we can see that they are using the GCR method. So we make, we under just a preconditioner. However, the total improvement is very bad. For another thing, we do some more works to transfer preconditioner GCR method to a GMR M method with MKL library and hyper preconditioners. And this is a very great improvement for the computing. And we can see that after this optimization, the total floating pound has been improved nearly twice. And the instruction sites has been improved from the SSE to the AVX. So we can see that we can efficiently use the newly architectures. And here is total time. And you can see that the model integration time has been increased 61%, and the total running time has also increased 40%. We can see that the number of iterations has been reduced from 32 to 8 iterations. So it is a good improvement for our work. And for another case is that we have to do some more works for our supercomputers applications. And here we can see that most of the supercomputers centers around the world will run a lot of applications. And some of the applications like WASP is very important. WASP is applications for the material science and quantum physics application. And sometimes it will take more than 12 or 20% of CPU hours around the supercomputers centers. Because WASP is a feature, it's just like LIMPAC. So you know that the power consuming for this kind of applications is very high. Maybe it takes 30% or 50% of the total power consuming consumption. So it is very important for us to optimize the WASP application. And we take very famous, our hot research field for the material science. We use a Heusler alloy, which contains about nearly 300 atoms. And we use the plane wave-based site. And you know that the computing amount of this case will be some kind of large. Well, we do the same thing. We see what happened while the application is running. And you can see that nearly half of the time consuming is caused by the DGEMM functions, which is to solve the eggmaning problem. And also we can see that nearly the other half is caused by the MPI withdrawal and MPI all reduced functions, which means that the communication is banned since it will affect our applications cannot make a big scalability. So we have to see what's happened. And we see that the floating pounds is not very high. And the interaction sites only use SD and start-ups, previous optimization we showed you. And we did three things, is that first we have to optimize these eggmaning functions, and then we have to do some memory access optimization, because we know that what we use the plane wave-based site, which in the memory, it will be a lot of arrays. So the memory access would be very intensive. So we have to optimize memory access, like both settings and also the coding. Besides, which is very important is that the network, because we can see that nearly half of the time is caused by the MPI functions. So we have to make some of the modeling to make sure the running of the Raspberry is suitable for our supercomputer topology. Okay, so we changed some of the parameters in the in-cars, and also we optimized the collective communication functions. And after optimization, you can see that the floating pounds has been increased very great. And most of the applications running with the AVX512 interaction site. And you can see that the total performance has been increased nearly 228%. Besides, I want to say that to show the different encryption sites performance, you know that with Xeon's E52650 processors, it only have AVX encryption site. And the V3 processors, we have AVX and FME interaction site. And from here to here, we can see that the performance has been increased 90%, which means that the FME interaction site is very efficient. And also, from the V3 processor to the newly Xeon scalable family processors, we can see that the most difference is AVX interaction site and the AVX512 interaction site. We can see that with the vectorization from 256 to 512, the performance has been increased nearly 17%. So that is to show why the state of the art hardware is very important for us to optimize our applications. And here, we showed the optimization network flux. And you can see that this is a non-persistent network, which is conducive to improving game scalability because sometimes we can harden the communications into our computing part. However, the scalability of VASP because it's a commercial code, you know that it's not very good. So if we want to make this kind of application into one into millions of cars, this kind of commercial code may not work. We have to do a lot of work for the commercial codes, physical models to suitable to our supercomputers architectures. So here, come to the third part is that we must leverage the latest mode algorithm, which means that we must optimize or develop new algorithm or new physical models which is suitable for today's supercomputers architectures and topologicals. Here I list a timeline, which is also in quantum physics areas. And we can see that a lot of the algorithm or the series are some kind of old. And I say old because these algorithms are developed decades before. So today's a lot of the applications like the VASP applications that I just show you is also based on this kind of series. So because just this kind of series, the parallel modes degree has been limited into this part. So the total scalability may be very low. That is why the VASP scalability is not good. So we must do the mode optimization by two parts, the applications sense and also the computing sense. So it's some kind of interdisciplinary work. Because for our application sense, we must think or must develop our algorithm or our mode to suitable for our computers. And also for our computer scientists, we must think our new architecture to suitable for applications. So if we can make these two parts work together, so large-scale complication would be reached. And the first case we show here is to show how can we do that. It's also work we did for our customers in China. This is kind of code for simulating new spin phonics. Because we know that today's processors like the desktop, our supercomputers are based on a lot of semiconductors, the silicon. And we use the voltage high and low to represent bit 1 or 0. However, if we can use the electron spin, which is at the spin right up and down to represent the bit 1 or 0, the total power consumption will be reduced nearly 100,000 times. So if we can use spin phonics to make our processors, maybe our supercomputers, power problem may be not a problem. And also maybe our cell phones do not need to recharge every day. However, their code is very bad. So the work is very great. The code is very bad. They only can simulate, just use turn cars, very low, just desktop. And the matrix scale is less than the power of 10 of the four. And they can only simulate the three electrons. So we analyze the problem. They use the configuration to answer method, which just like our atoms. We know that for most of the time, the electrons are located in a very low energy state. However, they make the simulation with full consideration of the non-occupied state. We know that there are a lot of ways for the computation. So for our optimization mode, the first thing is that we can get rid of this useless computing. And besides, they use the four metrics to record the sparse metrics, which means that there are a lot of normally waste into records zero ones. So we do this optimization just to change the fourth CI to the partial CI. And also, we change the QR method with an integration method, which is suitable for parallelism. And also, we optimize the vector interactions and also the collective communications. And after that, you can see that there is a very great performance improvement. The total parallel degree is from 10 cars to nearly 4,000 of cars. And the most important is that they can do a lot of new works. In Cisco, they have submitted a lot of works in the FIT review letters and some of the famous scientific research articles. And the last, I will show another case why the modeling innovation is very important here. I know a lot of your professors here are doing the global climate simulation work because you know that the climate simulation is complex because they will couple a lot of computing components or physical components to make the simulation. And with resolution increase, the total number of calculation will increase very much. So, it is difficult for us to make a very exact, our climate change just using little or small supercomputers. So we must make our code run as large as possible. However, our customers in China make these codes very poor because they can only simulate the code in nearly less than a thousand of cars, several hundred. And for someone to simulate a decade simulation will take over two days. So this is a big problem. We did the optimization together with them and we found that the five parts of the application list here and you can see that most of the time consuming part except the ATM component, the other four is MPI. So we can see that there is a big problem. The ATM is a bottleneck. And we did some details, analyzed of this code. And we found that this kind of code is some kind of ATM driver, which means that the other component like the land and ATM and ice and ocean are coupled with this ATM driver. So the ATM would be a bottleneck. So the most of the thing is that we have to change this pseudo-parallel mode into real parallel mode. So we changed the ATM driver to a CPU driver, which means that the ATM bottlenecks have been bricked. And also we optimized ATM with the MPI plus open MP mode. And you can see that after this kind of optimization, the total performance has been increased nearly twice. And also the ATM has also increased 20%. So today I showed our methods to realize large-scale computing. We used three strategies to take the runtime performance data with a powerful profiling tool. And with the profiling data, we know the applications features and we know whether there is a bottleneck. So we can do a set of technologies innovation and also we can do the interdisciplinary innovation for the new models to optimize our application work so we can achieve a great application performance. However, maybe you can consider that the most important thing and the difficult thing is the inter-planaried innovation because we lack of these talents to do that. Some of our computer scientists only famous our computing world. However, our scientists from other research fields like physics and chemical and so on, they only know their own research work. They do not understand the computer science. So there is a big gap. So the most of the important thing may be to treat the people who know both parts. So one more thing I want to introduce is the ASA student computer challenge. This computer challenge provides a stage for the student practicing large-scale efficient optimization. This kind of event has been held eight years. It started in 2012. And over 7,000 young talents have drawn this event and more than 300 universities around the world have come into this wonderful event. And a lot of the young talents in this event has made a very good job. You can see that the AS14, some of the young talents have optimized three-dimensional electric wave model on the Tianhe 2 supercomputers with more than 200,000 cars. And the AS17, some of the young talents use the Sunway Type-Lite to simulate a golden bell press nominated application maximum with more than 10,000 cars. So you can see that the young talents are very powerful for tomorrow's future supercomputers work. So maybe I can release a video to you to see what happens or what is the ASA event. And we also like all of you are professors to encourage your students to come into this event and have a great experience with large-scale application. So this is a video we take from the ASA 19 event. We are all focused. We work together. We have the same goal. To explore more unknowns. To find more possibilities. To achieve a better future. Five days, 20 teams, new competitions. Set sail again. I think in the previous competitions, there was only one competition. Now there is cooperation. It is a new thing. And I think everyone will invest in this thing. It is a good new competition project. This time, the super team's design strengthened our cooperation and communication with other teams. We share some of our experience and help each other to fight for the same thing. It is a good new experience. Everyone is kind of competitive. But still very friendly and helpful. You can easily talk to other teams. And they will actually help you. They are willing to help you. You know that this is a very interesting event. We are focused on making our undergraduate student training for tomorrow's super computer scientists. This is what I show today. Thank you very much for your time. Thank you. Any questions from the audience? I have a question because it is really kind of a dream. You are doing a lot of work for the optimization of existing applications. You have three pillars. You can improve the code. How actually it happens? Users coming to your firm and you have a contract. You are working together. It is a team work. It is for how many years? Typically, how long does it take? How it happens? When they pay your team or the team is going to the institution? For example, this climate people coming to your firm and say, we have this code and we want to improve it. We want to optimize it. And how then it happens? You conclude a contract with this firm. Most of the time, we work with our customers together. First of all, you know that optimization will take a lot of, take a very long time to do that work and it is very difficult. And we have special application teams. That is my team. And we have experts to work with our customers together to simulate and to analyze what their application is and what their aim is for. And after that, we will make some of the tests with their applications or their workloads and then find the bottlenecks and then we do the optimization. Maybe some kind of the computing work is doing by our teams and some of the physical models, you can see that maybe it's, maybe work by our customers, but we have to work together. You know that it's interdisciplinary work. And you already said, we will last several months or even years. So before we achieve a very good, a very good or useful performance. That is total procedure. Actually, it was how to become your customer. Sorry. Yeah, yeah, yeah. Something is my colleague. Yeah. Okay. You show very nice. What is your policy of feeding back your optimization to the community, to the project that, to the open source projects that you use. Okay. Does this optimization actually find their way back into the main development trunk? Okay. And present we have not put our optimized code into the communication, into the community. You know that there are a lot of the codes developed by our customers yourself, just in host code. So maybe this kind of codes we cannot just upload into the GitHub and so on. And for the other open source projects, and for the other open source optimization like the lamps or context based on so on, we showed in the TFA tool express, TNH2 supercomputers. We haven't uploaded into the community because maybe the, you know, the computer centers won't like us to do that. I don't know why. Okay. So I have a question. I mean, your company sells both hardware and I guess software services or at least you have shown that software is very important to make good use of the hardware. If you would have to advise people how to invest their money, you know, would you say 50% for the hardware, 50% for the software or what do you think is a good ratio of investment? Oh, you mean for the solution, the HPC solution. Okay. If you look for the holistic solution, you know, the system that runs the application best, how would you invest? For my opinion, I think the software is more important than hardware. So I think you have to put more than 50% on the software, not the hardware. Good. I will take your answer to whoever finds this. Does your boss agree with you? Maybe not. So it's my own opinion. Okay. Okay. Okay. If no more questions, let's thank our speaker again. Thank you very much.
 Good morning, ladies and gentlemen.  So first of all, thank you for making it on time for our Thursdays program, especially  after I heard you had a pretty good party last night.  Most of you left at maybe after 10 p.m., something like that.  So I do appreciate that you can come and join our presentation today.  So we will start today's program with the keynote presentation given by Dr. Liu Yu from  InSpar China.  You probably heard about InSpar, the name of InSpar.  In Chinese, actually the name means the wave or the tide.  The topic that Professor Chao was talking about yesterday.  So it's really making some wave.  I think the most recent number, your company has maybe over 80 systems in the top 500 list.  So it's quite some accomplishment they have made in the recent years.  So Dr. Liu, I have known him for quite a while.  He's coming from a background of computational physics.  His role is a little bit similar to me at the Supercomputing Center.  So he's in charge of the HPC application support team in InSpar, which means he gets the honor  or say the role to deal with scientists from all these different domains.  He needs to optimize until all different kinds of code.  The last time when I met with him, we were in Saudi Arabia trying to optimize their climate  modeling code.  So you can see, he will tell you some more stories in his slides.  So now I should give the time to Dr. Liu to share with you his fantastic experience at  InSpar.  Let's welcome him.  Thank you.  Thank you.  Thank you, Professor Fu.  Yeah.  Good morning, everyone.  I come from InSpar and it is very good honor for me to show our work to the world.  I think before Professor Fu has introduced our company, I think most of you do not have  heard InSpar before.  So what's where are we come from?  Who are we?  Professor Fu just said we are an international software manufacturer all over the world and  we have shared number three server market around the world and number one in China.  And also we are the famous HPC and AI software provider and solution provider in China and  all over the world.  And also we have a great participant in the open computing organizations such as OCP,  ODCC and Open19.  Besides, I want to say that we are the only one company who can both provide power chips  and x86 chips around the world.  And InSpar has a lot of research and design centers around the world and the fund creates  around the world.  In Europe, our base is located in Frankfurt in Germany and our front-line in Europe is  in Hungary.  So we can provide European customers with local service.  Because of many years of experience in the high-performance computing, InSpar also found  that there are a lot of challenges in this world of computing.  You know that we make supercomputers to help our scientific research.  However, you know that today's supercomputers are very big and large and large.  However, most of our scientists think that our calculation work has not been accelerated  as the computer grows.  The big problem is that complex architecture.  You can see that the number one computer architecture has improved from the single CPU processor  to a lot of accelerators and even complex architectures.  So there are big problems is that the hardware utilization will be very low because the new  hardware or software is old and cannot use that.  So the application scalability would also be very bad.  So how can we solve this problem?  First of all, our supercomputers is built just for computing.  So the main component of our supercomputers is the compute component, like the CPUs,  GPUs, and other accelerators.  So if we want our compute component run as fast as possible with no bottlenecks, so the  data input or output must be no bottlenecks, like the storage IO bandwidth, the memory  bandwidth, PCIe, and the link, and the other bandwidth, and so on.  Because our today's work is very large, we are among a lot of computing nodes, so the  communication between the nodes should not be bottlenecked.  So how can we solve this problem?  For many years, the appearance of HPC innovation is for summarizing our three strategies.  That is, we first build a powerful profiling tool and then use the new list or the start-of-the-art  hardware, and then we have to move our latest algorithm on physical models to that hardware.  So somebody may ask, what is building a powerful profiling tool?  And what is profiling tool?  Why how we to build that?  I want to say that there is a very old Chinese saying.  I don't know whether you have seen that before.  It's in Chinese, we call it the 公寓善其事,必先利其其.  In English, it means that if you want to make good work, you have to sharpen your tools.  You can see, oh, sorry, you can see this interesting picture.  If you want your work much good, all the efficiency, so you must sharpen your tools.  So from the previous slides, we see that we want to help our computing run faster.  We have to make our computer components, data throughput components and communication components  know bottleneck as fast as possible.  So first of all, we have to see whether it's fast enough or whether there are bottlenecks.  So for the computer components, we are not only how to see the utilization rate, like  the user says, and so on, but we also have to see the actual floating pound speed while  our application is running.  And which component or which instruction size has been provided that floating pound.  Besides, for the memory bandwidth or the PCI bandwidth or the IO bandwidth, although the  great index to see whether there are bottlenecks for our data input into the compute component.  And also, the infinite band are the opa bandwidth.  Here we list the three figures here.  You can see that here is the floating pound speed while our application is running.  And here is a memory bandwidth.  We can see that some application that you are nearly 200 gigabytes in a single node.  So maybe that is some kind of bottleneck.  So somebody may ask, we can use some open source or Intel's Weetium to do that.  But I want to emphasize is that because our supercomputer is a big or huge machine, so  it's not desktop service, desktop machine.  So your work had to just as the forest, you just not have to see the single trees, but  you have to see the whole behavior of what the forest have to do.  So the perfect tool is very important.  In the following, I will show several cases to see why the perfect tool is very important.  Here I list the first example to show the powerful tools real power.  Okay, here I list the CIDR outer atmospheric modeling system that is WALF.  We know that this is a very complex model and some of the algorithm may be old.  So and it's also a computing intensive application.  So if you are going to make CIDR run much faster, we must do some computing or communication  optimization.  Our work for this optimization is doing for one of our customers in China.  They want to do a simulation of the North China's weather forecast, which covers about  60 hours of weather forecasting.  And they want to do that is the total computing process would be less than two hours, which  includes the WPS process, WALF computing process, and also the post-processing process.  However, and the first, our customer claims that the fastest time, the fastest time of  only the WALF computing would last more than 150 minutes.  So how can we solve this problem?  First of all, we have to use a tool to see what's happened while the WALF is running.  There we list the floating pound speed while the WALF is running.  And we can see that there are a lot of gaps between the floating pound, which means that  there is no actual computing.  Since we analyzed into details, we found that these are a lot of IO process.  So first of all, maybe we have to optimize our IO bandwidth.  Besides, we have CZAN and more CZAN, the floating pound speed is very low.  And the inflection size may be not in efficiency, but this is a little work.  So first of all, we have to optimize our IO bandwidth.  And we did three things.  Three parts.  The first one is electrical performance, and we use a quick IO in our name list.  And also the P9CF, you can see that after optimization, and the gaps are nearly disappeared.  But there is a big problem in that.  You can see that maximum of the network flux has been increased.  And there is a continuous feature of the network flux.  This may affect our application's scalability.  So the next one, what we have to do is to slow down or to let you down the maximum  of the network flux.  What we do is to make the MPI process fewer, because we know that if there are a lot of  MPI process with the communication, the communication is very complex.  So if we can use some open P mode, the communication complexity may be lower down.  That is what we have done.  And we can see that after optimization, the network flux has been down very efficiently,  and the performance has been improved.  And also after the MPI optimization, the memory has also been dropped, which is very useful  for our performance improvement.  And we can see that we just did a very simple optimization.  It's very simple, just by seeing what's happened for the application running.  You can see that we have a very great performance improvement.  So since we show that the real power of the providing tool.  But I have already shown that, you know that from the application features by the providing  tool, we also have the running feature is that the floating points are not very high,  and some of the kind of the interaction sites have not been efficiently used.  So the next thing of our optimization strategy is that you have to make the new hardware  or new architectures much more efficiently.  So that is the hardness of the state of architecture.  Okay.  Here, I show why this is very important.  Because if we are, application can use the new architecture and the new instruction  site, it means that the machine's performance will be released fully.  Here we show three cases that what we have done in the Tianhe 2 supercomputer in China.  You know that Tianhe 2 is an architecture with Intel Xeon, CPUs, and also the mic processors.  So we optimize these applications on this machine and with the lamps, we can scale  it to more than 24,000 cars and the quantum space with more than 28,000 cars.  And also, this is the GTC's plus simulation application, which we both use the mic and  the Xeon processors, and we can use more than 400,000 cars on this machine.  So it is very useful for the state of architecture optimization.  For details, I will show you a case to show how can we do this work.  This is another global scale atmospheric modeling system, which is developed by one of our customers.  And this simulates the global weather forecast with a resolution of 0.125 degrees.  And they want to make the simulation less than three hours.  And first, they use more than 8,000 cars, but the simulation time is very long, more  than one day.  So we have to analyze what's wrong with the application.  First of all, we list the most time-causing consuming functions and also the program parts  of what happened while the program is running.  We can see that the most of the time-consuming part is model integration, which we refer  to the Hamhoff's equation.  It's nearly 50%.  And here we list the features while the application is running.  We also can see that the total floating pound is not very high for today's hardware architectures.  And we can see that most of the interaction sites are provided by SSE.  However, the most powerful interaction sites has not been used.  So this is the very important thing for us to optimize.  We know that the original code, say, so the Hamhoff's equation is to decrease into this  form.  And the this form is as AX equals B. We know that this is to solve a sparse matrix.  We know that this form is a common one.  There are a lot of commercial or open source mathematics libraries that have been optimized  for this kind of computing.  So if we can use highly optimized code for this computing, maybe we can accelerate the  Hamhoff's equations computing.  That is what we have done.  For more things, if we can lower the number of iterations, because we know that some of  the, usually we solve the Hamhoff's equation by interaction method.  So if we can lower the number of iterations, the total running time would be shorted.  OK?  And the original one, we can see that they are using the GCR method.  So we make, we under just a preconditioner.  However, the total improvement is very bad.  For another thing, we do some more works to transfer preconditioner GCR method to a GMR  M method with MKL library and hyper preconditioners.  And this is a very great improvement for the computing.  And we can see that after this optimization, the total floating pound has been improved  nearly twice.  And the instruction sites has been improved from the SSE to the AVX.  So we can see that we can efficiently use the newly architectures.  And here is total time.  And you can see that the model integration time has been increased 61%, and the total  running time has also increased 40%.  We can see that the number of iterations has been reduced from 32 to 8 iterations.  So it is a good improvement for our work.  And for another case is that we have to do some more works for our supercomputers applications.  And here we can see that most of the supercomputers centers around the world will run a lot of  applications.  And some of the applications like WASP is very important.  WASP is applications for the material science and quantum physics application.  And sometimes it will take more than 12 or 20% of CPU hours around the supercomputers  centers.  Because WASP is a feature, it's just like LIMPAC.  So you know that the power consuming for this kind of applications is very high.  Maybe it takes 30% or 50% of the total power consuming consumption.  So it is very important for us to optimize the WASP application.  And we take very famous, our hot research field for the material science.  We use a Heusler alloy, which contains about nearly 300 atoms.  And we use the plane wave-based site.  And you know that the computing amount of this case will be some kind of large.  Well, we do the same thing.  We see what happened while the application is running.  And you can see that nearly half of the time consuming is caused by the DGEMM functions,  which is to solve the eggmaning problem.  And also we can see that nearly the other half is caused by the MPI withdrawal and MPI  all reduced functions, which means that the communication is banned since it will affect  our applications cannot make a big scalability.  So we have to see what's happened.  And we see that the floating pounds is not very high.  And the interaction sites only use SD and start-ups, previous optimization we showed  you.  And we did three things, is that first we have to optimize these eggmaning functions,  and then we have to do some memory access optimization, because we know that what we  use the plane wave-based site, which in the memory, it will be a lot of arrays.  So the memory access would be very intensive.  So we have to optimize memory access, like both settings and also the coding.  Besides, which is very important is that the network, because we can see that nearly half  of the time is caused by the MPI functions.  So we have to make some of the modeling to make sure the running of the Raspberry is  suitable for our supercomputer topology.  Okay, so we changed some of the parameters in the in-cars, and also we optimized the  collective communication functions.  And after optimization, you can see that the floating pounds has been increased very great.  And most of the applications running with the AVX512 interaction site.  And you can see that the total performance has been increased nearly 228%.  Besides, I want to say that to show the different encryption sites performance, you know that  with Xeon's E52650 processors, it only have AVX encryption site.  And the V3 processors, we have AVX and FME interaction site.  And from here to here, we can see that the performance has been increased 90%, which  means that the FME interaction site is very efficient.  And also, from the V3 processor to the newly Xeon scalable family processors, we can see  that the most difference is AVX interaction site and the AVX512 interaction site.  We can see that with the vectorization from 256 to 512, the performance has been increased  nearly 17%.  So that is to show why the state of the art hardware is very important for us to optimize  our applications.  And here, we showed the optimization network flux.  And you can see that this is a non-persistent network, which is conducive to improving  game scalability because sometimes we can harden the communications into our computing  part.  However, the scalability of VASP because it's a commercial code, you know that it's not  very good.  So if we want to make this kind of application into one into millions of cars, this kind  of commercial code may not work.  We have to do a lot of work for the commercial codes, physical models to suitable to our  supercomputers architectures.  So here, come to the third part is that we must leverage the latest mode algorithm, which  means that we must optimize or develop new algorithm or new physical models which is  suitable for today's supercomputers architectures and topologicals.  Here I list a timeline, which is also in quantum physics areas.  And we can see that a lot of the algorithm or the series are some kind of old.  And I say old because these algorithms are developed decades before.  So today's a lot of the applications like the VASP applications that I just show you  is also based on this kind of series.  So because just this kind of series, the parallel modes degree has been limited into this part.  So the total scalability may be very low.  That is why the VASP scalability is not good.  So we must do the mode optimization by two parts, the applications sense and also the  computing sense.  So it's some kind of interdisciplinary work.  Because for our application sense, we must think or must develop our algorithm or our  mode to suitable for our computers.  And also for our computer scientists, we must think our new architecture to suitable for  applications.  So if we can make these two parts work together, so large-scale complication would be reached.  And the first case we show here is to show how can we do that.  It's also work we did for our customers in China.  This is kind of code for simulating new spin phonics.  Because we know that today's processors like the desktop, our supercomputers are based  on a lot of semiconductors, the silicon.  And we use the voltage high and low to represent bit 1 or 0.  However, if we can use the electron spin, which is at the spin right up and down to represent  the bit 1 or 0, the total power consumption will be reduced nearly 100,000 times.  So if we can use spin phonics to make our processors, maybe our supercomputers, power  problem may be not a problem.  And also maybe our cell phones do not need to recharge every day.  However, their code is very bad.  So the work is very great.  The code is very bad.  They only can simulate, just use turn cars, very low, just desktop.  And the matrix scale is less than the power of 10 of the four.  And they can only simulate the three electrons.  So we analyze the problem.  They use the configuration to answer method, which just like our atoms.  We know that for most of the time, the electrons are located in a very low energy state.  However, they make the simulation with full consideration of the non-occupied state.  We know that there are a lot of ways for the computation.  So for our optimization mode, the first thing is that we can get rid of this useless computing.  And besides, they use the four metrics to record the sparse metrics, which means that  there are a lot of normally waste into records zero ones.  So we do this optimization just to change the fourth CI to the partial CI.  And also, we change the QR method with an integration method, which is suitable for  parallelism.  And also, we optimize the vector interactions and also the collective communications.  And after that, you can see that there is a very great performance improvement.  The total parallel degree is from 10 cars to nearly 4,000 of cars.  And the most important is that they can do a lot of new works.  In Cisco, they have submitted a lot of works in the FIT review letters and some of the famous  scientific research articles.  And the last, I will show another case why the modeling innovation is very important here.  I know a lot of your professors here are doing the global climate simulation work because  you know that the climate simulation is complex because they will couple a lot of computing  components or physical components to make the simulation.  And with resolution increase, the total number of calculation will increase very much.  So, it is difficult for us to make a very exact, our climate change just using little  or small supercomputers.  So we must make our code run as large as possible.  However, our customers in China make these codes very poor because they can only simulate  the code in nearly less than a thousand of cars, several hundred.  And for someone to simulate a decade simulation will take over two days.  So this is a big problem.  We did the optimization together with them and we found that the five parts of the application  list here and you can see that most of the time consuming part except the ATM component,  the other four is MPI.  So we can see that there is a big problem.  The ATM is a bottleneck.  And we did some details, analyzed of this code.  And we found that this kind of code is some kind of ATM driver, which means that the other  component like the land and ATM and ice and ocean are coupled with this ATM driver.  So the ATM would be a bottleneck.  So the most of the thing is that we have to change this pseudo-parallel mode into real  parallel mode.  So we changed the ATM driver to a CPU driver, which means that the ATM bottlenecks have been  bricked.  And also we optimized ATM with the MPI plus open MP mode.  And you can see that after this kind of optimization, the total performance has been increased nearly  twice.  And also the ATM has also increased 20%.  So today I showed our methods to realize large-scale computing.  We used three strategies to take the runtime performance data with a powerful profiling  tool.  And with the profiling data, we know the applications features and we know whether there is a bottleneck.  So we can do a set of technologies innovation and also we can do the interdisciplinary innovation  for the new models to optimize our application work so we can achieve a great application  performance.  However, maybe you can consider that the most important thing and the difficult thing is  the inter-planaried innovation because we lack of these talents to do that.  Some of our computer scientists only famous our computing world.  However, our scientists from other research fields like physics and chemical and so on,  they only know their own research work.  They do not understand the computer science.  So there is a big gap.  So the most of the important thing may be to treat the people who know both parts.  So one more thing I want to introduce is the ASA student computer challenge.  This computer challenge provides a stage for the student practicing large-scale efficient  optimization.  This kind of event has been held eight years.  It started in 2012.  And over 7,000 young talents have drawn this event and more than 300 universities around  the world have come into this wonderful event.  And a lot of the young talents in this event has made a very good job.  You can see that the AS14, some of the young talents have optimized three-dimensional electric  wave model on the Tianhe 2 supercomputers with more than 200,000 cars.  And the AS17, some of the young talents use the Sunway Type-Lite to simulate a golden  bell press nominated application maximum with more than 10,000 cars.  So you can see that the young talents are very powerful for tomorrow's future supercomputers  work.  So maybe I can release a video to you to see what happens or what is the ASA event.  And we also like all of you are professors to encourage your students to come into this  event and have a great experience with large-scale application.  So this is a video we take from the ASA 19 event.  We are all focused.  We work together.  We have the same goal.  To explore more unknowns.  To find more possibilities.  To achieve a better future.  Five days, 20 teams, new competitions.  Set sail again.  I think in the previous competitions, there was only one competition.  Now there is cooperation.  It is a new thing.  And I think everyone will invest in this thing.  It is a good new competition project.  This time, the super team's design strengthened our cooperation and communication with other teams.  We share some of our experience and help each other to fight for the same thing.  It is a good new experience.  Everyone is kind of competitive.  But still very friendly and helpful.  You can easily talk to other teams.  And they will actually help you.  They are willing to help you.  You know that this is a very interesting event.  We are focused on making our undergraduate student training for tomorrow's super computer scientists.  This is what I show today. Thank you very much for your time.  Thank you.  Any questions from the audience?  I have a question because it is really kind of a dream.  You are doing a lot of work for the optimization of existing applications.  You have three pillars.  You can improve the code.  How actually it happens?  Users coming to your firm and you have a contract.  You are working together.  It is a team work.  It is for how many years?  Typically, how long does it take?  How it happens?  When they pay your team or the team is going to the institution?  For example, this climate people coming to your firm and say,  we have this code and we want to improve it.  We want to optimize it.  And how then it happens?  You conclude a contract with this firm.  Most of the time, we work with our customers together.  First of all, you know that optimization will take a lot of,  take a very long time to do that work and it is very difficult.  And we have special application teams.  That is my team.  And we have experts to work with our customers together to simulate and to analyze  what their application is and what their aim is for.  And after that, we will make some of the tests with their applications or their workloads  and then find the bottlenecks and then we do the optimization.  Maybe some kind of the computing work is doing by our teams  and some of the physical models, you can see that maybe it's,  maybe work by our customers, but we have to work together.  You know that it's interdisciplinary work.  And you already said, we will last several months or even years.  So before we achieve a very good, a very good or useful performance.  That is total procedure.  Actually, it was how to become your customer.  Sorry.  Yeah, yeah, yeah.  Something is my colleague.  Yeah.  Okay.  You show very nice.  What is your policy of feeding back your optimization to the community,  to the project that, to the open source projects that you use.  Okay.  Does this optimization actually find their way back into the main development trunk?  Okay.  And present we have not put our optimized code into the communication,  into the community.  You know that there are a lot of the codes developed by our customers  yourself, just in host code.  So maybe this kind of codes we cannot just upload into the GitHub and so on.  And for the other open source projects,  and for the other open source optimization like the lamps or context based on so on,  we showed in the TFA tool express, TNH2 supercomputers.  We haven't uploaded into the community because maybe the, you know,  the computer centers won't like us to do that.  I don't know why.  Okay.  So I have a question.  I mean, your company sells both hardware and I guess software services or at least you have shown that software is very important to make good use of the hardware.  If you would have to advise people how to invest their money, you know, would you say 50% for the hardware, 50% for the software or what do you think is a good ratio of investment?  Oh, you mean for the solution, the HPC solution.  Okay.  If you look for the holistic solution, you know, the system that runs the application best, how would you invest?  For my opinion, I think the software is more important than hardware.  So I think you have to put more than 50% on the software, not the hardware.  Good.  I will take your answer to whoever finds this.  Does your boss agree with you?  Maybe not.  So it's my own opinion.  Okay.  Okay.  Okay.  If no more questions, let's thank our speaker again.  Thank you very much.
_Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_0_743.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_744_995.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_996_1979.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_1980_3167.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_3168_3923.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_3924_4139.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_4140_6371.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_6372_7427.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_7428_8195.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_8196_8507.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_8508_10247.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_10248_14855.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_14856_15203.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_15204_18539.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_18540_18839.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_18840_20159.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_20160_22607.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_22608_25919.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_25920_29999.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_30000_33287.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_33288_36059.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_36060_36383.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_36384_37271.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_37272_41015.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_41016_41087.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_41088_43019.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_43020_43415.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_43416_44795.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_44796_46547.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_46548_47099.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_47100_49211.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_49212_49355.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_49356_50927.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_50928_52979.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_52980_54191.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54192_54287.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54288_54347.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54348_54395.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54396_54443.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54444_54515.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54516_54671.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54672_54803.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_54804_55055.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55056_55175.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55176_55259.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55260_55451.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55452_55499.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55500_55559.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55560_55655.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_55656_56159.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_56160_56843.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_56844_56903.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_56904_57239.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_57240_58211.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_58212_59423.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_59424_64343.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_64344_65723.png _Inspur_Sponsored_Keynote__Large-Scale_Optimization_Strategi_65724_65799.png
Good morning, ladies and gentlemen. So first of all, thank you for making it on time for our Thursdays program, especially after I heard you had a pretty good party last night. Most of you left at maybe after 10 p.m., something like that. So I do appreciate that you can come and join our presentation today. So we will start today's program with the keynote presentation given by Dr. Liu Yu from InSpar China. You probably heard about InSpar, the name of InSpar. In Chinese, actually the name means the wave or the tide. The topic that Professor Chao was talking about yesterday. So it's really making some wave. I think the most recent number, your company has maybe over 80 systems in the top 500 list. So it's quite some accomplishment they have made in the recent years. So Dr. Liu, I have known him for quite a while. He's coming from a background of computational physics. His role is a little bit similar to me at the Supercomputing Center. So he's in charge of the HPC application support team in InSpar, which means he gets the honor or say the role to deal with scientists from all these different domains. He needs to optimize until all different kinds of code. The last time when I met with him, we were in Saudi Arabia trying to optimize their climate modeling code. So you can see, he will tell you some more stories in his slides. So now I should give the time to Dr. Liu to share with you his fantastic experience at InSpar. Let's welcome him. Thank you. Thank you. Thank you, Professor Fu. Yeah. Good morning, everyone. I come from InSpar and it is very good honor for me to show our work to the world. I think before Professor Fu has introduced our company, I think most of you do not have heard InSpar before. So what's where are we come from? Who are we? Professor Fu just said we are an international software manufacturer all over the world and we have shared number three server market around the world and number one in China. And also we are the famous HPC and AI software provider and solution provider in China and all over the world. And also we have a great participant in the open computing organizations such as OCP, ODCC and Open19. Besides, I want to say that we are the only one company who can both provide power chips and x86 chips around the world. And InSpar has a lot of research and design centers around the world and the fund creates around the world. In Europe, our base is located in Frankfurt in Germany and our front-line in Europe is in Hungary. So we can provide European customers with local service. Because of many years of experience in the high-performance computing, InSpar also found that there are a lot of challenges in this world of computing. You know that we make supercomputers to help our scientific research. However, you know that today's supercomputers are very big and large and large. However, most of our scientists think that our calculation work has not been accelerated as the computer grows. The big problem is that complex architecture. You can see that the number one computer architecture has improved from the single CPU processor to a lot of accelerators and even complex architectures. So there are big problems is that the hardware utilization will be very low because the new hardware or software is old and cannot use that. So the application scalability would also be very bad. So how can we solve this problem? First of all, our supercomputers is built just for computing. So the main component of our supercomputers is the compute component, like the CPUs, GPUs, and other accelerators. So if we want our compute component run as fast as possible with no bottlenecks, so the data input or output must be no bottlenecks, like the storage IO bandwidth, the memory bandwidth, PCIe, and the link, and the other bandwidth, and so on. Because our today's work is very large, we are among a lot of computing nodes, so the communication between the nodes should not be bottlenecked. So how can we solve this problem? For many years, the appearance of HPC innovation is for summarizing our three strategies. That is, we first build a powerful profiling tool and then use the new list or the start-of-the-art hardware, and then we have to move our latest algorithm on physical models to that hardware. So somebody may ask, what is building a powerful profiling tool? And what is profiling tool? Why how we to build that? I want to say that there is a very old Chinese saying. I don't know whether you have seen that before. It's in Chinese, we call it the 公寓善其事,必先利其其. In English, it means that if you want to make good work, you have to sharpen your tools. You can see, oh, sorry, you can see this interesting picture. If you want your work much good, all the efficiency, so you must sharpen your tools. So from the previous slides, we see that we want to help our computing run faster. We have to make our computer components, data throughput components and communication components know bottleneck as fast as possible. So first of all, we have to see whether it's fast enough or whether there are bottlenecks. So for the computer components, we are not only how to see the utilization rate, like the user says, and so on, but we also have to see the actual floating pound speed while our application is running. And which component or which instruction size has been provided that floating pound. Besides, for the memory bandwidth or the PCI bandwidth or the IO bandwidth, although the great index to see whether there are bottlenecks for our data input into the compute component. And also, the infinite band are the opa bandwidth. Here we list the three figures here. You can see that here is the floating pound speed while our application is running. And here is a memory bandwidth. We can see that some application that you are nearly 200 gigabytes in a single node. So maybe that is some kind of bottleneck. So somebody may ask, we can use some open source or Intel's Weetium to do that. But I want to emphasize is that because our supercomputer is a big or huge machine, so it's not desktop service, desktop machine. So your work had to just as the forest, you just not have to see the single trees, but you have to see the whole behavior of what the forest have to do. So the perfect tool is very important. In the following, I will show several cases to see why the perfect tool is very important. Here I list the first example to show the powerful tools real power. Okay, here I list the CIDR outer atmospheric modeling system that is WALF. We know that this is a very complex model and some of the algorithm may be old. So and it's also a computing intensive application. So if you are going to make CIDR run much faster, we must do some computing or communication optimization. Our work for this optimization is doing for one of our customers in China. They want to do a simulation of the North China's weather forecast, which covers about 60 hours of weather forecasting. And they want to do that is the total computing process would be less than two hours, which includes the WPS process, WALF computing process, and also the post-processing process. However, and the first, our customer claims that the fastest time, the fastest time of only the WALF computing would last more than 150 minutes. So how can we solve this problem? First of all, we have to use a tool to see what's happened while the WALF is running. There we list the floating pound speed while the WALF is running. And we can see that there are a lot of gaps between the floating pound, which means that there is no actual computing. Since we analyzed into details, we found that these are a lot of IO process. So first of all, maybe we have to optimize our IO bandwidth. Besides, we have CZAN and more CZAN, the floating pound speed is very low. And the inflection size may be not in efficiency, but this is a little work. So first of all, we have to optimize our IO bandwidth. And we did three things. Three parts. The first one is electrical performance, and we use a quick IO in our name list. And also the P9CF, you can see that after optimization, and the gaps are nearly disappeared. But there is a big problem in that. You can see that maximum of the network flux has been increased. And there is a continuous feature of the network flux. This may affect our application's scalability. So the next one, what we have to do is to slow down or to let you down the maximum of the network flux. What we do is to make the MPI process fewer, because we know that if there are a lot of MPI process with the communication, the communication is very complex. So if we can use some open P mode, the communication complexity may be lower down. That is what we have done. And we can see that after optimization, the network flux has been down very efficiently, and the performance has been improved. And also after the MPI optimization, the memory has also been dropped, which is very useful for our performance improvement. And we can see that we just did a very simple optimization. It's very simple, just by seeing what's happened for the application running. You can see that we have a very great performance improvement. So since we show that the real power of the providing tool. But I have already shown that, you know that from the application features by the providing tool, we also have the running feature is that the floating points are not very high, and some of the kind of the interaction sites have not been efficiently used. So the next thing of our optimization strategy is that you have to make the new hardware or new architectures much more efficiently. So that is the hardness of the state of architecture. Okay. Here, I show why this is very important. Because if we are, application can use the new architecture and the new instruction site, it means that the machine's performance will be released fully. Here we show three cases that what we have done in the Tianhe 2 supercomputer in China. You know that Tianhe 2 is an architecture with Intel Xeon, CPUs, and also the mic processors. So we optimize these applications on this machine and with the lamps, we can scale it to more than 24,000 cars and the quantum space with more than 28,000 cars. And also, this is the GTC's plus simulation application, which we both use the mic and the Xeon processors, and we can use more than 400,000 cars on this machine. So it is very useful for the state of architecture optimization. For details, I will show you a case to show how can we do this work. This is another global scale atmospheric modeling system, which is developed by one of our customers. And this simulates the global weather forecast with a resolution of 0.125 degrees. And they want to make the simulation less than three hours. And first, they use more than 8,000 cars, but the simulation time is very long, more than one day. So we have to analyze what's wrong with the application. First of all, we list the most time-causing consuming functions and also the program parts of what happened while the program is running. We can see that the most of the time-consuming part is model integration, which we refer to the Hamhoff's equation. It's nearly 50%. And here we list the features while the application is running. We also can see that the total floating pound is not very high for today's hardware architectures. And we can see that most of the interaction sites are provided by SSE. However, the most powerful interaction sites has not been used. So this is the very important thing for us to optimize. We know that the original code, say, so the Hamhoff's equation is to decrease into this form. And the this form is as AX equals B. We know that this is to solve a sparse matrix. We know that this form is a common one. There are a lot of commercial or open source mathematics libraries that have been optimized for this kind of computing. So if we can use highly optimized code for this computing, maybe we can accelerate the Hamhoff's equations computing. That is what we have done. For more things, if we can lower the number of iterations, because we know that some of the, usually we solve the Hamhoff's equation by interaction method. So if we can lower the number of iterations, the total running time would be shorted. OK? And the original one, we can see that they are using the GCR method. So we make, we under just a preconditioner. However, the total improvement is very bad. For another thing, we do some more works to transfer preconditioner GCR method to a GMR M method with MKL library and hyper preconditioners. And this is a very great improvement for the computing. And we can see that after this optimization, the total floating pound has been improved nearly twice. And the instruction sites has been improved from the SSE to the AVX. So we can see that we can efficiently use the newly architectures. And here is total time. And you can see that the model integration time has been increased 61%, and the total running time has also increased 40%. We can see that the number of iterations has been reduced from 32 to 8 iterations. So it is a good improvement for our work. And for another case is that we have to do some more works for our supercomputers applications. And here we can see that most of the supercomputers centers around the world will run a lot of applications. And some of the applications like WASP is very important. WASP is applications for the material science and quantum physics application. And sometimes it will take more than 12 or 20% of CPU hours around the supercomputers centers. Because WASP is a feature, it's just like LIMPAC. So you know that the power consuming for this kind of applications is very high. Maybe it takes 30% or 50% of the total power consuming consumption. So it is very important for us to optimize the WASP application. And we take very famous, our hot research field for the material science. We use a Heusler alloy, which contains about nearly 300 atoms. And we use the plane wave-based site. And you know that the computing amount of this case will be some kind of large. Well, we do the same thing. We see what happened while the application is running. And you can see that nearly half of the time consuming is caused by the DGEMM functions, which is to solve the eggmaning problem. And also we can see that nearly the other half is caused by the MPI withdrawal and MPI all reduced functions, which means that the communication is banned since it will affect our applications cannot make a big scalability. So we have to see what's happened. And we see that the floating pounds is not very high. And the interaction sites only use SD and start-ups, previous optimization we showed you. And we did three things, is that first we have to optimize these eggmaning functions, and then we have to do some memory access optimization, because we know that what we use the plane wave-based site, which in the memory, it will be a lot of arrays. So the memory access would be very intensive. So we have to optimize memory access, like both settings and also the coding. Besides, which is very important is that the network, because we can see that nearly half of the time is caused by the MPI functions. So we have to make some of the modeling to make sure the running of the Raspberry is suitable for our supercomputer topology. Okay, so we changed some of the parameters in the in-cars, and also we optimized the collective communication functions. And after optimization, you can see that the floating pounds has been increased very great. And most of the applications running with the AVX512 interaction site. And you can see that the total performance has been increased nearly 228%. Besides, I want to say that to show the different encryption sites performance, you know that with Xeon's E52650 processors, it only have AVX encryption site. And the V3 processors, we have AVX and FME interaction site. And from here to here, we can see that the performance has been increased 90%, which means that the FME interaction site is very efficient. And also, from the V3 processor to the newly Xeon scalable family processors, we can see that the most difference is AVX interaction site and the AVX512 interaction site. We can see that with the vectorization from 256 to 512, the performance has been increased nearly 17%. So that is to show why the state of the art hardware is very important for us to optimize our applications. And here, we showed the optimization network flux. And you can see that this is a non-persistent network, which is conducive to improving game scalability because sometimes we can harden the communications into our computing part. However, the scalability of VASP because it's a commercial code, you know that it's not very good. So if we want to make this kind of application into one into millions of cars, this kind of commercial code may not work. We have to do a lot of work for the commercial codes, physical models to suitable to our supercomputers architectures. So here, come to the third part is that we must leverage the latest mode algorithm, which means that we must optimize or develop new algorithm or new physical models which is suitable for today's supercomputers architectures and topologicals. Here I list a timeline, which is also in quantum physics areas. And we can see that a lot of the algorithm or the series are some kind of old. And I say old because these algorithms are developed decades before. So today's a lot of the applications like the VASP applications that I just show you is also based on this kind of series. So because just this kind of series, the parallel modes degree has been limited into this part. So the total scalability may be very low. That is why the VASP scalability is not good. So we must do the mode optimization by two parts, the applications sense and also the computing sense. So it's some kind of interdisciplinary work. Because for our application sense, we must think or must develop our algorithm or our mode to suitable for our computers. And also for our computer scientists, we must think our new architecture to suitable for applications. So if we can make these two parts work together, so large-scale complication would be reached. And the first case we show here is to show how can we do that. It's also work we did for our customers in China. This is kind of code for simulating new spin phonics. Because we know that today's processors like the desktop, our supercomputers are based on a lot of semiconductors, the silicon. And we use the voltage high and low to represent bit 1 or 0. However, if we can use the electron spin, which is at the spin right up and down to represent the bit 1 or 0, the total power consumption will be reduced nearly 100,000 times. So if we can use spin phonics to make our processors, maybe our supercomputers, power problem may be not a problem. And also maybe our cell phones do not need to recharge every day. However, their code is very bad. So the work is very great. The code is very bad. They only can simulate, just use turn cars, very low, just desktop. And the matrix scale is less than the power of 10 of the four. And they can only simulate the three electrons. So we analyze the problem. They use the configuration to answer method, which just like our atoms. We know that for most of the time, the electrons are located in a very low energy state. However, they make the simulation with full consideration of the non-occupied state. We know that there are a lot of ways for the computation. So for our optimization mode, the first thing is that we can get rid of this useless computing. And besides, they use the four metrics to record the sparse metrics, which means that there are a lot of normally waste into records zero ones. So we do this optimization just to change the fourth CI to the partial CI. And also, we change the QR method with an integration method, which is suitable for parallelism. And also, we optimize the vector interactions and also the collective communications. And after that, you can see that there is a very great performance improvement. The total parallel degree is from 10 cars to nearly 4,000 of cars. And the most important is that they can do a lot of new works. In Cisco, they have submitted a lot of works in the FIT review letters and some of the famous scientific research articles. And the last, I will show another case why the modeling innovation is very important here. I know a lot of your professors here are doing the global climate simulation work because you know that the climate simulation is complex because they will couple a lot of computing components or physical components to make the simulation. And with resolution increase, the total number of calculation will increase very much. So, it is difficult for us to make a very exact, our climate change just using little or small supercomputers. So we must make our code run as large as possible. However, our customers in China make these codes very poor because they can only simulate the code in nearly less than a thousand of cars, several hundred. And for someone to simulate a decade simulation will take over two days. So this is a big problem. We did the optimization together with them and we found that the five parts of the application list here and you can see that most of the time consuming part except the ATM component, the other four is MPI. So we can see that there is a big problem. The ATM is a bottleneck. And we did some details, analyzed of this code. And we found that this kind of code is some kind of ATM driver, which means that the other component like the land and ATM and ice and ocean are coupled with this ATM driver. So the ATM would be a bottleneck. So the most of the thing is that we have to change this pseudo-parallel mode into real parallel mode. So we changed the ATM driver to a CPU driver, which means that the ATM bottlenecks have been bricked. And also we optimized ATM with the MPI plus open MP mode. And you can see that after this kind of optimization, the total performance has been increased nearly twice. And also the ATM has also increased 20%. So today I showed our methods to realize large-scale computing. We used three strategies to take the runtime performance data with a powerful profiling tool. And with the profiling data, we know the applications features and we know whether there is a bottleneck. So we can do a set of technologies innovation and also we can do the interdisciplinary innovation for the new models to optimize our application work so we can achieve a great application performance. However, maybe you can consider that the most important thing and the difficult thing is the inter-planaried innovation because we lack of these talents to do that. Some of our computer scientists only famous our computing world. However, our scientists from other research fields like physics and chemical and so on, they only know their own research work. They do not understand the computer science. So there is a big gap. So the most of the important thing may be to treat the people who know both parts. So one more thing I want to introduce is the ASA student computer challenge. This computer challenge provides a stage for the student practicing large-scale efficient optimization. This kind of event has been held eight years. It started in 2012. And over 7,000 young talents have drawn this event and more than 300 universities around the world have come into this wonderful event. And a lot of the young talents in this event has made a very good job. You can see that the AS14, some of the young talents have optimized three-dimensional electric wave model on the Tianhe 2 supercomputers with more than 200,000 cars. And the AS17, some of the young talents use the Sunway Type-Lite to simulate a golden bell press nominated application maximum with more than 10,000 cars. So you can see that the young talents are very powerful for tomorrow's future supercomputers work. So maybe I can release a video to you to see what happens or what is the ASA event. And we also like all of you are professors to encourage your students to come into this event and have a great experience with large-scale application. So this is a video we take from the ASA 19 event. We are all focused. We work together. We have the same goal. To explore more unknowns. To find more possibilities. To achieve a better future. Five days, 20 teams, new competitions. Set sail again. I think in the previous competitions, there was only one competition. Now there is cooperation. It is a new thing. And I think everyone will invest in this thing. It is a good new competition project. This time, the super team's design strengthened our cooperation and communication with other teams. We share some of our experience and help each other to fight for the same thing. It is a good new experience. Everyone is kind of competitive. But still very friendly and helpful. You can easily talk to other teams. And they will actually help you. They are willing to help you. You know that this is a very interesting event. We are focused on making our undergraduate student training for tomorrow's super computer scientists. This is what I show today. Thank you very much for your time. Thank you. Any questions from the audience? I have a question because it is really kind of a dream. You are doing a lot of work for the optimization of existing applications. You have three pillars. You can improve the code. How actually it happens? Users coming to your firm and you have a contract. You are working together. It is a team work. It is for how many years? Typically, how long does it take? How it happens? When they pay your team or the team is going to the institution? For example, this climate people coming to your firm and say, we have this code and we want to improve it. We want to optimize it. And how then it happens? You conclude a contract with this firm. Most of the time, we work with our customers together. First of all, you know that optimization will take a lot of, take a very long time to do that work and it is very difficult. And we have special application teams. That is my team. And we have experts to work with our customers together to simulate and to analyze what their application is and what their aim is for. And after that, we will make some of the tests with their applications or their workloads and then find the bottlenecks and then we do the optimization. Maybe some kind of the computing work is doing by our teams and some of the physical models, you can see that maybe it's, maybe work by our customers, but we have to work together. You know that it's interdisciplinary work. And you already said, we will last several months or even years. So before we achieve a very good, a very good or useful performance. That is total procedure. Actually, it was how to become your customer. Sorry. Yeah, yeah, yeah. Something is my colleague. Yeah. Okay. You show very nice. What is your policy of feeding back your optimization to the community, to the project that, to the open source projects that you use. Okay. Does this optimization actually find their way back into the main development trunk? Okay. And present we have not put our optimized code into the communication, into the community. You know that there are a lot of the codes developed by our customers yourself, just in host code. So maybe this kind of codes we cannot just upload into the GitHub and so on. And for the other open source projects, and for the other open source optimization like the lamps or context based on so on, we showed in the TFA tool express, TNH2 supercomputers. We haven't uploaded into the community because maybe the, you know, the computer centers won't like us to do that. I don't know why. Okay. So I have a question. I mean, your company sells both hardware and I guess software services or at least you have shown that software is very important to make good use of the hardware. If you would have to advise people how to invest their money, you know, would you say 50% for the hardware, 50% for the software or what do you think is a good ratio of investment? Oh, you mean for the solution, the HPC solution. Okay. If you look for the holistic solution, you know, the system that runs the application best, how would you invest? For my opinion, I think the software is more important than hardware. So I think you have to put more than 50% on the software, not the hardware. Good. I will take your answer to whoever finds this. Does your boss agree with you? Maybe not. So it's my own opinion. Okay. Okay. Okay. If no more questions, let's thank our speaker again. Thank you very much.
world. And InSpar has a lot of research and For many years, the appearance of HPC innovation some computing or communication optimization. Our work for this optimization is doing for one of thing of our optimization strategy is that you optimization. For details, I will show you a case mathematics libraries that have been optimized the world will run a lot of applications. And the applications running with the AVX512 our supercomputers are based on a lot of realize large-scale computing. We used three strategies to take the runtime performance data profiling data, we know the applications features communication with other teams. We share some of optimization of existing applications. You have our teams and some of the physical models, you