Fault tolerance insuring technique for communication network of multidimensional torus topology
The article describes a method for fault tolerance ensuring of a communication network with a multidimensional torus topology, tested while developing the Angara high-speed interconnect router. The high-speed data transmission is associated with a number of difficulties that should be overcome by the hardware designer to achieve acceptable fault tolerance. Problematics of the issue becomes especially acute with an increase in the number of nodes in the system, since the probability of failures is greatly enhanced with expansion in the number of connections. The work on the fault tolerance ensuring was performed in the following areas: - quality improving of the transmitted signal by parameters fine-tuning of the high-speed transceivers; - automated control of the parameters of the signals eye diagram and optimal parameters selection of the transmitting and receiving sides of the data transmission channel; - developing a fault-tolerant link integrity control algorithm with the ability of the damaged data retransmission; - developing a data flow control mechanism to control the credit information of the virtual channels; - developing network layer algorithms for network state monitoring, and quick routing tables rebuilding. The algorithms and mechanisms described in the article continuity ensuring f the transmitted data flow and their guaranteed delivery over the communication lines between the nodes, exclusion of losses, duplications and distortions of packets during transmission. The conducted measurements have shown that the bit error rate (BER) was about 10-12, which corresponded to the permissible values. The methods described in the article allow obtaining the availability factor of more than 0.99 for large computing systems. At present, they are implemented in the equipment of the serially produced Angara network equipment, and are also widely applied in the design of the second generation Angara high speed interconnect router.