Prediction of Typhoon Intensity Based on Gated Attention Transformer

Typhoons occur frequently in the southeast coastal areas of China. Predicting the intensity of the coming typhoon accurately is of great significance for timely rescue and disaster relief. Various recurrent neural networks and their variants have good effects in short-sequence prediction, especially the attention model. But performance degradation often occurs in long sequence prediction. We propose a transformer model with attention gates mechanism (Gated Attention Transformer, GATran). On the basis of multi-headed attention in Transformer, an binary attention gates generator based on Long Short Term Memory network (LSTM) is added. The main function of these attention gates is to control whether attention is assigned to input positions. Closing part of the gates can make the model pay more attention to important input positions. By using the dataset of typhoons in China offshore from 1945 to 2020 published by the Chinese Academy of Sciences, we compared GATran with Lstm, Seq2seq, and traditional Transformer experimentally. The results show that the actual typhoon's wind speed error of GATran model is lower than other models in long sequence prediction.