Research on intelligent analysis and visualization of big data based on ELK

In the era of big data, the analysis and processing of data has many difficult problems, such as difficult integration of heterogeneous data, inability to guarantee timeliness, weak analysis ability and weak visualization ability. In response to these problems, this paper conducts secondary development of the open-source ELK (Elasticsearch, Logstash, Kibana) log analysis system to better analyze and visualize the data. In order to show the effectiveness of the system more clearly, this paper takes the collected operation and maintenance data of a certain server as an example, and presents the important information obtained by searching, summarizing and analyzing the logs in the form of various dynamic charts or visualizations. It can be seen from the results that the system can better and more vividly display data distribution and operation and maintenance and provide a method and tool for data processing and visual presentation, which can help people better grasp information or make auxiliary decision-making.