We propose a methodology to detect and classify rootkit exploits. The goal of this research is to provide system administrators, researchers, and security personnel with the information necessary in order to take the best possible recovery actions concerning systems that are compromised by rootkits. There is no such methodology available at present to perform this function. This may also help to detect and fingerprint additional instances and prevent further security instances involving rootkits. A formal framework was developed in order to define rootkit exploits as an existing rootkit, a modification to an existing, or an entirely new rootkit. A methodology was then described in order to apply this framework against rootkits that are to be investigated. We then proposed some new methods to detect and characterize specific types of rootkit exploits. These methods consisted of identifying unique string signatures of binary executable files as well as examining the system call table within the system kernel. We established a Honeynet in order to aid in our research efforts and then applied our methodology to a previously unseen rootkit that was targeted against the Honeynet. By using our methodology we were able to uniquely characterize this rootkit and identify some unique signatures that could be used in the detection of this specific rootkit. We applied our methodology against nine additional rootkit exploits and were able to identify unique characteristics for each of these rootkits. These characteristics could also be used in the prevention and detection of these rootkits.
['John G. Levine', 'Henry L. Owen']
A methodology for detecting and classifying rootkit exploits