17921413
A self-learning worm using importance scanning
The use of side information by an attacker can help a worm speed up the propagation. This philosophy has been the basis for advanced worm scanning mechanisms such as hitlist scanning, routable scanning, and importance scanning. Some of these scanning methods use information on vulnerable hosts. Such information, however, may not be easy to collect before a worm is released. Questions then arise whether and how a worm can self-learn and use such information while propagating, and how virulent the resulting worm may be. In this paper, we design a self-learning worm using importance scanning. An optimal yet practical importance-scanning strategy is derived based on a new metric. A self-learning worm is demonstrated to have the ability to accurately estimate the underlying vulnerable-host distribution if a sufficient number of infected hosts are observed. Experimental results based on parameters chosen from Code Red show that after accurately estimating the distribution of vulnerable hosts, a self-learning worm can spread much faster than a random-scanning worm, a permutation-scanning worm, and a Class A routing worm. Some guidelines for detecting and defending against such self learning worms are also discussed.
9213286
Modeling the spread of active worms
Active worms spread in an automated fashion and can flood the Internet in a very short time. Modeling the spread of active worms can help us understand how active worms spread, and how we can monitor and defend against the propagation of worms effectively. In this paper, we present a mathematical model, referred to as the Analytical Active Worm Propagation (AAWP) model, which characterizes the propagation of worms that employ random scanning. We compare our model with the Epidemiological model and Weaver's simulator. Our results show that our model can characterize the spread of worms effectively. Taking the Code Red v2 worm as an example, we give a quantitative analysis for monitoring, detecting and defending against worms. Furthermore, we extend our AAWP model to understand the spread of worms that employ local subnet scanning. To the best of our knowledge, there is no model for the spread of a worm that employs the localized scanning strategy and we believe that this is the first attempt on understanding local subnet scanning quantitatively.
6185019
iFind: a web image search engine
iFind© (v1.0) is a web-based image retrieval system developed at Microsoft Research China. It provides the functionalities of text based image search, query by image example, and their combination. Images in the database are indexed by their lowlevel (visual) features, high-level (semantic) features (collected from image’s environment, e.g., the Web), and optionally, annotations if they are available. The key technology in the system is the integrated semantics and feature based image retrieval and relevance feedback approach and data mining of users’ feedback log. When the user provides feedback images, the system can refine the retrieval result based on the user’s feedback. In the meantime, the system updates the annotation of feedback images by increasing the linkage to the positive examples’ annotation and decreasing the linkage to the negative examples’ annotation. The updated annotation can further help to improve image retrieval results of the system in later use. Log mining can also help yield more accurate retrieval results by refining the semantic features. 2. Implementation
16533233
Improving text classification using local latent semantic indexing
Latent semantic indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called "local relevancy weighted LSI" to improve text classification by performing a separate single value decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension.
3449616
Fast and memory-efficient regular expression matching for deep packet inspection
Packet content scanning at high speed has become extremely important due to its applications in network security, network monitoring, HTTP load balancing, etc. In content scanning, the packet payload is compared against a set of patterns specified as regular expressions. In this paper, we first show that memory requirements using traditional methods are prohibitively high for many patterns used in packet scanning applications. We then propose regular expression rewrite techniques that can effectively reduce memory usage. Further, we develop a grouping scheme that can strategically compile a set of regular expressions into several engines, resulting in remarkable improvement of regular expression matching speed without much increase in memory usage. We implement a new DFA-based packet scanner using the above techniques. Our experimental results using real-world traffic and patterns show that our implementation achieves a factor of 12 to 42 performance improvement over a commonly used DFA- based scanner. Compared to the state-of-art NFA-based implementation, our DFA-based packet scanner achieves 50 to 700 times speedup.
1853937
Numerically Stable Real Number Codes Based on Random Matrices
Error correction codes defined over real-number field have been studied and recognized as useful in many applications. However, most real-number codes in literature are quite suspect in their numerical stability. In this paper, we introduce a class of real-number codes based on random generator matrices over real-number fields. Codes over complex-number field are also discussed. Experiment results demonstrate our codes are numerically much more stable than existing codes in literature.
17550632
OCFS: optimal orthogonal centroid feature selection for text categorization
Text categorization is an important research area in many Information Retrieval (IR) applications. To save the storage space and computation time in text categorization, efficient and effective algorithms for reducing the data before analysis are highly desired. Traditional techniques for this purpose can generally be classified into feature extraction and feature selection. Because of efficiency, the latter is more suitable for text data such as web documents. However, many popular feature selection techniques such as Information Gain (IG) andχ2-test (CHI) are all greedy in nature and thus may not be optimal according to some criterion. Moreover, the performance of these greedy methods may be deteriorated when the reserved data dimension is extremely low. In this paper, we propose an efficient optimal feature selection algorithm by optimizing the objective function of Orthogonal Centroid (OC) subspace learning algorithm in a discrete solution space, called Orthogonal Centroid Feature Selection (OCFS). Experiments on 20 Newsgroups (20NG), Reuters Corpus Volume 1 (RCV1) and Open Directory Project (ODP) data show that OCFS is consistently better than IG and CHI with smaller computation time especially when the reduced dimension is extremely small.