140623
Optimal Algorithms for Finding User Access Sessions from Very Large Web Logs
Although efficient identification of user access sessions from very large web logs is an unavoidable data preparation task for the success of higher level web log mining, little attention has been paid to algorithmic study of this problem. In this paper we consider two types of user access sessions, interval sessions and gap sessions. We design two efficient algorithms for finding respectively those two types of sessions with the help of some proposed structures. We present theoretical analysis of the algorithms and prove that both algorithms have optimal time complexity and certain error-tolerant properties as well. We conduct empirical performance analysis of the algorithms with web logs ranging from 100 megabytes to 500 megabytes. The empirical analysis shows that the algorithms just take several seconds more than the baseline time, i.e., the time needed for reading the web log once sequentially from disk to RAM, testing whether each user access record is valid or not, and writing each valid user access record back to disk. The empirical analysis also shows that our algorithms are substantially faster than the sorting based session finding algorithms. Finally, optimal algorithms for finding user access sessions from distributed web logs are also presented.
7399730
An Almost Linear Time 2.8334-Approximation Algorithm for the Disc Covering Problem
The disc covering problem asks to cover a set of points on the plane with a minimum number of fix-sized discs. We develop an O(n(logn)2(loglogn)2) deterministic time 2.8334-approximation algorithm for this problem. Previous approximation algorithms [7,3,6], when used to achieve the same approximation ratio for the disc covering problem, will have much higher time complexity than our algorithms.
11075192
Fast Construction of Generalized Suffix Trees Over a Very Large Alphabet
The work in this paper is motivated by the real-world problems such as mining frequent traversal path patterns from very large Web logs. Generalized suffix trees over a very large alphabet can be used to solve such problems. However, traditional algorithms such as the Weiner, Ukkonen and McCreight algorithms are not sufficient assurance of practicality because of large magnitudes of the alphabet and the set of strings in those real-world problems. Two new algorithms are designed for fast construction of generalized suffix trees over a very large alphabet, and their performance is analyzed in comparison with the well-known Ukkonen algorithm. It is shown that these two algorithms have better performance, and can deal with large alphabets and large string sets well.
17642246
Evolving an Experience Base for Software Process Research
Since 1996 the USC Center for Software Engineering has been accumulating a large amount of software process experience through many real-client project software engineering practices. Through the application of the Experience Factory approach, we have collected and evolved this experience into an experience base (eBASE) which has been leveraged successfully for empirically based software process research. Through eBASE we have realized tangible benefits in automating, organizational learning, and strategic advantages for software engineering research. We share our rationale for creating and evolving eBASE, give examples of how the eBASE has been used in recent process research, discuss current limitations and challenges with eBASE, and what we hope to do achieve in the future with it.
17921413
A self-learning worm using importance scanning
The use of side information by an attacker can help a worm speed up the propagation. This philosophy has been the basis for advanced worm scanning mechanisms such as hitlist scanning, routable scanning, and importance scanning. Some of these scanning methods use information on vulnerable hosts. Such information, however, may not be easy to collect before a worm is released. Questions then arise whether and how a worm can self-learn and use such information while propagating, and how virulent the resulting worm may be. In this paper, we design a self-learning worm using importance scanning. An optimal yet practical importance-scanning strategy is derived based on a new metric. A self-learning worm is demonstrated to have the ability to accurately estimate the underlying vulnerable-host distribution if a sufficient number of infected hosts are observed. Experimental results based on parameters chosen from Code Red show that after accurately estimating the distribution of vulnerable hosts, a self-learning worm can spread much faster than a random-scanning worm, a permutation-scanning worm, and a Class A routing worm. Some guidelines for detecting and defending against such self learning worms are also discussed.
45057829
Preclassification of handwritten Chinese characters based on basic stroke substructures
A method for preclassification of handwritten Chinese characters is presented. A set of basic stroke substructures is defined using the consistent stroke connection relations. A knowledge guided recognition process is employed to identify the types of the extracted basic stroke substructures found in a handwritten character. Then a 1-D character coding scheme is given to represent the character and the code can be also used for character preclassification.
38006851
Approximation Algorithms for Independent Sets in Map Graphs
This paper presents polynomial-time approximation algorithms for the problem of computing a maximum independent set in a given map graph G with or without weights on its vertices. If G is given together with a map, then a ratio of 1+δ can be achieved in O(n2) time for any given constant δ > 0, no matter whether each vertex of G is given a weight or not. In case G is given without a map, a ratio of 4 can be achieved in O(n7) time if no vertex is given a weight, while a ratio of O(log n) can be achieved in O(n7 log n) time otherwise. Behind the design of our algorithms are several fundamental results for map graphs; these results can be used to design good approximation algorithms for coloring and vertex cover in map graphs, and may find applications to other problems on map graphs as well.
16272457
SimFusion: measuring similarity using unified relationship matrix
In this paper we use a Unified Relationship Matrix (URM) to represent a set of heterogeneous data objects (e.g., web pages, queries) and their interrelationships (e.g., hyperlinks, user click-through sequences). We claim that iterative computations over the URM can help overcome the data sparseness problem and detect latent relationships among heterogeneous data objects, thus, can improve the quality of information applications that require com- bination of information from heterogeneous sources. To support our claim, we present a unified similarity-calculating algorithm, SimFusion. By iteratively computing over the URM, SimFusion can effectively integrate relationships from heterogeneous sources when measuring the similarity of two data objects. Experiments based on a web search engine query log and a web page collection demonstrate that SimFusion can improve similarity measurement of web objects over both traditional content based algorithms and the cutting edge SimRank algorithm.