11939257
Building a web thesaurus from web link structure
Thesaurus has been widely used in many applications, including information retrieval, natural language processing, and question answering. In this paper, we propose a novel approach to automatically constructing a domain-specific thesaurus from the Web using link structure information. The proposed approach is able to identify new terms and reflect the latest relationship between terms as the Web evolves. First, a set of high quality and representative websites of a specific domain is selected. After filtering out navigational links, link analysis is applied to each website to obtain its content structure. Finally, the thesaurus is constructed by merging the content structures of the selected websites. The experimental results on automatic query expansion based on our constructed thesaurus show 20% improvement in search precision compared to the baseline.
14828051
MRSSA: an iterative algorithm for similarity spreading over interrelated objects
We introduce the Multiple Relationship Similarity Spreading Algorithm (MRSSA) to enhance IR effectiveness. This method has similarity computed in an iterative "spreading" fashion for multiple object types, combining both inter- and intra-object relationships. We demonstrate the value of this approach in the context of the WWW, where the key objects are web pages and queries, Relationships considered are derived from hyperlinks (in- and out-links) and click-through logs.
432736
ReCoM: reinforcement clustering of multi-type interrelated data objects
Most existing clustering algorithms cluster highly related data objects such as Web pages and Web users separately. The interrelation among different types of data objects is either not considered, or represented by a static feature space and treated in the same ways as other attributes of the objects. In this paper, we propose a novel clustering approach for clustering multi-type interrelated data objects, ReCoM (Reinforcement Clustering of Multi-type Interrelated data objects). Under this approach, relationships among data objects are used to improve the cluster quality of interrelated data objects through an iterative reinforcement clustering process. At the same time, the link structure derived from relationships of the interrelated data objects is used to differentiate the importance of objects and the learned importance is also used in the clustering process to further improve the clustering results. Experimental results show that the proposed approach not only effectively overcomes the problem of data sparseness caused by the high dimensional relationship space but also significantly improves the clustering accuracy.
17711119
IRC: an iterative reinforcement categorization algorithm for interrelated Web objects
Most existing categorization algorithms deal with homogeneous Web data objects, and consider interrelated objects as additional features when taking the interrelationships with other types of objects into account. However, focusing on any single aspects of these interrelationships and objects does not fully reveal their true categories. In this paper, we propose a categorization algorithm, the iterative reinforcement categorization algorithm (IRC), to exploit the full interrelationships between the heterogeneous objects on the Web. IRC attempts to classify the interrelated Web objects by iterative reinforcement between individual classification results of different types via the interrelationships. Experiments on a clickthrough log dataset from MSN search engine show that, with the Fl measures, IRC achieves a 26.4% improvement over a pure content-based classification method, a 21% improvement over a query metadata-based method, and a 16.4% improvement over a virtual document-based method. Furthermore, our experiments show that IRC converges rapidly.
459986
A New Statistical Approach To Chinese Pinyin Input
Chinese input is one of the key challenges for Chinese PC users. This paper proposes a statistical approach to Pinyin-based Chinese input. This approach uses a trigram-based language model and a statistically based segmentation. Also, to deal with real input, it also includes a typing model which enables spelling correction in sentence-based Pinyin input, and a spelling model for English which enables modeless Pinyin input.
6802371
Improving web search results using affinity graph
In this paper, we propose a novel ranking scheme named Affinity Ranking (AR) to re-rank search results by optimizing two metrics: (1) diversity -- which indicates the variance of topics in a group of documents; (2) information richness -- which measures the coverage of a single document to its topic. Both of the two metrics are calculated from a directed link graph named Affinity Graph (AG). AG models the structure of a group of documents based on the asymmetric content similarities between each pair of documents. Experimental results in Yahoo! Directory, ODP Data, and Newsgroup data demonstrate that our proposed ranking algorithm significantly improves the search performance. Specifically, the algorithm achieves 31% improvement in diversity and 12% improvement in information richness relatively within the top 10 search results.
9213286
Modeling the spread of active worms
Active worms spread in an automated fashion and can flood the Internet in a very short time. Modeling the spread of active worms can help us understand how active worms spread, and how we can monitor and defend against the propagation of worms effectively. In this paper, we present a mathematical model, referred to as the Analytical Active Worm Propagation (AAWP) model, which characterizes the propagation of worms that employ random scanning. We compare our model with the Epidemiological model and Weaver's simulator. Our results show that our model can characterize the spread of worms effectively. Taking the Code Red v2 worm as an example, we give a quantitative analysis for monitoring, detecting and defending against worms. Furthermore, we extend our AAWP model to understand the spread of worms that employ local subnet scanning. To the best of our knowledge, there is no model for the spread of a worm that employs the localized scanning strategy and we believe that this is the first attempt on understanding local subnet scanning quantitatively.
7582784
Hierarchical Clustering-Merging for Multidimensional Index Structures
The R-tree family index structures are among the most common index structures used in multidimensional databases. To improve the search performance it is very important to reduce the overlap between bounding regions in the R-tree. However the arbitrary insertion order in the tree construction procedure might result in tree structures inefficient in the search operations. In this paper we propose a new technique called Hierarchical Clustering-Merging (HCM) to improve the tree construction procedure of the R-tree family index structures. With this technique we can take advantage of the data distribution information in the data set to achieve an optimized tree structure and improve the search performance.
8030523
Video management in peer-to-peer systems
Providing scalable video services in a peer-to-peer (P2P) environment is challenging. Since videos are typically large and require high communication bandwidth for delivery, many peers may be unwilling to cache them in whole to serve others. In this paper, the authors addressed two fundamental research problems in providing scalable P2P video services, namely (1) how a host can find enough video pieces, which may scatter among the whole system, to assemble a complete video, and (2) given a limited buffer size, what part of a video a host should cache. A new distributed video management technique was proposed. This scheme organizes hosts into a number of cells, each of which is a distinct set of hosts which together can supply a video in its entirety. A client looking for a video can stop its search as soon as it finds a host that caches any part of the video. Caching operations can be coordinated within each cell to balance data redundancy in the system. The extensive study on a Gnutella-like simulation network shows convincingly the performance advantage of the new scheme.
3449616
Fast and memory-efficient regular expression matching for deep packet inspection
Packet content scanning at high speed has become extremely important due to its applications in network security, network monitoring, HTTP load balancing, etc. In content scanning, the packet payload is compared against a set of patterns specified as regular expressions. In this paper, we first show that memory requirements using traditional methods are prohibitively high for many patterns used in packet scanning applications. We then propose regular expression rewrite techniques that can effectively reduce memory usage. Further, we develop a grouping scheme that can strategically compile a set of regular expressions into several engines, resulting in remarkable improvement of regular expression matching speed without much increase in memory usage. We implement a new DFA-based packet scanner using the above techniques. Our experimental results using real-world traffic and patterns show that our implementation achieves a factor of 12 to 42 performance improvement over a commonly used DFA- based scanner. Compared to the state-of-art NFA-based implementation, our DFA-based packet scanner achieves 50 to 700 times speedup.
14422953
On-line learning of rectangles
This paper solves the following open problem: Is there an algorithmfor on-line learning of rectangles <inline-equation><f><pr align="c"><ll>i=1</ll><ul>d</ul></pr></f><?Pub Caret></inline-equation><?Pub Fmt italic>a<subscrpt>i</subscrpt>,a<subscrpt>i</subscrpt>+1,…,b<subscrpt>i</subscrpt>}<?Pub Fmt /italic>over a discrete domain<?Pub Fmt italic>{1,…,n}<supscrpt>d</supscrpt><?Pub Fmt /italic>whose error bound is polylogarithmic in the size<?Pub Fmt italic>n<supscrpt>d</supscrpt><?Pub Fmt /italic> of the domain(i.e. polynomial in <?Pub Fmt italic>d<?Pub Fmt /italic> and log<?Pub Fmt italic>n<?Pub Fmt /italic> )? We give a positive solution byintroducing a new design technique that appears to be of some intereston its own. The new learning algorithm for rectangles consists of<?Pub Fmt italic>2d<?Pub Fmt /italic> separate search strategies thatsearch for the parameters <?Pub Fmt italic>a<subscrpt>1</subscrpt>,b<subscrpt>1</subscrpt>,…,a<subscrpt>d</subscrpt>,b<subscrpt>d</subscrpt><?Pub Fmt /italic>of the target rectangle. A learning algorithm with this type of modulardesign ends to fail because of the well known “credit assignmentproblem”: Which of the <?Pub Fmt italic>2d<?Pub Fmt /italic> localsearch strategies should be “blamed” when the globalalgorithm makes an error? We overcome this difficulty by employing localsearch strategies (“error tolerant binary search”) that areable to tolerate certain types of wrong credit assignments.
Section 4 contains another application of this design technique: analgorithm for learning the union of two rectangles in the plane.