6847311
On the Complexity of Rocchio's Similarity-Based Relevance Feedback Algorithm
In this paper, we prove for the first time that the learning complexity of Rocchio's algorithm is O(d + d2(log d + log n)) over the discretized vector space {0,...,n–1}d, when the inner product similarity measure is used. The upper bound on the learning complexity for searching for documents represented by a monotone linear classifier (q,0) over {0,...,n–1}d can be improved to O(d + 2k(n–1)(log d + log(n–1))), where k is the number of nonzero components in q. An Ω((d2)log n) lower bound on the learning complexity is also obtained for Rocchio's algorithm over {0,...,n–1}d. In practice, Rocchio's algorithm often uses fixed query updating factors. When this is the case, the lower bound is strengthened to 2$^{{\it \Omega}(d)}$ over the binary vector space {0,1}d. In general, if the query updating factors are bounded by O(nc) for some constant c≥ 0, an ${\it \Omega}(n^{d-1-c}/(n-1))$ lower bound is obtained over {0,...,n–1}d.
140623
Optimal Algorithms for Finding User Access Sessions from Very Large Web Logs
Although efficient identification of user access sessions from very large web logs is an unavoidable data preparation task for the success of higher level web log mining, little attention has been paid to algorithmic study of this problem. In this paper we consider two types of user access sessions, interval sessions and gap sessions. We design two efficient algorithms for finding respectively those two types of sessions with the help of some proposed structures. We present theoretical analysis of the algorithms and prove that both algorithms have optimal time complexity and certain error-tolerant properties as well. We conduct empirical performance analysis of the algorithms with web logs ranging from 100 megabytes to 500 megabytes. The empirical analysis shows that the algorithms just take several seconds more than the baseline time, i.e., the time needed for reading the web log once sequentially from disk to RAM, testing whether each user access record is valid or not, and writing each valid user access record back to disk. The empirical analysis also shows that our algorithms are substantially faster than the sorting based session finding algorithms. Finally, optimal algorithms for finding user access sessions from distributed web logs are also presented.
5722664
A Visualization System using Data Mining Techniques for Identifying Information Sources
The Visual Analysis System (VAS) was developed to couple emerging successes in data mining with information visualization techniques in order to create a richly interactive environment for information retrieval from the World Wide Web. VAS's retrieval strategy operates by first using a conventional search engine to form a core set,of retrieved documents. This core set is expanded by crawling from links in these documents to form a much larger set. The link topology of the larger set is then examined using data mining techniques to identify pages likely to be both relevant to the query and reliable as sources of information. Information visualization techniques are used to display the filtered set in a form amenable to the use of perceptual processes to filter and guide further user directed search. (Contains 16 references.) (MES) Reproductions supplied by EDRS are the best that can be made from the original document. N tr) 00 7171A Visualization System using Data Mining Techniques for Identifying Information Sources Richard H. Fowler, Tarkan Karadayi, Zhixiang Chen, Xiannong Meng, Wendy A. Lawrence Fowler Department of Computer Science University of Texas Pan American United States {fowler, tkaradayi, chen, meng, wfowler @panam.edu} PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY
7399730
An Almost Linear Time 2.8334-Approximation Algorithm for the Disc Covering Problem
The disc covering problem asks to cover a set of points on the plane with a minimum number of fix-sized discs. We develop an O(n(logn)2(loglogn)2) deterministic time 2.8334-approximation algorithm for this problem. Previous approximation algorithms [7,3,6], when used to achieve the same approximation ratio for the disc covering problem, will have much higher time complexity than our algorithms.
28060413
The Architecture of Yarrow: A Real-Time Intelligent Meta-Search Engine
44651708
Personalized Web Search with Clusters
6845908
Mining ratio rules via principal sparse non-negative matrix factorization
Association rules are traditionally designed to capture statistical relationship among itemsets in a given database. To additionally capture the quantitative association knowledge, Korn et al. (1998) proposed a paradigm named ratio rules for quantifiable data mining. However, their approach is mainly based on principle component analysis (PCA) and as a result, it cannot guarantee that the ratio coefficient is nonnegative. This may lead to serious problems in the rules' application. In this paper, we propose a method, called principal sparse nonnegative matrix factorization (PSNMF), for learning the associations between itemsets in the form of ratio rules. In addition, we provide a support measurement to weigh the importance of each rule for the entire dataset.
8030523
Video management in peer-to-peer systems
Providing scalable video services in a peer-to-peer (P2P) environment is challenging. Since videos are typically large and require high communication bandwidth for delivery, many peers may be unwilling to cache them in whole to serve others. In this paper, the authors addressed two fundamental research problems in providing scalable P2P video services, namely (1) how a host can find enough video pieces, which may scatter among the whole system, to assemble a complete video, and (2) given a limited buffer size, what part of a video a host should cache. A new distributed video management technique was proposed. This scheme organizes hosts into a number of cells, each of which is a distinct set of hosts which together can supply a video in its entirety. A client looking for a video can stop its search as soon as it finds a host that caches any part of the video. Caching operations can be coordinated within each cell to balance data redundancy in the system. The extensive study on a Gnutella-like simulation network shows convincingly the performance advantage of the new scheme.
19262472
Relevance Feedback and Learning in Content-Based Image Search
A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.
1777247
Web-page classification through summarization
Web-page classification is much more difficult than pure-text classification due to a large variety of noisy information embedded in Web pages. In this paper, we propose a new Web-page classification algorithm based on Web summarization for improving the accuracy. We first give empirical evidence that ideal Web-page summaries generated by human editors can indeed improve the performance of Web-page classification algorithms. We then propose a new Web summarization-based classification algorithm and evaluate it along with several other state-of-the-art text summarization algorithms on the LookSmart Web directory. Experimental results show that our proposed summarization-based classification algorithm achieves an approximately 8.8% improvement as compared to pure-text-based classification algorithm. We further introduce an ensemble classifier using the improved summarization algorithm and show that it achieves about 12.9% improvement over pure-text based methods.
46287200
A Parallel Algorithm for Finding a Triconnected Component Separator with an Application