40322419
A Quadratic Lower Bound for Rocchio's Similarity-Based Relevance Feedback Algorithm
It is shown in [4] that Rocchio's similarity-based relevance feedback algorithm makes Ωn mistakes in searching for a collection of documents represented by a monotone disjunction of at most k relevant features or terms over the n-dimensional binary vector space {0, 1}n. In practice, Rocchio's algorithm often uses a fixed query updating factor and a fixed classification threshold. When this is the case, we strengthen the work in [4] in this paper and prove that Rocchio's algorithm makes Ωkn---k mistakes in searching for the same collection of documents over the binary vector space {0, 1}n. A quadratic lower bound is obtained when k is proportional to n. An Okn---k2 upper bound is also obtained.
140623
Optimal Algorithms for Finding User Access Sessions from Very Large Web Logs
Although efficient identification of user access sessions from very large web logs is an unavoidable data preparation task for the success of higher level web log mining, little attention has been paid to algorithmic study of this problem. In this paper we consider two types of user access sessions, interval sessions and gap sessions. We design two efficient algorithms for finding respectively those two types of sessions with the help of some proposed structures. We present theoretical analysis of the algorithms and prove that both algorithms have optimal time complexity and certain error-tolerant properties as well. We conduct empirical performance analysis of the algorithms with web logs ranging from 100 megabytes to 500 megabytes. The empirical analysis shows that the algorithms just take several seconds more than the baseline time, i.e., the time needed for reading the web log once sequentially from disk to RAM, testing whether each user access record is valid or not, and writing each valid user access record back to disk. The empirical analysis also shows that our algorithms are substantially faster than the sorting based session finding algorithms. Finally, optimal algorithms for finding user access sessions from distributed web logs are also presented.
28060413
The Architecture of Yarrow: A Real-Time Intelligent Meta-Search Engine
44651708
Personalized Web Search with Clusters
7399730
An Almost Linear Time 2.8334-Approximation Algorithm for the Disc Covering Problem
The disc covering problem asks to cover a set of points on the plane with a minimum number of fix-sized discs. We develop an O(n(logn)2(loglogn)2) deterministic time 2.8334-approximation algorithm for this problem. Previous approximation algorithms [7,3,6], when used to achieve the same approximation ratio for the disc covering problem, will have much higher time complexity than our algorithms.
5722664
A Visualization System using Data Mining Techniques for Identifying Information Sources
The Visual Analysis System (VAS) was developed to couple emerging successes in data mining with information visualization techniques in order to create a richly interactive environment for information retrieval from the World Wide Web. VAS's retrieval strategy operates by first using a conventional search engine to form a core set,of retrieved documents. This core set is expanded by crawling from links in these documents to form a much larger set. The link topology of the larger set is then examined using data mining techniques to identify pages likely to be both relevant to the query and reliable as sources of information. Information visualization techniques are used to display the filtered set in a form amenable to the use of perceptual processes to filter and guide further user directed search. (Contains 16 references.) (MES) Reproductions supplied by EDRS are the best that can be made from the original document. N tr) 00 7171A Visualization System using Data Mining Techniques for Identifying Information Sources Richard H. Fowler, Tarkan Karadayi, Zhixiang Chen, Xiannong Meng, Wendy A. Lawrence Fowler Department of Computer Science University of Texas Pan American United States {fowler, tkaradayi, chen, meng, wfowler @panam.edu} PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY
19262472
Relevance Feedback and Learning in Content-Based Image Search
A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.
16272457
SimFusion: measuring similarity using unified relationship matrix
In this paper we use a Unified Relationship Matrix (URM) to represent a set of heterogeneous data objects (e.g., web pages, queries) and their interrelationships (e.g., hyperlinks, user click-through sequences). We claim that iterative computations over the URM can help overcome the data sparseness problem and detect latent relationships among heterogeneous data objects, thus, can improve the quality of information applications that require com- bination of information from heterogeneous sources. To support our claim, we present a unified similarity-calculating algorithm, SimFusion. By iteratively computing over the URM, SimFusion can effectively integrate relationships from heterogeneous sources when measuring the similarity of two data objects. Experiments based on a web search engine query log and a web page collection demonstrate that SimFusion can improve similarity measurement of web objects over both traditional content based algorithms and the cutting edge SimRank algorithm.
9623784
Multi-type Features Based Web Document Clustering
Clustering has been demonstrated as a feasible way to explore the contents of document collection and organize search engine results. For this task, many features of Web page, such as content, anchor text, URL, hyperlink etc, can be exploited and different results can be obtained. We expect to provide a unified and even better result for end users. Some work have studied how to use several types of features together to perform clustering. Most of them focus on ensemble method or combination of similarity. In this paper, we propose a novel algorithm: Multi-type Features based Reinforcement Clustering (MFRC). This algorithm does not use a unique combine score for all feature spaces, but uses the intermediate clustering result in one feature space as additional information to gradually enhance clustering in other spaces. Finally a consensus can be achieved by such mutual reinforcement. And the experimental results show that MFRC also provides some performance improvement.
1407636
A New Statistical Approach to Personal Name Extraction
We propose a new statistical approach to extracting personal names from a corpus. One of the key points of our approach is that it can both automatically learn the characteristics of personal names from a large training corpus and make good use of human empirical knowledge (e.g., Context Free Grammar). Furthermore, our approach also assigns confidence measures to the extracted personal names, compared with traditional simple true/false determination. Another main contribution of this work is that we have applied the personal name extraction technology into a real application, which is a Chinese inputting system and have achieved an approximately 7% error rate reduction for all characters and 30% error rate reduction for personal names.
7983044
Algorithm-based checkpoint-free fault tolerance for parallel matrix computations on volatile resources
As the size of today's high performance computers increases from hundreds, to thousands, and even tens of thousands of processors, node failures in these computers are becoming frequent events. Although checkpoint/rollback-recovery is the typical technique to tolerate such failures, it often introduces a considerable overhead. Algorithm-based fault tolerance is a very cost-effective method to incorporate fault tolerance into matrix computations. However, previous algorithm-based fault tolerance methods for matrix computations are often derived using algorithms that are seldomly used in the practice of today's high performance matrix computations and have mostly focused on platforms where failed processors produce incorrect calculations. To fill this gap, this paper extends the existing algorithm-based fault tolerance to the volatile computing platform where the failed processor stops working and applies it to scalable high performance matrix computations with two dimensional block cyclic data distribution. We show the practicality of this technique by applying it to the ScaLAPACK/PBLAS matrix-matrix multiplication kernel. Experimental results demonstrate that the proposed approach is able to survive process failures with a very low performance overhead