5754331
Web-page summarization using clickthrough data
Most previous Web-page summarization methods treat a Web page as plain text. However, such methods fail to uncover the full knowledge associated with a Web page needed in building a high-quality summary, because many of these methods do not consider the hidden relationships in the Web. Uncovering the hidden knowledge is important in building good Web-page summarizers. In this paper, we extract the extra knowledge from the clickthrough data of a Web search engine to improve Web-page summarization. Wefirst analyze the feasibility in utilizing the clickthrough data to enhance Web-page summarization and then propose two adapted summarization methods that take advantage of the relationships discovered from the clickthrough data. For those pages that are not covered by the clickthrough data, we design a thematic lexicon approach to generate implicit knowledge for them. Our methods are evaluated on a dataset consisting of manually annotated pages as well as a large dataset that is crawled from the Open Directory Project website. The experimental results indicate that significant improvements can be achieved through our proposed summarizer as compared to the summarizers that do not use the clickthrough data.
432736
ReCoM: reinforcement clustering of multi-type interrelated data objects
Most existing clustering algorithms cluster highly related data objects such as Web pages and Web users separately. The interrelation among different types of data objects is either not considered, or represented by a static feature space and treated in the same ways as other attributes of the objects. In this paper, we propose a novel clustering approach for clustering multi-type interrelated data objects, ReCoM (Reinforcement Clustering of Multi-type Interrelated data objects). Under this approach, relationships among data objects are used to improve the cluster quality of interrelated data objects through an iterative reinforcement clustering process. At the same time, the link structure derived from relationships of the interrelated data objects is used to differentiate the importance of objects and the learned importance is also used in the clustering process to further improve the clustering results. Experimental results show that the proposed approach not only effectively overcomes the problem of data sparseness caused by the high dimensional relationship space but also significantly improves the clustering accuracy.
207643
User Modeling for Efficient Use of Multimedia Files
It is very common that a user likes to collect many multimedia files of their interests from the web or other sources for his/her daily use, such as in emails, presentations, and technical documents. This paper presents algorithms to learn user models, in particular, user intention models and preference models from the usage of these files. Such usages include downloading, inserting, and sending multimedia files. A user intention model predicts when the user may want to involve some multimedia objects in his currently working environment (e.g., an email) and provides more convenient and accurate help to the user. A user preference model describes the types and classes of the user's favorite multimedia files and helps an offline crawler to autonomously collect more useful multimedia files for the user. The algorithms have been implemented in our media agents system and shown their effectiveness in user modeling.
6802371
Improving web search results using affinity graph
In this paper, we propose a novel ranking scheme named Affinity Ranking (AR) to re-rank search results by optimizing two metrics: (1) diversity -- which indicates the variance of topics in a group of documents; (2) information richness -- which measures the coverage of a single document to its topic. Both of the two metrics are calculated from a directed link graph named Affinity Graph (AG). AG models the structure of a group of documents based on the asymmetric content similarities between each pair of documents. Experimental results in Yahoo! Directory, ODP Data, and Newsgroup data demonstrate that our proposed ranking algorithm significantly improves the search performance. Specifically, the algorithm achieves 31% improvement in diversity and 12% improvement in information richness relatively within the top 10 search results.
15310025
A study of relevance propagation for web search
Different from traditional information retrieval, both content and structure are critical to the success of Web information retrieval. In recent years, many relevance propagation techniques have been proposed to propagate content information between web pages through web structure to improve the performance of web search. In this paper, we first propose a generic relevance propagation framework, and then provide a comparison study on the effectiveness and efficiency of various representative propagation models that can be derived from this generic framework. We come to many conclusions that are useful for selecting a propagation model in real-world search applications, including 1) sitemap-based propagation models outperform hyperlink-based models in sense of both effectiveness and efficiency, and 2) sitemap-based term propagation is easier to be integrated into real-world search engines because of its parallel offline implementation and acceptable complexity. Some other more detailed study results are also reported in the paper.
14828051
MRSSA: an iterative algorithm for similarity spreading over interrelated objects
We introduce the Multiple Relationship Similarity Spreading Algorithm (MRSSA) to enhance IR effectiveness. This method has similarity computed in an iterative "spreading" fashion for multiple object types, combining both inter- and intra-object relationships. We demonstrate the value of this approach in the context of the WWW, where the key objects are web pages and queries, Relationships considered are derived from hyperlinks (in- and out-links) and click-through logs.
7765043
WebSail: From On-line Learning to Web Search
In this paper we report our research on building WebSail, an intelligent web search engine that is able to perform real-time adaptive learning. WebSail learns from the user's relevance feedback, so that it is able to speed up its search process and to enhance its search performance. We design an efficient adaptive learning algorithm TW2 to search for web documents. WebSail employs TW2 together with an internal index database and a real-time meta-searcher to perform real-time adaptive learning to find desired documents with as little relevance feedback from the user as possible. The architecture and performance of WebSail are also discussed.
27478069
Linear and sublinear time algorithms for mining frequent traversal path patterns from very large Web logs
This paper aims for designing algorithms for the problem of mining frequent traversal path patterns from very large Web logs with best possible efficiency. We devise two algorithms for this problem with the help of fast construction of "shallow" generalized suffix trees over a very large alphabet. These two algorithms have respectively provable linear time and sublinear complexity, and their performance is analyzed in comparison with the two a priori-like algorithms in (Chen et al., 1998) and the well-known Ukkonen algorithm for online suffix tree construction (1995). It is shown that these two algorithms are substantially efficient than the two apriori-like algorithms and the Ukkonen algorithm. The linear time algorithm has optimal performance in theory, while the sublinear time algorithm has better empirical performance.
27655532
On the Planar Two-Watchtower Problem
In this paper we study the following planar two-watchtower problem: given a terrain (X-monotone chain) T with n vertices, locate two vertical segments (watchtowers) uv and u′v′ which have to be erected on T such that every point on T is visible to the top of either watchtowers (u or u′) and the maximum height of uv, u′v′ is minimized. We present an O(n4) time algorithm to solve the discrete version of this problem when v, v′ must be on some vertices of T. Under a mild condition on solving a special cubic equation with three bounded variables in O(f3) time we can also generalize the algorithm to solve the general problem in O(n4 +n3f3) time. Using parametric search, the discrete problem can be solvedin O(n3 log2 n) time and the general problem can be solvedin O(n4 log2 n) time.
11497116
Nowhere zero flows in line graphs
Abstract Cai an Corneil (Discrete Math. 102 (1992) 103–106), proved that if a graph has a cycle double cover, then its line graph also has a cycle double cover, and that the validity of the cycle double cover conjecture on line graphs would imply the truth of the conjecture in general. In this note we investigate the conditions under which a graph G has a nowhere zero k-flow would imply that L ( G ), the line graph of G, also has a nowhere zero k-flow. The validity of Tutte's flow conjectures on line graphs would also imply the truth of these conjectures in general.
140623
Optimal Algorithms for Finding User Access Sessions from Very Large Web Logs
Although efficient identification of user access sessions from very large web logs is an unavoidable data preparation task for the success of higher level web log mining, little attention has been paid to algorithmic study of this problem. In this paper we consider two types of user access sessions, interval sessions and gap sessions. We design two efficient algorithms for finding respectively those two types of sessions with the help of some proposed structures. We present theoretical analysis of the algorithms and prove that both algorithms have optimal time complexity and certain error-tolerant properties as well. We conduct empirical performance analysis of the algorithms with web logs ranging from 100 megabytes to 500 megabytes. The empirical analysis shows that the algorithms just take several seconds more than the baseline time, i.e., the time needed for reading the web log once sequentially from disk to RAM, testing whether each user access record is valid or not, and writing each valid user access record back to disk. The empirical analysis also shows that our algorithms are substantially faster than the sorting based session finding algorithms. Finally, optimal algorithms for finding user access sessions from distributed web logs are also presented.