10179927
Automatic web page classification in a dynamic and hierarchical way
Automatic classification of web pages is an effective way to deal with the difficulty of retrieving information from the Internet. Although there are many automatic classification algorithms and systems that have been proposed, most of them ignore the conflict between the fixed number of categories and the growing number of web pages going into the system. They also require searching through all existing categories to make any classification. We propose a dynamic and hierarchical classification system that is capable of adding new categories as required, organizing the web pages into a tree structure, and classifying web pages by searching through only one path of the tree structure. Our test results show that our proposed single-path search technique reduces the search complexity and increases the accuracy by 6% comparing to related algorithms. Our dynamic-category expansion technique also achieves satisfying results on adding new categories into our system as required.
6632676
Applying Semantic Links for Classifying Web Pages
Automatic hypertext classification is an essential technique for organizing vast amount of Internet Web pages or HTML documents. One the of problems in classifying Web pages is that Web pages are usually short and contain insufficient text to clearly identify its category. Text classification mechanisms, by analyzing only the contents of the document itself, are relatively ineffective in classifying short Web pages. This paper proposes a new hypertext classification mechanism to address the problem by analyzing not only the Web page itself but also its linked Web pages referred by the URLs contained within the page. The URLs are treated as semantic links. The hypothesis is that the linked Web pages contain related information to help identifying the category of the Web page. Experimental results show that the proposed approach could increase the accuracy by 35% over the approach of analyzing only the Web page itself.
6302139
Speeding up keyword search for search engines
In the current information age, the dominant method for information search is by providing few keywords to a search engine. Keyword search is currently one of the most important operations in search engines and numerous other applications. In this paper we propose a new text indexing technique for improving the performance of keyword search. Our proposed technique not only speeds up searching operations but also the operations for inserting and for deleting keywords, which are particularly important for the ever increasing and dynamic changing databases such as that for search engines. We propose to partition all
1825969
Agent space architecture for search engines
The future of computing is moving from individual processing units to communities of self organizing agents. In This work we propose an agent and network based architecture for parallel and distributed computing called agent space architecture. Our architecture builds upon the notions of agent and object space and utilizes multicast networks. The building blocks for our proposed architecture consist of an active processing unit called agent, a shared place for communication called space, and a communication medium called multicast network. One unique feature of our architecture is that we extend the concept of object space to become an active space. Our active space functions as a rendezvous, a repository, a cache, a responder, a notifier, and a manager of its own resources. The organization of our architecture is as general as network topology. Any number of agents, spaces, or networks can be added to achieve high performance. It is as scalable as Ethernet and adding agents or spaces is as easy as plug and play. High availability and fault tolerance is achieved through multiple agents, spaces, and networks. All these features are particularly beneficial for challenging applications such as search engine, which is used as a test case to implement and to test our proposed architecture.
6698078
Automatically Discovering the Number of Clusters in Web Page Datasets
Clustering is well suited for Web mining by automatically organizing Web pages into categories each of which contains Web pages having similar contents. However, one problem in clustering is the lack of general methods to automatically determine the number of categories or clusters. For the Web domain in particular, currently there is no such method suitable for Web page clustering. In an attempt to address this problem, we discover a constant factor that characterizes the Web domain, based on which we propose a new method for automatically determining the number of clusters in Web page datasets. We discover that the measure of average inter-cluster similarity reaches a constant of 1.7 when all our experiments produced the best results for clustering Web pages. We determines the number of clusters by using the constant as the stopping factor in our clustering process by arranging individual Web pages into clusters and then arranging the clusters into larger clusters and so on until the average inter-cluster similarity approaches the constant. Having the new method described in this paper together with our new Bidirectional Hierarchical Clustering algorithm reported elsewhere, we have developed a clustering system suitable for mining the Web.
17009479
Bidirectional hierarchical clustering for Web mining
We propose a new bidirectional hierarchical clustering system for addressing challenges of Web mining. The key feature of the approach is that it aims to maximize the intra-cluster similarity in the bottom-up cluster-merging phase and it ensures to minimize the inter-cluster similarity in the top-down refinement phase. This two-pass approach achieves better clustering than existing one-pass approaches. We also propose a new cluster-merging criterion for allowing more than two clusters to be merged in each step and a new measure of similarity for taking into consideration not only the inter-connectivity between clusters but also the internal connectivity within the clusters. These result in reducing the average complexity for creating the final hierarchical structure of clusters from O(n/sup 2/) to O(n). The hierarchical structure represents a semantic structure between concepts of clusters and is directly applicable to the future of semantic net.
3203090
What are real DTDs like?
DTDs have proved important in a variety of areas: transformations between XML and databases, XML storage, XML publishing, consistency analysis of XML specifications, typechecking, and optimization of XML queries. Much of this work depends on certain assumptions about DTDs, e.g., the absence of recursion and non-determinism. With this comes the need to justify these assumptions against DTDs in the real world. This paper surveys a number of DTDs collected from the Web, and provides statistics with respect to a variety of criteria commonly discussed in XML resesarch.
13929475
Effects of Web retail service quality and product categories on consumer behavior: a research model and empirical exploration
This paper explores how Web retail service quality at the point of purchase influences consumers' perceptions of value and willingness to buy. In particular, this study highlights how Web retail service quality has different effects on perceived product quality, value, and willingness to buy according to product categories. We provide a research model on the basis of service quality, product categories on the Web, and marketing theories for consumers' purchase behavior. Consumer perceptions of Web retail service quality are collected from a sample of shoppers who actively search for products over the Internet. The results imply that functional Web retail service quality has a direct effect on willingness to buy and technical Web retail service quality influences consumer perceptions of product quality and value. Furthermore, product categories moderate the relationships among Web retail service quality, perceived product quality, perceived value, and willingness to buy. Implications for Internet marketing and future research directions are discussed.
6530554
Adaptive Index Utilization in Memory-Resident Structural Joins
We consider adaptive index utilization as a fine-grained problem in autonomic databases in which an existing index is dynamically determined to be used or not in query processing. As a special case, we study this problem for structural joins, the core operator in XML query processing, in the main memory. We find that index utilization is beneficial for structural joins only under certain join selectivity and distribution of matching elements. Therefore, we propose adaptive algorithms to decide whether to use an index probe or a data scan for each step of matching during the processing of a structural join operator. Our adaptive algorithms are based on the history, the look-ahead information, or both. We have developed a cost model to facilitate this adaptation and have conducted experiments with both synthetic and real-world data sets. Our results show that adaptively utilizing indexes in a structural join improves the performance by taking advantage of both sequential scans and index probes.
2642886
Cache-Conscious Automata for XML Filtering
Hardware cache behavior is an important factor in the performance of memory-resident, data-intensive systems such as XML filtering engines. A key data structure in several recent XML filters is the automaton, which is used to represent the long-running XML queries in the main memory. In this paper, we study the cache performance of automaton-based XML filtering through analytical modeling and system measurement. Furthermore, we propose a cache-conscious automaton organization technique, called the hot buffer, to improve the locality of automaton state transitions. Our results show that 1) our cache performance model for XML filtering automata is highly accurate and 2) the hot buffer improves the cache performance as well as the overall performance of automaton-based XML filtering
16011778
A Uniform System for Publishing and Maintaining XML Data
XML has become the prime standard for data exchange on the Web. To exchange data residing in Databases, one needs to publish it in XML. Data publishing is often done with a predefined “schema.” The chapter proposes a new approach for schema-directed publishing of relational data in XML, based on the novel notion of attribute transformation grammars (ATGs). ATGs provide guidance on how to define views conforming to target schemas (DTDs), and automatically ensure schema conformance. The chapter also develops an incremental algorithm for maintaining XML views produced by ATGs, based on new incremental computation techniques that capitalize on the hierarchical structure of XML data and unique features of the ATGs. Taking real-life data from Gene Ontology (GO), this chapter also demonstrates how this system can efficiently publish the GO data in XML with respect to a predefined recursive DTD, and how it incrementally updates the target XML data in response to changes to the underlying GO database.