11059202
Mining web navigations for intelligence
The Internet is one of the fastest growing areas of intelligence gathering. We present a statistical approach, called principal clusters analysis, for analyzing millions of user navigations on the Web. This technique identifies prominent navigation clusters on different topics. Furthermore, it can determine information items that are useful starting points to explore a topic, as well as key documents to explore the topic in greater detail. Trends can be detected by observing navigation prominence over time. We apply this technique on a large popular website. The results show promise in web intelligence mining.
14252554
Discovering and Reconciling Semantic Conflicts: A Data Mining Perspective
Current approaches to semantic interoperability require human intervention in detecting potential conflicts and in defining how those conflicts may be resolved. This is a major impedance to achieving “logical connectivity”, especially when the dumber of disparate sources is large. In this paper, we demonstrate that the detection and reconciliation of semantic conflicts can be automated using tools and techniques developed by the data mining community. We describe a process for discovering such rules that represent the relationships among semanticaly related attributes and illustrate the effectiveness of our approach with examples.
15496678
Discovery of context-specific ranking functions for effective information retrieval using genetic programming
The Internet and corporate intranets have brought a lot of information. People usually resort to search engines to find required information. However, these systems tend to use only one fixed ranking strategy regardless of the contexts. This poses serious performance problems when characteristics of different users, queries, and text collections are taken into account. We argue that the ranking strategy should be context specific and we propose a , new systematic method that can automatically generate ranking strategies for different contexts based on genetic programming (GP). The new method was tested on TREC data and the results are very promising.
16272457
SimFusion: measuring similarity using unified relationship matrix
In this paper we use a Unified Relationship Matrix (URM) to represent a set of heterogeneous data objects (e.g., web pages, queries) and their interrelationships (e.g., hyperlinks, user click-through sequences). We claim that iterative computations over the URM can help overcome the data sparseness problem and detect latent relationships among heterogeneous data objects, thus, can improve the quality of information applications that require com- bination of information from heterogeneous sources. To support our claim, we present a unified similarity-calculating algorithm, SimFusion. By iteratively computing over the URM, SimFusion can effectively integrate relationships from heterogeneous sources when measuring the similarity of two data objects. Experiments based on a web search engine query log and a web page collection demonstrate that SimFusion can improve similarity measurement of web objects over both traditional content based algorithms and the cutting edge SimRank algorithm.
15362027
ETANA-DL: managing complex information applications - an archaeology digital library
Summary form only given. Archaeological research results in the generation of large quantities of heterogeneous information managed by different projects using custom information systems. We demonstrate a prototype digital library (DL) for integrating and managing archaeological data and providing services useful to various user communities. ETANA-DL is a model-based, componentized, extensible, archaeological DL that manages complex information sources using the client-server paradigm of the open archives initiative protocol for metadata harvesting (OAI-PMH). We present a prototype of ETANA-DL that builds on the 5S meta-model for developing for archaeological systems. We demonstrate our approach, used to integrate and handle highly heterogeneous data and the various services that ETANA-DL currently supports - searching, browsing, personal collections, workflows, discussions, annotations, etc. We are working on the structural aspects of the meta-model, and have used the results to develop advanced cross-collection searching and browsing services.
4216648
Optimizing web search using web click-through data
The performance of web search engines may often deteriorate due to the diversity and noisy information contained within web pages. User click-through data can be used to introduce more accurate description (metadata) for web pages, and to improve the search performance. However, noise and incompleteness, sparseness, and the volatility of web pages and queries are three major challenges for research work on user click-through log mining. In this paper, we propose a novel iterative reinforced algorithm to utilize the user click-through data to improve search performance. The algorithm fully explores the interrelations between queries and web pages, and effectively finds "virtual queries" for web pages and overcomes the challenges discussed above. Experiment results on a large set of MSN click-through log data show a significant improvement on search performance over the naive query log mining algorithm as well as the baseline search engine.
11711193
Video-Based Face Recognition Using Bayesian Inference Model
There has been a flurry of works on video sequence-based face recognition in recent years. One of the hard problems in this area is how to effectively combine the facial configuration and temporal dynamics for the recognition task. The proposed method treats this problem in two steps. We first construct several view specific appearance sub-manifolds learned from the training video frames using locally linear embedding (LLE). A general Bayesian inference model is then fit on the recognition task, transforming the complicated maximum likelihood estimation to some elegant distance measures in the learned sub-manifolds. Experimental results on a middle-scale video database demonstrate the effectiveness and flexibility of our proposed method.
38519945
Propagating XML constraints to relations
We present a technique for refining the design of relational storage for XML data based on XML key propagation. Three algorithms are presented: one checks whether a given functional dependency is propagated from XML keys via a predefined view; the others compute a minimum cover for all functional dependencies on a universal relation given XML keys. Experimental results show that these algorithms are efficient in practice. We also investigate the complexity of propagating other XML constraints to relations, and the effect of increasing the power of the transformation language. Computing XML key propagation is a first step toward establishing a connection between XML data and its relational representation at the semantic level.
2869108
Propagating XML constraints to relations
We present a technique for refining the design of relational storage for XML data. The technique is based on XML key propagation: given a set of keys on XML data and a mapping (transformation) from the XML data to relations, what functional dependencies must hold on the relations produced by the mapping? With the functional dependencies one can then convert the relational design into, e.g. 3NF, BCNF, and thus develop efficient relational storage for XML data. We provide several algorithms for computing XML key propagation. One algorithm is to check whether a functional dependency is propagated from a set of XML keys via a predefined mapping; this allows one to determine whether or not the relational design is in a normal form. The others are to compute a minimum cover for all functional dependencies that are propagated from a set of XML keys and hold on a universal relation; these provide guidance for how to design a relational schema for storing XML data. These algorithms show that XML key propagation and its associated minimum cover can be computed in polynomial time. Our experimental results verify that these algorithms are efficient in practice. We also investigate the complexity of propagating other XML constraints to relations. The ability to compute XML key propagation is a first step toward establishing a connection between XML data and its relational representation at the semantic level.
6232084
Query Optimization for Semistructured Data Using Path Constraints in a Deterministic Data Model
Path constraints have been studied for semistructured data modeled as a rooted edge-labeled directed graph [4, 11-13]. In this model, the implication problems associated with many natural path constraints are undecidable [11, 13]. A variant of the graph model, called the deterministic data model, was recently proposed in [10]. In this model, data is represented as a graph with deterministic edge relations, i.e., the edges emanating from any node in the graph have distinct labels. This model is more appropriate for representing, e.g., ACeDB [27] databases and Web sites. This paper investigates path constraints for the deterministic data model. It demonstrates the application of path constraints to, among others, query optimization. Three classes of path constraints are considered: the language Pc introduced in [11], an extension of Pc, denoted by Pcw, by including wildcards in path expressions, and a generalization of Pcw, denoted by P*c, by representing paths as regular expressions. The implication problems for these constraint languages are studied in the context of the deterministic data model. It is shown that in contrast to the undecidability result of [11], the implication and finite implication problems for Pc are decidable in cubic-time and are finitely axiomatizable. Moreover, the implication problems are decidable for Pcw. However, the implication problems for P*c are undecidable.
10449065
Beyond XML Query Languages
A query language is essential, if XML is to serve effectively as an exchange medium for large data sets. The design of query languages for XML is in its infancy, and the choice of a standard may be governed more by user acceptance than by any understanding of underlying principles. One would hope that expressive power, performance, and compatibility with other languages will be considered in choosing among alternatives, but it is likely that several contenders will co-exist for some time. It is worth observing that, during the 20-year development of relational query languages, several competing languages were developed; and even today there are several relational query language standards. In spite of this, a great deal of technology was developed that was independent of the surface syntax of a query language. This included technology "below" the language such as efficient execution models and work "above" the level of language such as techniques for view definition and maintenance, triggers, etc. At Penn we are working on some of these language-independent issues. We include a summary of them here. They include execution and data models to support XML and semistructured query languages; the use of schemas and constraints in optimizing XML query languages; and tools for extracting data form existing sources and presenting it as XML. Comments Postprint version. Published in Query Language Workshop, November 1998. Publisher URL: http://www.w3.org/TandS/QL/QL98/pp/penn.html Author(s) Peter Buneman, Alin Deutsch, Wenfei Fan, Hartmut Liefke, Arnaud Sahuguet, and Wang-Chiew Tan This conference paper is available at ScholarlyCommons: http://repository.upenn.edu/db_research/14 Beyond XML Query Languages Peter Buneman Alin Deutsch Wenfei Fan Hartmut Liefke Arnaud Sahuguet Wang-Chiew Tan University of Pennsylvania fpeter,adeutsch,wfan,liefke,sahuguet,wctang@saul.cis.upenn.edu