2328843
Data-rich section extraction from HTML pages
We propose a novel algorithm, DSE (data-rich subtree extraction) to recognize and extract the data-rich section of an HTML page. We apply the DSE algorithm as a pre-processing "clean-up" step for two typical Web information retrieval problems: topic distillation and Web information extraction. Our experiments show that, for the test data sets used, the DSE algorithm can correctly identify data-rich sections of HTML pages with 100% accuracy. Therefore, it can effectively reduce the root set size for the topic distillation problem thereby improving the precision and accuracy of the IETS algorithm. Furthermore, when applied to the Web information extraction problem using the IEPAD algorithm, it can decrease the number of patterns discovered by this algorithm, thus shortening its time cost to generalize a wrapper for HTML pages.
11616193
Automatic Hierarchical Classification of Structured Deep Web Databases
We present a method that automatically classifies structured deep Web databases according to a pre-defined topic hierarchy. We assume that there are some manually classified databases, i.e., training databases, in every node of the topic hierarchy. Each training database is probed using queries constructed from the node titles of the topic hierarchy and the query result counts reported by the database are used to represent the content of the database. Hence, when adding a new database it can be probed by the same set of queries and classified to a node whose training databases are most similar to the new one. Specifically, a support vector machine classifier is trained on each internal node of the topic hierarchy with these training databases and the new database can be classified into the hierarchy top-down level by level. A feature extension method is proposed to create discriminant features. Experiments run on real structured Web databases collected from the Internet show that this classification method is quite accurate.
46369194
Mining Frequent Agreement Subtrees in Phylogenetic Databases
2286283
Query result ranking over e-commerce web databases
To deal with the problem of too many results returned from an E-commerce Web database in response to a user query, this paper proposes a novel approach to rank the query results. Based on the user query, we speculate how much the user cares about each attribute and assign a corresponding weight to it. Then, for each tuple in the query result, each attribute value is assigned a score according to its "desirableness" to the user. These attribute value scores are combined according to the attribute weights to get a final ranking score for each tuple. Tuples with the top ranking scores are presented to the user first. Our ranking method is domain independent and requires no user feedback. Experimental results demonstrate that this ranking method can effectively capture a user's preferences.
32133517
Information Extraction from the Structured Part of Office Documents
The contents of office documents can be divided into structured and unstructured parts. In this paper, we present a sample-based approach to analyzing a document to form its layout and conceptual structures, and then extracting information from the structured part of the office documents. We represent a document's layout structure as an ordered labeled tree structure using nested segmentation, and its conceptual structure as a set of attribute type pairs. The layout similarities between the document to be processed and samples are identified by employing an approximate tree matching method. The conceptual similarities are identified by analyzing document and sample contents, and by measuring the degree of conceptual closeness. Finally, the information is extracted by instantiating the attributes specified in the conceptual structure based on the result of document structure analysis.
236203
New Techniques for DNA Sequence Classification
DNA sequence classification is the activity of determining whether or not an unlabeled sequence S belongs to an existing class C. This paper proposes two new techniques for DNA sequence classification. The first technique works by comparing the unlabeled sequence S with a group of active motifs discovered from the elements of C and by distinction with elements outside of C. The second technique generates and matches gapped fingerprints of S with elements of C. Experimental results obtained by running these algorithms on long and well conserved Alu sequences demonstrate the good performance of the presented methods compared with FASTA. When applied to less conserved and relatively short functional sites such as splice-junctions, a variation of the second technique combining fingerprinting with consensus sequence analysis gives better results than the current classifiers employing text compression and machine learning algorithms.
5981810
An efficient ontology comparison tool for semantic Web applications
With the growing access to heterogeneous and independent data repositories, determining the semantic difference of two ontologies is critical in information retrieval, information integration and semantic web applications. In this paper, we propose an ontology comparison tool based on a novel senses refinement algorithm, which builds a senses set to accurately represent the semantics of the input ontology. The senses refinement algorithm automatically extracts senses from the electronic lexical database WordNet (locally installed or online), removes unnecessary senses based on the relationship among the entity classes of the ontology, and specifies relations and constraints of the concepts in the refined senses set. The senses refinement converts the measurement of ontology difference into simple set operations based on set theory, thus ensures the efficiency and accuracy of the ontology comparison. Our experimental studies show that the proposed senses refinement algorithm outperforms the naive senses set construction algorithm in terms of efficiency and accuracy.
94229
Translating unknown queries with web corpora for cross-language information retrieval
It is crucial for cross-language information retrieval (CLIR) systems to deal with the translation of unknown queries due to that real queries might be short. The purpose of this paper is to investigate the feasibility of exploiting the Web as the corpus source to translate unknown queries for CLIR. We propose an online translation approach to determine effective translations for unknown query terms via mining of bilingual search-result pages obtained from Web search engines. This approach can alleviate the problem of the lack of large bilingual corpora, translate many unknown query terms, provide flexible query specifications, and extract semantically-close translations to benefit CLIR tasks -- especially for cross-language Web search.
17168623
Traffic classification for application specific peering
Peer-to-Peer (P2P) applications [4], [6], [2], [3] have increased traffic significantly. We propose application specific peering that allows searches for a resource (using existing or modified P2P protocols) to be directed to a copy of resource on an ISP's network, using a novel technique that automatically classifies traffic (shown in Fig 1) as belonging to an ISP, its customers, peers, or their customers. We examined a large amount of traffic belonging to three popular P2P protocols and used a graph transformation to detect connected components. We partition traffic in each component into signaling and data sections based on request/response sizes of P2P protocols and extract high traffic volume (by bytes) entities by examining large connected components in the graph. We discuss the important components of our traffic classification system here. We analyzed router-level data from multiple Internet Gateway Routers (IGRs) across AT&T IP backbone using Cisco's NetFlow services. Note that statistics are from a subset of IGRs at some AT&T's peering links. They are not representative of overall traffic pattem on AT&T IP backbone. We examined DirectConnect[2], Gnutella[3], and FastTrack[4], extracting records that matched the default application ports (source or destina• tion, 411/412, 6346/6347, and 1214 (FastTrack) respectively), involving TCP traffic. Data was gathered over three separate months in 2001, each lasting 5-7 days. The number of netflow records varies from a low of 0.5M (DirectConnect) to a high
40322050
Complete on Average Boolean Satisfiability
Abstract We present in this paper a dynamic binary coding scheme α on CNF formulas ψ , and show that under a uniform distribution μ α on binary string α ( ψ ), SAT is complete on average, where μ α ( ψ ) is proportional to ∣ α ( ψ )∣ −2 2 −∣ α ( ψ )∣ . We then show that there is k 0 >2 such that for all k ⩾ k 0 , k SAT under μ α is complete on average.
12667585
A novel self-configuration mechanism for heterogeneous P2P networks
We propose a simple and efficient mechanism to self-configure semi-structured overlay networks for heterogeneous peer-to-peer systems with no global knowledge. The overlay network is constructed as a hierarchical graph based on peer power which is an aggregate measure of available resources. We use index based, random walk JOIN and LEAVE protocols to build the overlay network in which node degree and workload correspond to its peer power. We also propose an index based query scheme for peers to search documents in the network. Performance studies demonstrate that the proposed self-configuration protocol and search scheme are efficient and viable. The self-configured P2P network has significantly lower query cost than traditional Gnutella-like P2P systems. We further propose a caching assisted query mechanism where query results are cached by intermediate nodes along the search path, thereby exploiting the surplus storage power of the peers and further improving query efficiency. Caching assisted query permits a quicker response to subsequent queries for popular data without incurring the excessive overhead of globally replicated directory information. Our experimental study reveals that the caching assisted query provides an additional reduction of up to 40% in messages needed per query.