2595430
A new approach to intranet search based on information extraction
This paper is concerned with 'intranet search'. By intranet search, we mean searching for information on an intranet within an organization. We have found that search needs on an intranet can be categorized into types, through an analysis of survey results and an analysis of search log data. The types include searching for definitions, persons, experts, and homepages. Traditional information retrieval only focuses on search of relevant documents, but not on search of special types of information. We propose a new approach to intranet search in which we search for information in each of the special types, in addition to the traditional relevance search. Information extraction technologies can play key roles in such kind of 'search by type' approach, because we must first extract from the documents the necessary information in each type. We have developed an intranet search system called 'Information Desk'. In the system, we try to address the most important types of search first - finding term definitions, homepages of groups or topics, employees' personal information and experts on topics. For each type of search, we use information extraction technologies to extract, fuse, and summarize information in advance. The system is in operation on the intranet of Microsoft and receives accesses from about 500 employees per month. Feedbacks from users and system logs show that users consider the approach useful and the system can really help people to find information. This paper describes the architecture, features, component technologies, and evaluation results of the system.
14292213
Title extraction from bodies of HTML documents and its application to web page retrieval
This paper is concerned with automatic extraction of titles from the bodies of HTML documents. Titles of HTML documents should be correctly defined in the title fields; however, in reality HTML titles are often bogus. It is desirable to conduct automatic extraction of titles from the bodies of HTML documents. This is an issue which does not seem to have been investigated previously. In this paper, we take a supervised machine learning approach to address the problem. We propose a specification on HTML titles. We utilize format information such as font size, position, and font weight as features in title extraction. Our method significantly outperforms the baseline method of using the lines in largest font size as title (20.9%-32.6% improvement in F1 score). As application, we consider web page retrieval. We use the TREC Web Track data for evaluation. We propose a new method for HTML documents retrieval using extracted titles. Experimental results indicate that the use of both extracted titles and title fields is almost always better than the use of title fields alone; the use of extracted titles is particularly helpful in the task of named page finding (23.1% -29.0% improvements).
41726126
An Approach to Acquire Semantic Relationships Between Words from Web Document
In this paper, we focus on the semantic relationships acquisition from Chinese web documents motivated by the large requirement of web question answering system in e-Learning. With our scheme, we dwindle in numbers of text to be analyzed and obtain initial sentence-level text in pre-process phase. Then linguistic rules, which are broken down into unambiguous and ambiguous, designed for Chinese phrases are applied to these sentence-level text to extract the synonymy relationship, hyponymy relationship, hypernymy relationship and parataxis relationship. Lastly, candidates are refined using two heuristics. Compared to other previous works, we apply not only strict unambiguous linguistic rules but also loose ambiguous linguistic rules to extract relationships and proposed efficient approach to refine the outputs of these rules. Experiments show that this method can acquire semantic relationships efficiently and effectively.
27295065
Automatic extraction of titles from general documents using machine learning
In this paper, we propose a machine learning approach to title extraction from general documents. By general documents, we mean documents that can belong to any one of a number of specific genres, including presentations, book chapters, technical papers, brochures, reports, and letters. Previously, methods have been proposed mainly for title extraction from research papers. It has not been clear whether it could be possible to conduct automatic title extraction from general documents. As a case study, we consider extraction from Office including Word and PowerPoint. In our approach, we annotate titles in sample documents (for Word and PowerPoint, respectively) and take them as training data, train machine learning models, and perform title extraction using the trained models. Our method is unique in that we mainly utilize formatting information such as font size as features in the models. It turns out that the use of formatting information can lead to quite accurate extraction from general documents. Precision and recall for title extraction from Word are 0.810 and 0.837, respectively, and precision and recall for title extraction from PowerPoint are 0.875 and 0.895, respectively in an experiment on intranet data. Other important new findings in this work include that we can train models in one domain and apply them to other domains, and more surprisingly we can even train models in one language and apply them to other languages. Moreover, we can significantly improve search ranking results in document retrieval by using the extracted titles.
28386762
Taxonomy Building and Machine Learning Based Automatic Classification for Knowledge-Oriented Chinese Questions
In this paper, we propose a taxonomy for knowledge-oriented question, and study the machine learning based classification for knowledge-oriented Chinese questions. By knowledge-oriented questions, we mean questions carrying information or knowledge about something, which cannot be well described by previous taxonomies. We build the taxonomy after the study of previous work and analysis of 6776 Chinese knowledge-oriented questions collected from different realistic sources. Then we investigate the new task of knowledge-oriented Chinese questions classification based on this taxonomy. In our approach, the popular SVM learning method is employed as classification algorithm. We explore different features and their combinations and different kernel functions for the classification, and use different performance metrics for evaluation. The results demonstrate that the proposed approach is desirable and robust. Thorough error analysis is also conduced.
4391987
Automatic extraction of titles from general documents using machine learning
We propose a machine learning approach to title extraction from general documents. By general documents, we mean documents that can belong to any one of a number of specific genres, including presentations, book chapters, technical papers, brochures, reports, and letters. Previously, methods have been proposed mainly for title extraction from research papers. It has not been clear whether it could be possible to conduct automatic title extraction from general documents. As a case study, we consider extraction from Office including Word and PowerPoint. In our approach, we annotate titles in sample documents (for Word and PowerPoint respectively) and take them as training data, train machine learning models, and perform title extraction using the trained models. Our method is unique in that we mainly utilize formatting information such as font size as features in the models. It turns out that the use of formatting information can lead to quite accurate extraction from general documents. Precision and recall for title extraction from Word is 0.810 and 0.837 respectively, and precision and recall for title extraction from PowerPoint is 0.875 and 0.895 respectively in an experiment on intranet data. Other important new findings in this work include that we can train models in one domain and apply them to another domain, and more surprisingly we can even train models in one language and apply them to another language. Moreover, we can significantly improve search ranking results in do document retrieval by using the extracted titles
14585493
Learning a locality preserving subspace for visual recognition
We have demonstrated that the face recognition performance can be improved significantly in low dimensional linear subspaces. Conventionally, principal component analysis (PCA) and linear discriminant analysis (LDA) are considered effective in deriving such a face subspace. However, both of them effectively see only the Euclidean structure of face space. We propose a new approach to mapping face images into a subspace obtained by locality preserving projections (LPP) for face analysis. We call this Laplacianface approach. Different from PCA and LDA, LPP finds an embedding that preserves local information, and obtains a face space that best detects the essential manifold structure. In this way, the unwanted variations resulting from changes in lighting, facial expression, and pose may be eliminated or reduced. We compare the proposed Laplacianface approach with eigenface and fisherface methods on three test datasets. Experimental results show that the proposed Laplacianface approach provides a better representation and achieves lower error rates in face recognition.
14708530
An Evaluation of Sparse Direct Symmetric Solvers: An Introduction and Preliminary Findings
In recent years a number of solvers for the direct solution of large sparse, symmetric linear systems of equations have been developed. These include solvers that are designed for the solution of positive-definite systems as well as those that are principally intended for solving indefinite problems. The available choice can make it difficult for users to know which solver is the most appropriate for their applications. In this study, we use performance profiles as a tool for evaluating and comparing the performance of serial sparse direct solvers on an extensive set of symmetric test problems taken from a range of practical applications.
9572409
High-Throughput Multicast Routing Metrics in Wireless Mesh Networks
The stationary nature of nodes in a mesh network has shifted the main design goal of routing protocols from maintaining connectivity between source and destination nodes to finding high-throughput paths between them. In recent years, numerous link-quality-based routing metrics have been proposed for choosing high-throughput paths for unicast protocols. In this paper we study routing metrics for high-throughput tree or mesh construction in multicast protocols. We show that there is a fundamental difference between unicast and multicast routing in how data packets are transmitted at the link layer, and accordingly there is a difference in how the routing metrics for each of these primitives are designed. We adapt certain routing metrics for unicast for high-throughput multicast routing and propose news ones not previously used for high-throughput. We then study the performance improvement achieved by using different link-quality-based routing metrics via extensive simulation and experiments on a mesh network testbed, using ODMRP as a representative multicast protocol. Our testbed experiment results show that ODMRP enhanced with linkquality routing metrics can achieve up to 17.5% throughput improvement as compared to the original ODMRP.
18579380
On the Equivalence of Forward and Reverse Query Caching in Peer-to-Peer Overlay Networks
Peer-to-peer systems such as Gnutella and Kazaa are used by millions of people for sharing music and many other files over the Internet, and they account for a significant portion of the Internet traffic. The traffic in a peer-to-peer overlay network is different from that in WWW in that each peer is both a client and a server. This suggests that one can deploy a forward cache at the Internet gateway of a network to reduce the amount of queries going outside, or a revere cache at the gateway to reduce the amount of queries going inside, which in turn reduces the queries that are forwarded outside. In this paper, we study the effectiveness of forward and reverse caching at the gateway via analysis and experimental measurement. Our study shows that forward caching and reverse caching at the gateway are equally effective at reducing query and query reply traffic across the gateway.
2476194
Assisted Peer-to-Peer Search with Partial Indexing
In the past few years, peer-to-peer (P2P) networks have become a promising paradigm for building a wide variety of distributed systems and applications. The most popular P2P application till today is file sharing, e.g., Gnutella, Kazza, etc. These systems are usually referred to as unstructured, and search in unstructured P2P networks usually involves flooding or random walking. On the other hand, in structured P2P networks (DHTs), search is usually performed by looking up a distributed inverted index. The efficiency of the search mechanism is the key to the scalability of a P2P content sharing system. So far, neither unstructured nor structured P2P networks alone can solve the search problem in a satisfactory way. In this paper, we propose to combine the strengths of both unstructured and structured P2P networks to achieve more efficient search. Specifically, we propose to enhance search in unstructured P2P overlay networks by building a partial index of shared data using a structured P2P network. The index maintains two types of information: the top interests of peers and globally unpopular data, both characterized by data properties. The proposed search protocol, assisted search with partial indexing, makes use of the index to improve search in three ways: first, the index assists peers to find other peers with similar interests and the unstructured search overlay is formed to reflect peer interests. Second, the index also provides search hints for those data difficult to locate by exploring peer interest locality, and these hints can be used for second-chance search. Third, the index helps to locate unpopular data items. Experiments based on a P2P file sharing trace show that the assisted search with a lightweight partial indexing service can significantly improve the success rate in locating data than Gnutella and a hit-rate-based protocol in unstructured P2P systems, while incurring low search latency and overheads.