1436962
Predicting query times
We outline the need for search engines to provide user feedback on the expected time for a query, describe a scheme for learning a model of query time by observing sample queries, and discuss the results obtained for a set of actual user queries on a document collection using the MG search engine.
5326806
Modeling for Optimal Probability Prediction
We present a general modeling method for optimal probability prediction over future observations, in which model dimensionality is determined as a natural by-product. This new method yields several estimators, and we establish theoretically that they are optimal (either overall or under stated restrictions) when the number of free parameters is infinite. As a case study, we investigate the problem of fitting logistic models in finite-sample situations. Simulation results on both artificial and practical datasets are supportive.
5074474
Maximum likelihood computation based on the Fisher scoring and Gauss-Newton quadratic approximations
The Fisher scoring and Gauss-Newton methods are two known methods for maximum likelihood computation. This paper provides a generalization for each method in a unified manner so that they can be used for some difficult maximum likelihood computation, when, for example, there exist constraints on the parameters. A generalized method does not use directly the Newton-type iteration formulas of these methods, but, instead, uses the corresponding quadratic functions transformed from them. It proceeds by repeatedly approximating the log-likelihood function with the quadratic functions in the neighborhoods of the current iterates and optimizing each quadratic function within the parameter space. It is shown that each quadratic function has a weighted linear regression formulation, which can be conveniently solved. This generalization also extends the applicability of the Fisher scoring method to situations when the expected Fisher information matrices are unavailable in closed form. Fast computation can generally be anticipated, owing to their small rates of convergence and a rapid solution of each linear regression problem. While the generalized Gauss-Newton method may sometimes suffer for the so-called large residual problem, the generalized Fisher scoring method has performed consistently well in the numerical experiments we conducted.
570803
Provable dimension detection using principal component analysis
We present simple algorithms for detecting the dimension k of a smooth manifold M ⊂ Rd from a set P of point samples, provided that P satisfies a standard sampling condition as in previous results. The best running time so far is O(d2O(k7 log k)) worst-case by Giesen and Wagner after the adaptive neighborhood graph is constructed in O(d|P|2) worst-case time. Given the adaptive neighborhood graph, for any l ≥ 1, our algorithm outputs the true dimension with probability at least 1-2-l in O(2O(k)kd(k + l log d)) expected time. Our experimental results validate the effectiveness of our approach in computing the dimension. A further advantage is that both the algorithm and its analysis can be generalized to the noisy case, in which outliers and a small perturbation of the samples are allowed.
6114683
The evidential reasoning approach for MADA under both probabilistic and fuzzy uncertainties
Abstract Many multiple attribute decision analysis (MADA) problems are characterised by both quantitative and qualitative attributes with various types of uncertainties. Incompleteness (or ignorance) and vagueness (or fuzziness) are among the most common uncertainties in decision analysis. The evidential reasoning (ER) approach has been developed in the 1990s and in the recent years to support the solution of MADA problems with ignorance, a kind of probabilistic uncertainty. In this paper, the ER approach is further developed to deal with MADA problems with both probabilistic and fuzzy uncertainties. In this newly developed ER approach, precise data, ignorance and fuzziness are all modelled under the unified framework of a distributed fuzzy belief structure, leading to a fuzzy belief decision matrix. A utility-based grade match method is proposed to transform both numerical data and qualitative (fuzzy) assessment information of various formats into the fuzzy belief structure. A new fuzzy ER algorithm is developed to aggregate multiple attributes using the information contained in the fuzzy belief matrix, resulting in an aggregated fuzzy distributed assessment for each alternative. Different from the existing ER algorithm that is of a recursive nature, the new fuzzy ER algorithm provides an analytical means for combining all attributes without iteration, thus providing scope and flexibility for sensitivity analysis and optimisation. A numerical example is provided to illustrate the detailed implementation process of the new ER approach and its validity and wide applicability.
2488079
Web-Based Unsupervised Learning for Query Formulation in Question Answering
Converting questions to effective queries is crucial to open-domain question answering systems. In this paper, we present a web-based unsupervised learning approach for transforming a given natural-language question to an effective query. The method involves querying a search engine for Web passages that contain the answer to the question, extracting patterns that characterize fine-grained classification for answers, and linking these patterns with n-grams in answer passages. Independent evaluation on a set of questions shows that the proposed approach outperforms a naive keyword-based approach in terms of mean reciprocal rank and human effort.
18269009
Maximum Flows and Critical Vertices in AND/OR Graphs
We will study this problem and present an algorithm for finding the minimum-time-cost solution graph in an AND/OR graph. We will also study the following problems which often appear in industry when using AND/OR graphs to model manufacturing processes or to model problem solving processes: finding maximum (additive and nonadditive) flows and critical vertices in an AND/OR graph. Though there are well known polynomial time algorithms for the corresponding problems in the traditional graph theory, we will show that generally it is NP-hard to find a non-additive maximum flow in an AND/OR graph, and it is both NP-hard and coNP-hard to find a set of critical vertices in an AND/OR graph. We will also present a polynomial time algorithm for finding a maximum additive flow in an AND/OR graph, and discuss the relative complexity of these problems.
27423282
Visualization of DNA/RNA Structure using Temporal CGRs
We present a new DNA/RNA sequence visualization technique, Temporal Chaos Game Representation (TCGR), which is an extension of Frequency Chaos Game Representation (FCGR). Unlike earlier CGR variants, TCGR illustrates the distribution of nucleotide sub-patterns at different positions within a sequence, and it can be applied to sets of sequences as well as individual sequences. TCGRs are suitable for short or long sequences. Preliminary experiments examining patterns of nucleotide occurrences in micro-RNA data illustrate not only the informative nature of the images, but clearly indicate the variability of distributions based on position, and the similarity and differences of distributions across various species