18395713
Value Iteration over Belief Subspace
Partially Observable Markov Decision Processes (POMDPs) provide an elegant framework for AI planning tasks with uncertainties. Value iteration is a well-known algorithm for solving POMDPs. It is notoriously difficult because at each step it needs to account for every belief state in a continuous space. In this paper, we show that value iteration can be conducted over a subset of belief space. Then, we study a class of POMDPs, namely informative POMDPs, where each observation provides good albeit incomplete information about world states. For informative POMDPs, value iteration can be conducted over a small subset of belief space. This yields two advantages: First, fewer vectors are in need to represent value functions. Second, value iteration can be accelerated. Empirical studies are presented to demonstrate these two advantages.
16189288
An Alternative Formulation of Dynamic-Programming Updates for POMDPs
2749219
A Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes
We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incorporated into any existing POMDP value iteration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called incremental pruning. We find that the technique can make incremental pruning run several orders of magnitude faster.
646781
Value iteration working with belief subset
Value iteration is a popular algorithm for solving POMDPs. However, it is inefficient in practice. The primary reason is that it needs to conduct value updates for all the belief states in the (continuous) belief space. In this paper, we study value iteration working with a subset of the belief space, i.e., it conducts value updates only for belief states in the subset. We present a way to select belief subset and describe an algorithm to conduct value iteration over the selected subset. The algorithm is attractive in that it works with belief subset but also retains the quality of the generated values. Given a POMDP, we show how to <i>a priori</i> determine whether the selected subset is a <i>proper</i> subset of belief space. If this is the case, the algorithm carries the advantages of representation in space and efficiency in time.
12895493
Improvement of HITS-based algorithms on web documents
In this paper, we present two ways to improve the precision of HITS-based algorithms on Web documents. First, by analyzing the limitations of current HITS-based algorithms, we propose a new weighted HITS-based method that assigns appropriate weights to in-links of root documents. Then, we combine content analysis with HITS-based algorithms and study the effects of four representative relevance scoring methods, VSM, Okapi, TLS, and CDR, using a set of broad topic queries. Our experimental results show that our weighted HITS-based method performs significantly better than Bharat's improved HITS algorithm. When we combine our weighted HITS-based method or Bharat's HITS algorithm with any of the four relevance scoring methods, the combined methods are only marginally better than our weighted HITS-based method. Between the four relevance-scoring methods, there is no significant quality difference when they are combined with a HITS-based algorithm.
9644406
Moving vehicles segmentation based on Bayesian framework for Gaussian motion model
Accurate moving vehicles segmentation is an interesting yet difficult problem in intelligent transportation system. Robust segmentation is obstructed by some problems: the interferential moving objects in dynamic scenes, the vibration of cameras and the change of lighting conditions are all examples. In this paper, we propose Gaussian motion model for moving vehicles segmentation in the dynamic scenes. By investigating the distinction between the motion vectors of the dynamic background and those of the moving vehicles, we find the fact that the motion vectors of the moving vehicles cluster in a small region while those of the dynamic background are dispersive. Gaussian motion model is then proposed to model the motion of the moving vehicles and that of the dynamic background. Bayesian framework is utilized to classify the motion in the scenes to improve the robustness of the model and EM algorithm is used to estimate the parameters of the model. Experimental results on typical scenes, such as waving trees, camera vibration, snowfall and fog, show that the proposed model can segment the moving vehicles correctly in the complex scenes. Quantitative evaluations demonstrate that the proposed method outperform the existing methods.
16936043
Complete Anytime Beam Search
Beam search executes a state-space search, but may abandon nonpromising search avenues in order to reduce complexity. Although it has existed for more than two decades and has been applied to many real-world problems, beam search still suffers from the drawback of possible termination with no solution or a solution of unsatisfactory quality. In this paper, we first propose a domain-independent heuristic for node pruning, and a method to reduce the possibility that beam search will fail. We then develop a complete beam search algorithm. The new algorithm can not only find an optimal solution, but can also reach better solutions sooner than its underlying search method. We apply complete beam search to the maximum boolean satisfiability and the symmetric and asymmetric Traveling Salesman Problems. Our experimental results show that the domain-independent pruning heuristic is effective and the new algorithm significantly improves the performance of its underlying search algorithm.
8235488
Enhancing data cache reliability by the addition of a small fully-associative replication cache
Soft error conscious cache design is a necessity for reliable computing. ECC or parity-based integrity checking technique in use today either compromises performance for reliability or vice versa, and the N modular redundancy (NMR) scheme is too costly for microprocessors and applications with stringent cost constraint. This paper proposes a novel and cost-effective solution to enhance data reliability with minimum impact on performance. The idea is to add a small fully-associative cache to store the replica(s) of every write to the L1 data cache. The replicas can be used to detect and correct soft errors. The replication cache can also be used to increase performance by reducing the L1 data cache miss rate. Our experiments show that more than 97% read hits of the L1 data cache can find replicas available in a replication cache of 8 blocks.
6522122
Rearrangement Clustering: Pitfalls, Remedies, and Applications
Given a matrix of values in which the rows correspond to objects and the columns correspond to features of the objects, rearrangement clustering is the problem of rearranging the rows of the matrix such that the sum of the similarities between adjacent rows is maximized. Referred to by various names and reinvented several times, this clustering technique has been extensively used in many fields over the last three decades. In this paper, we point out two critical pitfalls that have been previously overlooked. The first pitfall is deleterious when rearrangement clustering is applied to objects that form natural clusters. The second concerns a similarity metric that is commonly used. We present an algorithm that overcomes these pitfalls. This algorithm is based on a variation of the Traveling Salesman Problem. It offers an extra benefit as it automatically determines cluster boundaries. Using this algorithm, we optimally solve four benchmark problems and a 2,467-gene expression data clustering problem. As expected, our new algorithm identifies better clusters than those found by previous approaches in all five cases. Overall, our results demonstrate the benefits of rectifying the pitfalls and exemplify the usefulness of this clustering technique. Our code is available at our websites.