8016353
Restricted Value Iteration: Theory and Algorithms
Value iteration is a popular algorithm for finding near optimal policies for POMDPs. It is ineffcient due to the need to account for the entire belief space, which necessitates the solution of large numbers of linear programs. In this paper, we study value iteration restricted to belief subsets. We show that, together with properly chosen belief subsets, restricted value iteration yields near-optimal policies and we give a condition for determining whether a given belief subset would bring about savings in space and time. We also apply restricted value iteration to two interesting classes of POMDPs, namely informative POMDPs and near-discernible POMDPs.
2749219
A Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes
We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incorporated into any existing POMDP value iteration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called incremental pruning. We find that the technique can make incremental pruning run several orders of magnitude faster.
14044536
Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding optimal policies for POMDPs. It typically takes a large number of iterations to converge. This paper proposes a method for accelerating the convergence of value iteration. The method has been evaluated on an array of benchmark problems and was found to be very effective: It enabled value iteration to converge after only a few iterations on all the test problems.
18395713
Value Iteration over Belief Subspace
Partially Observable Markov Decision Processes (POMDPs) provide an elegant framework for AI planning tasks with uncertainties. Value iteration is a well-known algorithm for solving POMDPs. It is notoriously difficult because at each step it needs to account for every belief state in a continuous space. In this paper, we show that value iteration can be conducted over a subset of belief space. Then, we study a class of POMDPs, namely informative POMDPs, where each observation provides good albeit incomplete information about world states. For informative POMDPs, value iteration can be conducted over a small subset of belief space. This yields two advantages: First, fewer vectors are in need to represent value functions. Second, value iteration can be accelerated. Empirical studies are presented to demonstrate these two advantages.
16189288
An Alternative Formulation of Dynamic-Programming Updates for POMDPs
646781
Value iteration working with belief subset
Value iteration is a popular algorithm for solving POMDPs. However, it is inefficient in practice. The primary reason is that it needs to conduct value updates for all the belief states in the (continuous) belief space. In this paper, we study value iteration working with a subset of the belief space, i.e., it conducts value updates only for belief states in the subset. We present a way to select belief subset and describe an algorithm to conduct value iteration over the selected subset. The algorithm is attractive in that it works with belief subset but also retains the quality of the generated values. Given a POMDP, we show how to <i>a priori</i> determine whether the selected subset is a <i>proper</i> subset of belief space. If this is the case, the algorithm carries the advantages of representation in space and efficiency in time.
2426741
An analysis and application of distributed constraint satisfaction and optimization algorithms in sensor networks
This research is motivated by a distributed scheduling problem in distributed sensor networks, in which computational resources are scarce. To cope with limited computational resources and restricted real-time requirement, it is imperative to apply distributed algorithms that have low overhead on resource requirement and high anytime performance. In this paper, We study distributed stochastic algorithm (DSA) and distributed breakout algorithm (DBA), two distributed algorithms developed earlier for distributed constraint satisfaction problems. We experimentally investigate their properties and compare their performance using our distributed scheduling problem as a benchmark. We first formulate the scheduling problem as a distributed multi-coloring problem. We then experimentally show that the solution quality and communication cost of DSA exhibit phase-transition or threshold behavior, in that the performance degenerates abruptly and dramatically when the degree of parallel executions of distributed agents increases beyond some critical value. The results show that when controlled properly, DSA is superior to DBA, having better or competitive solution quality and significantly smaller communication cost than DBA. Therefore, DSA is the algorithm of choice for our distributed scan scheduling problem.
60947848
Backbone Guided Local Search for Maximum Satisfiability
Maximum satisfiability (Max-SAT) is more general and more difficult to solve than satisfiability (SAT). In this paper, we first investigate the effectiveness of Walksat, one of the best local search algorithms designed for SAT, on Max-SAT. We show that Walksat is also effective on Max-SAT, while its effectiveness degrades as the problem is more constrained. We then develop a novel method that exploits the backbone information in the local minima from Walksat and applies the backbone information in different ways to improve the performance of the Walksat algorithm. We call our new algorithm backbone guided Walksat (BGWalksat). On large random SAT and Max-SAT problems as well as instances from the SATLIB, BGWalksat significantly improves Walksat's performance.
6016662
Generalized optimal thresholding for biometric key generation using face images
In this paper, we study biometric key generation using face images. Given a face image, a set of biometric features are extracted. Each feature is compared with a threshold to generate a key-bit. By cascading the key-bits from all the features, we obtain one bio-key that can be used for security applications. The performance of a biometric key generation system, determined by the chosen thresholds, can be evaluated according to reliability and security. A generalized optimal thresholding method is proposed in this paper to improve the reliability by minimizing the authentication error rate and the security by maximizing the entropy of the generated key.
9644406
Moving vehicles segmentation based on Bayesian framework for Gaussian motion model
Accurate moving vehicles segmentation is an interesting yet difficult problem in intelligent transportation system. Robust segmentation is obstructed by some problems: the interferential moving objects in dynamic scenes, the vibration of cameras and the change of lighting conditions are all examples. In this paper, we propose Gaussian motion model for moving vehicles segmentation in the dynamic scenes. By investigating the distinction between the motion vectors of the dynamic background and those of the moving vehicles, we find the fact that the motion vectors of the moving vehicles cluster in a small region while those of the dynamic background are dispersive. Gaussian motion model is then proposed to model the motion of the moving vehicles and that of the dynamic background. Bayesian framework is utilized to classify the motion in the scenes to improve the robustness of the model and EM algorithm is used to estimate the parameters of the model. Experimental results on typical scenes, such as waving trees, camera vibration, snowfall and fog, show that the proposed model can segment the moving vehicles correctly in the complex scenes. Quantitative evaluations demonstrate that the proposed method outperform the existing methods.
16936043
Complete Anytime Beam Search
Beam search executes a state-space search, but may abandon nonpromising search avenues in order to reduce complexity. Although it has existed for more than two decades and has been applied to many real-world problems, beam search still suffers from the drawback of possible termination with no solution or a solution of unsatisfactory quality. In this paper, we first propose a domain-independent heuristic for node pruning, and a method to reduce the possibility that beam search will fail. We then develop a complete beam search algorithm. The new algorithm can not only find an optimal solution, but can also reach better solutions sooner than its underlying search method. We apply complete beam search to the maximum boolean satisfiability and the symmetric and asymmetric Traveling Salesman Problems. Our experimental results show that the domain-independent pruning heuristic is effective and the new algorithm significantly improves the performance of its underlying search algorithm.