14044536
Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding optimal policies for POMDPs. It typically takes a large number of iterations to converge. This paper proposes a method for accelerating the convergence of value iteration. The method has been evaluated on an array of benchmark problems and was found to be very effective: It enabled value iteration to converge after only a few iterations on all the test problems.
2749219
A Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes
We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incorporated into any existing POMDP value iteration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called incremental pruning. We find that the technique can make incremental pruning run several orders of magnitude faster.
18395713
Value Iteration over Belief Subspace
Partially Observable Markov Decision Processes (POMDPs) provide an elegant framework for AI planning tasks with uncertainties. Value iteration is a well-known algorithm for solving POMDPs. It is notoriously difficult because at each step it needs to account for every belief state in a continuous space. In this paper, we show that value iteration can be conducted over a subset of belief space. Then, we study a class of POMDPs, namely informative POMDPs, where each observation provides good albeit incomplete information about world states. For informative POMDPs, value iteration can be conducted over a small subset of belief space. This yields two advantages: First, fewer vectors are in need to represent value functions. Second, value iteration can be accelerated. Empirical studies are presented to demonstrate these two advantages.
646781
Value iteration working with belief subset
Value iteration is a popular algorithm for solving POMDPs. However, it is inefficient in practice. The primary reason is that it needs to conduct value updates for all the belief states in the (continuous) belief space. In this paper, we study value iteration working with a subset of the belief space, i.e., it conducts value updates only for belief states in the subset. We present a way to select belief subset and describe an algorithm to conduct value iteration over the selected subset. The algorithm is attractive in that it works with belief subset but also retains the quality of the generated values. Given a POMDP, we show how to <i>a priori</i> determine whether the selected subset is a <i>proper</i> subset of belief space. If this is the case, the algorithm carries the advantages of representation in space and efficiency in time.
16189288
An Alternative Formulation of Dynamic-Programming Updates for POMDPs
3023214
Take a walk and cluster genes: a TSP-based approach to optimal rearrangement clustering
Cluster analysis is a fundamental problem and technique in many areas related to machine learning. In this paper, we consider rearrangement clustering, which is the problem of finding sets of objects that share common or similar features by arranging the rows (objects) of a matrix (specifying object features) in such a way that adjacent objects are similar to each other (based on a similarity measure of the features) so as to maximize the overall similarity. Based on formulating this problem as the Traveling Salesman Problem (TSP), we develop a new TSP-based optimal clustering algorithm called TSPCluster. We overcome a flaw that is inherent in previous approaches by relaxing restrictions on dissimilarities between clusters. Our new algorithm has three important features: finding the optimal k clusters for a given k, automatically detecting cluster borders, and ascertaining a set of most viable clustering results that make good balances among maximizing the overall similarity within clusters and dissimilarity between clusters. We apply TSPCluster to cluster and display ~500 genes of flowering plant Arabidopsis which are regulated under various abiotic stress conditions. We compare TSPCluster to the bond energy algorithm and two existing clustering algorithms. Our TSPCluster code is available at (Climer & Zhang, 2004).
20221832
Moving cast shadows detection based on ratio edge
In this paper we propose a novel method for moving cast shadows detection. Based on the analysis to the physical model of moving shadows, we prove that the ratio edge is illumination invariant. The distribution of the ratio edge is discussed and a significance test is performed to classify each image pixel into foreground object or moving shadow. Experimental results on typical scenes show that the proposed method can detect moving shadows robustly
28880809
A feature-oriented approach to modeling requirements dependencies
There are many researches on requirements dependencies. However, most of them limit their views to the requirements phase of software development, few focus on the roles of requirements dependencies in the solution space of software. This paper presents a feature-oriented approach to modeling requirements dependencies. A feature is a set of tight-related requirements from user/customer-views. The feature-orientation provides a modular way to organize requirements and a proper granularity to analyze requirements dependencies. In this approach, we care about not only static feature dependencies (i. e. refinements and constraints), but also feature dependencies at the specification level (namely influences) and, furthermore, dynamic feature dependencies (namely interactions). Moreover, we also explore the underlying connections between these four kinds of feature dependency. By this way, this approach gives a more complete view of how requirements dependencies influence the whole process of software development.
43025995
Processing Probabilistic Range Query over Imprecise Data Based on Quality of Result
Sensors are employed in many applications to monitor entities such as environment temperature or illumination. Because of limited resources (e.g., battery power and wireless bandwidth), it is impractical for the database to track continuous changing readings from sensors at all times. Consequently, database may produce incorrect query results based on recorded data. However, if the error between the recorded value and the actual value is bounded, probabilistic approach can be applied to process queries over the imprecise data in database. The paper proposes a quality of result based probabilistic range query processing method. Two measures are defined to evaluate the quality of result. A probability model is proposed to estimate the quality of returned results. Based on the model, an algorithm is developed to choose the probability threshold according to user’s requirement on the quality of result.
9581791
A genetic clustering method for intrusion detection
Traditional intrusion detection methods lack extensibility in face of changing network configurations as well as adaptability in face of unknown attack types. Meanwhile, current machine-learning algorithms need labeled data for training first, so they are computational expensive and sometimes misled by artificial data. In this paper, a new detection algorithm, the Intrusion Detection Based on Genetic Clustering (IDBGC) algorithm, is proposed. It can automatically establish clusters and detect intruders by labeling normal and abnormal groups. Computer simulations show that this algorithm is effective for intrusion detection.