46568605
Reinforcement Learning Based on Multi-agent in RoboCup
Multi-agent systems form a particular type of distributed artificial intelligence systems. As an important character of players in game, autonomous agents' learning has become the main direction of researchers. In this paper, based on basic reinforcement learning, multi-agent reinforcement learning with specific context is proposed. The method is applied to RoboCup to learn coordination among agents. In the learning, the game field is divided into different areas, and the action choice is made dependent on the area in which the ball is currently located. This makes the state space and the action space decrease. After learning the optimal joint policy is determined. Comparison experiment between stochastic policy and this optimal policy shows the effectiveness of our approach.
14422830
Compiler-guided register reliability improvement against soft errors
With the scaling of technology, transient errors caused by external particle strikes have become a critical challenge for microprocessor design. As embedded processors are widely used in reliability-sensitive environments, it becomes increasingly important to develop cost-effective techniques to improve the processor reliability against soft errors. This paper focuses on studying the register file immunity against soft errors since modern processors typically employ a large number of registers, which are accessed very frequently. As a result, soft errors occurred in registers can easily propagate to functional units or the memory system, leading to silent data error (SDC) or system crash.To develop cost-effective techniques to fight soft errors for embedded processors, the first step is to understand the register file susceptibility to soft errors and its impact on the system reliability accurately. Toward this goal, this paper proposes the concept of register vulnerability factor (RVF) to characterize the probability that register transient errors can escape the register file and thus potentially impact the system reliability. Built upon the RVF concept, we then propose two cost-effective compiler-guided techniques to improve the register file reliability by lowering the RVF value. Our experiments indicate that on average, the RVF can be reduced to 9.1% and 9.5% by the hyperblock-based instruction re-scheduling and the reliability-oriented register assignment respectively, which can potentially lower the reliability cost significantly while protecting register files against transient errors.
43025995
Processing Probabilistic Range Query over Imprecise Data Based on Quality of Result
Sensors are employed in many applications to monitor entities such as environment temperature or illumination. Because of limited resources (e.g., battery power and wireless bandwidth), it is impractical for the database to track continuous changing readings from sensors at all times. Consequently, database may produce incorrect query results based on recorded data. However, if the error between the recorded value and the actual value is bounded, probabilistic approach can be applied to process queries over the imprecise data in database. The paper proposes a quality of result based probabilistic range query processing method. Two measures are defined to evaluate the quality of result. A probability model is proposed to estimate the quality of returned results. Based on the model, an algorithm is developed to choose the probability threshold according to user’s requirement on the quality of result.
17633305
The Impact of Cache Organization in Optimizing Microprocessor Power Consumption
In the recent years, power consumption has become increasingly an important design concern as silicon area and performance in modern computer systems design. Several factors have contributed to this trend. Perhaps the most visible have been the remarkable success and growth of Personal Digital Assistants (PDA’s), Cellular phones and pagers, etc .This work is an attempt to explore the impact of feature size shrinkage and cache configuration on optimizing power consumption in modern processors. The work studies two commercially known RISC microprocessors. The StrongARM and the Alpha-21064 microprocessors. The latter is the ancestor of the former and is therefore used here as the baseline in our analysis. Our analysis has illustrated quantitatively great power saving when technology is downsized and cache is well organized
12895493
Improvement of HITS-based algorithms on web documents
In this paper, we present two ways to improve the precision of HITS-based algorithms on Web documents. First, by analyzing the limitations of current HITS-based algorithms, we propose a new weighted HITS-based method that assigns appropriate weights to in-links of root documents. Then, we combine content analysis with HITS-based algorithms and study the effects of four representative relevance scoring methods, VSM, Okapi, TLS, and CDR, using a set of broad topic queries. Our experimental results show that our weighted HITS-based method performs significantly better than Bharat's improved HITS algorithm. When we combine our weighted HITS-based method or Bharat's HITS algorithm with any of the four relevance scoring methods, the combined methods are only marginally better than our weighted HITS-based method. Between the four relevance-scoring methods, there is no significant quality difference when they are combined with a HITS-based algorithm.
9354212
High-Performance Job-Shop Scheduling With A Time-Delay TD-lambda Network
Job-shop scheduling is an important task for manufacturing industries. We are interested in the particular task of scheduling payload processing for NASA's space shuttle program. This paper summarizes our previous work on formulating this task for solution by the reinforcement learning algorithm T D(). A shortcoming of this previous work was its reliance on hand-engineered input features. This paper shows how to extend the time-delay neural network (TDNN) architecture to apply it to irregular-length schedules. Experimental tests show that this TDNN-TD() network can match the performance of our previous hand-engineered system. The tests also show that both neural network approaches signiicantly out-perform the best previous (non-learning) solution to this problem in terms of the quality of the resulting schedules and the number of search steps required to construct them.
43211087
Simplify Decision Function of Reduced Support Vector Machines
Reduced Support Vector Machines (RSVM) was proposed as the alternate of standard support vector machines (SVM) in order to resolve the difficulty in the learning of nonlinear SVM for large data set problems. RSVM preselects a subset as support vectors and solves a smaller optimization problem, and it performs well with remarkable efficiency on training of SVM for large problem. All the training points of the subset will be support vectors, and more training points are selected into this subset results in high possibility to obtain RSVM with better generalization ability. So we first obtain the RSVM with more support vectors, and selects out training examples near classification hyper plane. Then only these training examples are used as training set to obtain a standard SVM with less support vector than that of RSVM. Computational results show that standard SVMs on the basis of RSVM have much less support vectors and perform equal generalization ability to that of RSVM.
1196346
An Efficient Hybrid Strategy for Temporal Planning
Temporal planning (TP) is notoriously difficult because it requires to solve a propositional STRIPS planning problem with temporal constraints. In this paper, we propose an efficient strategy for solving TP, which combines, in an innovative way, several well established and studied techniques in AI, OR and constraint programming. Our approach integrates graph planning (a well studied planning paradigm), max-SAT (a constraint optimization technique), and the Program Evaluation and Review Technique (PERT), a well established technique in OR. Our method first separates the logical and temporal constraints of a TP problem and solves it in two phases. In the first phase, we apply our new STRIPS planner to generate a parallel STRIPS plan with a minimum number of parallel steps. Our new STRIPS planner is based on a new max-SAT formulation, which leads to an effective incremental learning scheme and a goal-oriented variable selection heuristic. The new STRIPS planner can generate optimal parallel plans more efficiently than the well-known SATPLAN approach. In the second phase, we apply PERT to schedule the activities in a parallel plan to create a shortest temporal plan given the STRIPS plan. When applied to the first optimal parallel STRIPS plan, this simple strategy produces optimal temporal plans on most benchmarks we have tested. This strategy can also be applied to optimal STRIPS plans of different parallel steps in an anytime fashion to find an optimal temporal plan. Our experimental results show that the new strategy is effective and the resulting algorithm outperforms many existing temporal planners.
18126681
Phase Transitions and Backbones of 3-SAT and Maximum 3-SAT
Many real-world problems involve constraints that cannot be all satisfied. Solving an overconstrained problem then means to find solutions minimizing the number of constraints violated, which is an optimization problem. In this research, we study the behavior of the phase transitions and backbones of constraint optimization problems. We first investigate the relationship between the phase transitions of Boolean satisfiability, or precisely 3-SAT (a well-studied NP-complete decision problem), and the phase transitions of MAX 3-SAT (an NP-hard optimization problem). To bridge the gap between the easy-hard-easy phase transitions of 3-SAT and the easy-hard transitions of MAX 3-SAT, we analyze bounded 3-SAT, in which solutions of bounded quality, e.g., solutions with at most a constant number of constraints violated, are sufficient. We show that phase transitions are persistent in bounded 3-SAT and are similar to that of 3-SAT. We then study backbones of MAX 3-SAT, which are critically constrained variables that have fixed values in all optimal solutions. Our experimental results show that backbones of MAX 3-SAT emerge abruptly and experience sharp transitions from nonexistence when underconstrained to almost complete when overconstrained. More interestingly, the phase transitions of MAX 3-SAT backbones seem to concur with the phase transitions of satisfiability of 3-SAT. The backbone of MAX 3-SAT with size 0.5 approximately collocates with the 0.5 satisfiability of 3-SAT, and the backbone and satisfiability seems to follow a linear correlation near this 0.5-0.5 collocation.
16936043
Complete Anytime Beam Search
Beam search executes a state-space search, but may abandon nonpromising search avenues in order to reduce complexity. Although it has existed for more than two decades and has been applied to many real-world problems, beam search still suffers from the drawback of possible termination with no solution or a solution of unsatisfactory quality. In this paper, we first propose a domain-independent heuristic for node pruning, and a method to reduce the possibility that beam search will fail. We then develop a complete beam search algorithm. The new algorithm can not only find an optimal solution, but can also reach better solutions sooner than its underlying search method. We apply complete beam search to the maximum boolean satisfiability and the symmetric and asymmetric Traveling Salesman Problems. Our experimental results show that the domain-independent pruning heuristic is effective and the new algorithm significantly improves the performance of its underlying search algorithm.
2581690
Distributed breakout revisited
Distributed breakout algorithm (DBA) is an efficient method for solving distributed constraint satisfaction problems (CSP). Inspired by its potential of being an efficient, low-overhead agent coordination method for problems in distributed sensor networks, we study DBA's properties in this paper. We specifically show that on an acyclic graph of <i>n</i> nodes, DBA can find a solution in <i>O(n<sup>2</sup>)</i> synchronized distributed steps. This completeness result reveals DBA's superiority over conventional local search on acyclic graphs and implies its potential as a simple self-stabilization method for tree-structured distributed systems. We also show a worst case of DBA in a cyclic graph where it never terminates. To overcome this problem on cyclic graphs, we propose two stochastic variations to DBA. Our experimental analysis shows that stochastic DBAs are able to avoid DBA's worst-case scenarios and has similar performance as that of DBA.