5661913
Distributed Value Functions
Many interesting problems, such as power grids, network switches, and tra c ow, that are candidates for solving with reinforcement learning (RL), also have properties that make distributed solutions desirable. We propose an algorithm for distributed reinforcement learning based on distributing the representation of the value function across nodes. Each node in the system only has the ability to sense state locally, choose actions locally, and receive reward locally (the goal of the system is to maximize the sum of the rewards over all nodes and over all time). However each node is allowed to give its neighbors the current estimate of its value function for the states it passes through. We present a value function learning rule, using that information, that allows each node to learn a value function that is an estimate of a weighted sum of future rewards for all the nodes in the network. With this representation, each node can choose actions to improve the performance of the overall system. We demonstrate our algorithm on the distributed control of a simulated power grid. We compare it against other methods including: use of a global reward signal, nodes that act locally with no communication, and nodes that share rewards (but not value function) information with each other. Our results show that the distributed value function algorithm outperforms the others, and we conclude with an analysis of what problems are best suited for distributed value functions and the new research directions opened up by this work.
6423442
Rule-based anomaly pattern detection for detecting disease outbreaks
This paper presents an algorithm for performing early detection of disease outbreaks by searching a database of emergency department cases for anomalous patterns. Traditional techniques for anomaly detection are unsatisfactory for this problem because they identify individual data points that are rare due to particular combinations of features. When applied to our scenario, these traditional algorithms discover isolated outliers of particularly strange events, such as someone accidentally shooting their ear, that are not indicative of a new outbreak. Instead, we would like to detect anomalous patterns. These patterns are groups with specific characteristics whose recent pattern of illness is anomalous relative to historical patterns. We propose using a rule-based anomaly detection algorithm that characterizes each anomalous pattern with a rule. The significance of each rule is carefully evaluated using Fisher's Exact Test and a randomization test. Our algorithm is compared against a standard detection algorithm by measuring the number of false positives and the timeliness of detection. Simulated data, produced by a simulator that creates the effects of an epidemic on a city, is used for evaluation. The results indicate that our algorithm has significantly better detection times for common significance thresholds while having a slightly higher false positive rate.
2986922
Bayesian Network Anomaly Pattern Detection for Disease Outbreaks
Early disease outbreak detection systems typically monitor health care data for irregularities by comparing the distribution of recent data against a baseline distribution. Determining the baseline is difficult due to the presence of different trends in health care data, such as trends caused by the day of week and by seasonal variations in temperature and weather. Creating the baseline distribution without taking these trends into account can lead to unacceptably high false positive counts and slow detection times. This paper replaces the baseline method of (Wong et al., 2002) with a Bayesian network which produces the baseline distribution by taking the joint distribution of the data and conditioning on attributes that are responsible for the trends. We show that our algorithm, called WSARE 3.0, is able to detect outbreaks in simulated data with almost the earliest possible detection time while keeping a low false positive count. We also include the results of running WSARE 3.0 on real Emergency Department data
42032352
Static and dynamic distance metrics for feature-based code analysis
In this paper we present metrics to determine the distance between the features of a software system. Such a measurement can elucidate how features of the system being examined are close to each other. We first use an execution slice-based technique to identify a set of code (basic blocks in our case) that is used to implement each feature. Then, depending on whether the execution frequency of each block is considered during the construction of such sets of code, a static as well as a dynamic distance is computed for each pair of features. These two types of distance differ in that the former computes the distance between two features only by how these features are implemented in the system, while the latter also takes into account how each feature is executed based on a user's operational profile. In other words, the static distance quantitatively gives the closeness of two features from the system implementation point of view, whereas the dynamic distance presents such closeness from the users' execution point of view. To illustrate the use of our metrics, we report a case study on a Symbolic Hierarchical Automated Reliability and Performance Evaluator (SHARPE). The results of our study suggest that the distance metrics discussed in this paper can provide a good measurement, in a quantitative way, of how close two program features are. Such information can also serve as a good start to understanding how a modification made to one feature is likely to affect other features.
6734680
Compiler Optimizations for Adaptive EPIC Processors
Advances in VLSI technology have lead to a tremendous increase in the density and number of devices that can be manufactured in a single microchip. One of the interesting ways in which this silicon may be used is to leave portions of it uncommitted and re-programmable depending on an applications needs. In an earlier paper, we proposed a machine architecture for achieving this reconfigurability and compilation issues that such an architecture will face. In this paper, we will elaborate on the compiler optimization issues involved. In particular, we will outline a framework for code partitioning, instruction synthesis, configuration selection, resource allocation, and instruction scheduling. Partitioning is the problem of identifying code sections that may benefit by mapping them on to the programmable logic resources. The instruction synthesis phase generates suitable implementations for the candidates partitions and updates the machine description database with the new instructions. Configuration selection is the problem of narrowing down the choices of which synthesized instruction (from the set generated by the instruction synthesis phase) to use for each of the code regions that will be mapped to programmable logic. Unlike traditional optimizing compilers, the adaptive EPIC compiler must deal with the existence of synthesized instructions. Compilation techniques addressing each of these problems will be presented.
62666364
Applying design metrics to predict fault-proneness: a case study on a large-scale software system
The purpose of this study was to identify fault-prone functions that are likely to contain faults in a given software system. Five metrics were used: Di, an internal design metric which incorporates factors related to a function's internal structure; De, an external design metric which focuses on a function's external relationships to the rest of the software system; D(G), a composite design metric which is a linear combination of Di and De; and the union and intersection of Di, De, and D(G). Since the system being considered was already developed, a very important aspect of our study was to extract the design information directly from the source code rather than from the corresponding design documentation which may not exist or, if it does exist, it may be incomplete, difficult to understand, or not updated. To make the analysis more accurate and efficient, a metric analysis tool (χMetrics) was implemented. We conducted experiments using χMetrics on part of a distributed software system, written in C, with a client–server architecture, and identified a small percentage of its functions as good candidates for fault-proneness. Files containing these functions were then validated by the real defect data collected between a recent major release and its subsequent release for their fault-proneness. The results indicate that our metrics are good indicators of fault-prone functions. Two extra experiments were also conducted to show that function size cannot replace any of our metrics; and where the function size was factored out our metrics performed better than the normalized metrics. The important benefit of our metrics is that they help project managers determine where additional testing effort should be spent and possibly which fault-prone functions should be assigned to more experienced programmers if modifications are required. Copyright © 2000 John Wiley & Sons, Ltd.
1096223
Vascular segmentation in three-dimensional rotational angiography based on maximum intensity projections
Three-dimensional rotational angiography (3D-RA) is a relatively new and promising technique for imaging blood vessels. In this paper, we propose a novel 3D-RA vascular segmentation algorithm, which is fully automatic and very computationally efficient, based on the maximum intensity projections (MIP) of 3D-RA images. Validation results on 13 clinical 3D-RA datasets reveal that, according to the agreement between the segmentation results and the ground truth, our method (a) outperforms both the maximum a posteriori-expectation maximization (MAP-EM)-based method and the MAP-Markov random field (MAP-MRF)-based segmentation method, and (b) works comparably to the optimal global thresholding method. Experimental results also show that our method can successfully segment major vascular structures in 3D-RA and produce a lower false positive rate than that of the MAP-EM-based and MAP-MRF-based methods.
44943315
Establishing structural testing criteria for Java bytecode
This paper describes intra-method control-flow and data-flow testing criteria for the Java bytecode language. Six testing criteria are considered for the generation of testing requirements: four control-flow and two data-flow based. The main reason to work at a lower level is that, even when there is no source code, structural testing requirements can still be derived and used to assess the quality of a given test set. It can be used, for instance, to perform structural testing on third-party Java components. In addition, the bytecode can be seen as an intermediate language, so the analysis performed at this level can be mapped back to the original high-level language that generated the bytecode. To support the application of the testing criteria, we have implemented a tool named JaBUTi (Java Bytecode Understanding and Testing). JaBUTi is used to illustrate the application of the ideas developed in this paper. Copyright c © 2006 John Wiley & Sons, Ltd.