620811
Using data mining techniques to learn layouts of flat-file biological datasets
One of the major problems in biological data integration is that many data sources are stored as atlasses, with a variety of different layouts. Integrating data from such sources can be an extremely time-consuming task. We have been developing data mining techniques to help learn the layout of a dataset in a semi-automatic way. In this paper, we focus on the problem of identifying delimiters for optional fields. Since these fields do not occur in every record, frequency based methods are not able to identify the corresponding delimiters. We present a method which uses contrast analysis on the frequency of sequences to identify such delimiters and help complete the layout descriptions. We demonstrate the effectiveness of this technique using three atlasses biological datasets.
18526893
Scaling and parallelizing a scientific feature mining application using a cluster middleware
Summary form only given. As scientific simulations are generating large amounts of data, analyzing this data to gain insights into scientific phenomenon is increasingly becoming a challenge. We present a case study on the use of a cluster middleware for rapidly creating a scalable and parallel implementation of a scientific data analysis application. Using FREERIDE (framework for rapid implementation of data mining engines), we parallelize as well as scale to disk-resident datasets a feature extraction algorithm. We have developed a parallel algorithm for this problem which matches the communication and computation structure supported by the FREERIDE system. The main observations from our experimental results are as follows: 1) the overhead of using the middleware is quite small in most cases, 2) there is an overhead associated with breaking the datasets into more partitions or chunks, and 3) if the dataset is partitioned into the same number of chunks, the execution time stays proportional to the size of the dataset and inversely proportional to the number of nodes, i.e. the overhead of communication or reading disk-resident datasets is very small.
3788531
Degree-Light-Free Graphs and Hamiltonian Cycles
Abstract. Let G and H be graphs. G is said to be degree-light H-free if G is either H-free or, for every induced subgraph K of G with K≅H, and every {u,v}⊆K, distK(u,v)=2 implies max {d(u),d(v)}≥|V(G)|/2. In this paper, we will show that every 2-connected graph with either degree-light {K1,3, P6}-free or degree-light {K1,3, Z}-free is hamiltonian (with three exceptional graphs), where P6 is a path of order 6 and Z is obtained from P6 by adding an edge between the first and the third vertex of P6 (see Figure 1).
26388893
Grouping edge points into line segments by sequential Hough transformation
An algorithm to group edge points into digital line segments with Hough transformation is described. The edge points are mapped onto the parameter domain discretized at specific intervals, on which peaks appear to represent different line segments. By modeling each peak as a Gaussian function in the parameter domain, a region to which the edge points are supposed to be mapped is determined. Then the edge points are grouped and the parameters for a line segment are computed. For the edges including multiple line segments, a sequential Hough transformation for detecting peaks one by one in the parameter domain is implemented, and the points from the region around each peak are grouped, thus the line segments are described. Experiments show the robustness of the algorithm implemented on both the generated edges disturbed by different noise levels and real images taken from an indoor environment.
13183312
Wavelet based automatic thresholding for image segmentation
In this paper, a new systematic method to segment possible target areas based on wavelet transforms is presented. We develop an analytic model for the segmentation of targets, which uses a novel multiresolution analysis in concert with a Bayesian classifier to identify the possible target areas. A method is developed which adaptively chooses thresholds to segment targets from background, by using a multiscale analysis of the image probability density function (PDF). We present examples which demonstrate the efficiency of the technique on a variety of targets.
2953976
A performance study of monitoring and information services for distributed systems
Monitoring and information services form a key component of a distributed system, or Grid. A quantitative study of such services can aid in understanding the performance limitations, advise in the deployment of the monitoring system, and help evaluate future development work. To this end, we study the performance of three monitoring and information services for distributed systems: the Globus Toolkit/spl reg/ Monitoring and Discovery Service (MDS2), the European Data Grid Relational Grid Monitoring Architecture (R-GMA) and Hawkeye, part of the Condor project. We perform experiments to test their scalability with respect to number of users, number of resources and amount of data collected. Our study shows that each approach has different behaviors, often due to their different design goals. In the four sets of experiments we conducted to evaluate the performance of the service components under different circumstances, we found a strong advantage to caching or pre-fetching the data, as well as the need to have primary components at well-connected sites because of the high load seen by all systems.
17218997
Profiling program behavior for anomaly intrusion detection based on the transition and frequency property of computer audit data
Intrusion detection is an important technique in the defense-in-depth network security framework. In recent years, it has been a widely studied topic in computer network security. In this paper, we present two methods, namely, the Hidden Markov Models (HMM) method and the Self Organizing Maps (SOM) method, to profile normal program behavior for anomaly intrusion detection based on computer audit data. The HMM method utilizes the transition property of events while SOM method relies on the frequency property of events. Two data sets, CERT synthetic Sendmail system call data collected in the University of New Mexico (UNM) and Live FTP system call data collected in the CNSIS lab of Xi'an Jiaotong University, were used to assess the two methods. Testing results show that the HMM method using the transition property of events produces good detection performance while high computational expense is required both for training and detection. The HMM method is better than other two methods reported previously in terms of detection accuracy for the same data set. The SOM method considering the frequency property of events, on the other hand, is suitable for real-time intrusion detection because of its capability of processing a large amount of data with low computational overhead.