503545
Privacy-preserving clustering with distributed EM mixture modeling
Privacy and security considerations can prevent sharing of data, derailing data mining projects. Distributed knowledge discovery can alleviate this problem. We present a technique that uses EM mixture modeling to perform clustering on distributed data. This method controls data sharing, preventing disclosure of individual data items or any results that can be traced to an individual site.
8765334
Privacy preserving regression modelling via distributed computation
Reluctance of data owners to share their possibly confidential or proprietary data with others who own related databases is a serious impediment to conducting a mutually beneficial data mining analysis. We address the case of vertically partitioned data -- multiple data owners/agencies each possess a few attributes of every data record. We focus on the case of the agencies wanting to conduct a linear regression analysis with complete records without disclosing values of their own attributes. This paper describes an algorithm that enables such agencies to compute the exact regression coefficients of the global regression equation and also perform some basic goodness-of-fit diagnostics while protecting the confidentiality of their data. In more general settings beyond the privacy scenario, this algorithm can also be viewed as method for the distributed computation for regression analyses.
17019459
Semantic video classification by integrating flexible mixture model with adaptive EM algorithm
Digital video now plays an important role in medical education and healthcare, but our ability to automatic video indexing at semantic level is currently primitive. In this paper, we propose a novel framework to enable more effective semantic video classification and indexing in a specific surgery education video domain. Specifically, this framework includes: (a) A novel semantic-sensitive video content characterization and representation framework by using principal video shots and their perceptual multimodal features. (b) A novel semantic medical concept interpretation technique by using flexible mixture model. (c) A semantic video classifier by using an adaptive Expectation-Maximization (EM) algorithm for automatic parameter estimation and model selection (i.e., selecting the optimal number of mixture Gaussian components). Since more effective video content characterization framework has been integrated with an adaptive EM algorithm for video classification, our semantic video classifier has improved the classification accuracy significantly. For skin classification, its accuracy is close to 95.5%. For semantic surgical video classification, it achieves overall ≈ 84.6% accuracy.
15095892
An Efficient Distributed Algorithm for Computing Association Rules
Data mining aims to efficiently discover previously unknown knowledge from large databases. It is highly demanding in numerous real-life applications, such as marketing strategy, financial forecast, etc. One of the fundamental problems in the area is the efficient computation of association rules. In this paper, we shall investigate this problem in a distributed database. Particularly, we will present an efficient distributed algorithm for mining distributed association rules. Our experiment results suggest that the proposed algorithm outperforms the existing distributed algorithms. Further, we also study a distributed version of the problem of association rules; and extend our algorithm to solve this new problem.
9661555
Multicast in Fat-Tree-Based InfiniBand Networks
The multicast operation is a very commonly used operation in parallel applications. With the hardware supported multicast of the InfiniBand architecture (IBA), we propose a cyclic multicast scheme for fat-tree-based (m-port n-tree) InfiniBand networks. The basic concept of the proposed cyclic multicast scheme is to find the union sets of the output ports of switches in the paths between the source processing node and each destination processing node in a multicast group. Based on the union sets and the path selection scheme, the forwarding table for a given multicast group can be constructed. We implement the proposed multicast scheme along with the OpenSM multicast scheme and the unicast scheme on an m-port n-tree InfiniBand network simulator. The simulation results show that the proposed multicast scheme outperforms the unicast scheme for all simulated cases. For many-to-many and all-to-many cases, the cyclic multicast scheme outperforms the OpenSM multicast scheme. For many-to-all case, the performance of the cyclic multicast scheme is a little better than that of the OpenSM multicast scheme
1998178
A Hierarchical Approach to Internet Distance Prediction
Internet distance prediction gives pair-wise latency information with limited measurements. Recent studies have revealed that the quality of existing prediction mechanisms from the application perspective is short of satisfactory. In this paper, we explore the root causes and remedies for this problem. Our experience with different landmark selection schemes shows that although selecting nearby landmarks can increase the prediction accuracy for short distances, it can cause the prediction accuracy for longer distances to degrade. Such uneven prediction quality significantly impacts application performance. Instead of trying to select the landmark nodes in some "intelligent" fashion, we propose a hierarchical prediction approach with straightforward landmark selection. Hierarchical prediction utilizes multiple coordinate sets at multiple distance scales, with the "right" scale being chosen for prediction each time. Experiments with Internet measurement datasets show that this hierarchical approach is extremely promising for increasing the accuracy of network distance prediction.
7595419
Efficient Computation of k-Medians over Data Streams Under Memory Constraints
In this paper, we study the problem of efficiently computing k-medians over high-dimensional and high speed data streams. The focus of this paper is on the issue of minimizing CPU time to handle high speed data streams on top of the requirements of high accuracy and small memory. Our work is motivated by the following observation: the existing algorithms have similar approximation behaviors in practice, even though they make noticeably different worst case theoretical guarantees. The underlying reason is that in order to achieve high approximation level with the smallest possible memory, they need rather complex techniques to maintain a sketch, along time dimension, by using some existing off-line clustering algorithms. Those clustering algorithms cannot guarantee the optimal clustering result over data segments in a data stream but accumulate errors over segments, which makes most algorithms behave the same in terms of approximation level, in practice. We propose a new grid-based approach which divides the entire data set into cells (not along time dimension). We can achieve high approximation level based on a novel concept called (1−∊)-dominant. We further extend the method to the data stream context, by leveraging a density-based heuristic and frequent item mining techniques over data streams. We only need to apply an existing clustering once to computing k-medians, on demand, which reduces CPU time significantly. We conducted extensive experimental studies, and show that our approaches outperform other well-known approaches.
14038708
Efficiently computing frequent tree-like topology patterns in a Web environment
Data mining has recently emerged in a number of real life applications, such as banking, supermarketing, and interneting. The authors investigate a novel data mining problem derived from the applications of WWW. They present an efficient algorithm for solving the problem. The initial implementation suggests that the algorithm is very efficient and scalable in practice.