3194322
Efficient record linkage in large data sets
This paper describes an efficient approach to record linkage. Given two lists of records, the record-linkage problem consists of determining all pairs that are similar to each other where the overall similarity between two records is defined based on domain-specific similarities over individual attributes constituting the record. The record-linkage problem arises naturally in the context of data cleansing that usually precedes data analysis and mining. We explore a novel approach to this problem. For each attribute of records, we first map values to a multidimensional Euclidean space that preserves domain-specific similarity. Many mapping algorithms can be applied, and we use the FastMap approach as an example. Given the merging rule that defines when two records are Similar a set of attributes are chosen along which the merge will proceed A multidimensional similarity join over the chosen attributes is used to determine similar pairs of records. Our extensive experiments using real data sets show that our solution has very good efficiency and accuracy.
8510271
Supporting Efficient Record Linkage for Large Data Sets Using Mapping Techniques
This paper describes an efficient approach to record linkage. Given two lists of records, the record-linkage problem consists of determining all pairs that are similar to each other, where the overall similarity between two records is defined based on domain-specific similarities over individual attributes. The record-linkage problem arises naturally in the context of data cleansing that usually precedes data analysis and mining. Since the scalability issue of record linkage was addressed in [21], the repertoire of database techniques dealing with multidimensional data sets has significantly increased. Specifically, many effective and efficient approaches for distance-preserving transforms and similarity joins have been developed. Based on these advances, we explore a novel approach to record linkage. For each attribute of records, we first map values to a multidimensional Euclidean space that preserves domain-specific similarity. Many mapping algorithms can be applied, and we use the Fastmap approach [16] as an example. Given the merging rule that defines when two records are similar based on their attribute-level similarities, a set of attributes are chosen along which the merge will proceed. A multidimensional similarity join over the chosen attributes is used to find similar pairs of records. Our extensive experiments using real data sets show that our solution has very good efficiency and recall.
10902397
Protecting Individual Information Against Inference Attacks in Data Publishing
In many data-publishing applications, the data owner needs to protect sensitive information pertaining to individuals. Meanwhile, certain information is required to be published. The sensitive information could be considered as leaked, if an adversary can infer the real value of a sensitive entry with a high confidence. In this paper we study how to protect sensitive data when an adversary can do inference attacks using association rules derived from the data. We formulate the inference attack model, and develop complexity results on computing a safe partial table. We classify the general problem into subcases based on the requirements of publishing information, and propose the corresponding algorithms for finding a safe partial table to publish. We have conducted an empirical study to evaluate these algorithms on real data.
2113121
Computing capabilities of mediators
Existing data-integration systems based on the mediation architecture employ a variety of mechanisms to describe the query-processing capabilities of sources. However, these systems do not compute the capabilities of the mediators based on the capabilities of the sources they integrate. In this paper, we proposed a framework to capture a rich variety of query-processing capabilities of data sources and mediators. We present algorithms to compute the set of supported queries of a mediator, based on the capability limitations of its sources. Our algorithms take into consideration a variety of query-processing techniques employed by mediators to enhance the set of supported queries.
1386080
Using Constraints to Describe Source Contents in Data Integration Systems
In data integration systems, information sources often have constraints, such as "all houses stored at a source have a unique address." These constraints are useful in computing answers to queries and facilitating information exchange.
7967297
2D BubbleUp: Managing Parallel Disks for Media Servers
In this study we present a scheme called two-dimensional BubbleUp (2DB) for managing parallel disks in a multimedia server. Its goal is to reduce initial latency for interactive multimedia applications, while balancing disk loads to maintain high throughput. The 2DB scheme consists of a data placement and a request scheduling policy. The data placement policy replicates frequently accessed data and places them cyclically throughout the disks. The request scheduling policy attempts to maintain free ``service slots'''' in the immediate future. These slots can then be used to quickly service newly arrived requests. Through examples and simulation, we show that our scheme significantly reduces initial latency and maintains throughput comparable to that of the traditional schemes.
35008704
Image segmentation based on a multiresolution Bayesian framework
A multiresolution Markov random field (MMRF) approach to texture segmentation which makes use of both regional and boundary information is proposed in this work. The algorithm is designed to improve the efficiency and solve two problems: over-segmentation and under-segmentation. At each resolution level, regional information and boundary information between textured regions is extracted to describe each block using the multiresolution Fourier transform (MFT). The image is then modelled as a sequence of Markov random fields (MRFs) of varying resolution and a Gibbs sampling scheme is applied to label the constituent sites at each scale by seeking the configuration with minimal interaction energy. The interaction energy is defined as a function of the regional and boundary information. The constraints of region context, and boundary smoothness and connectivity are also encoded in the interaction energy. Once the algorithm converges at a given scale, the segmentation result is propagated down to the next resolution for further refinement.
8468902
Segmentation and recognition of multi-attribute motion sequences
In this work, we focus on fast and efficient recognition of motions in multi-attribute continuous motion sequences. 3D motion capture data, animation motion data, and sensor data from gesture sensing devices are examples of multi-attribute continuous motion sequences. These sequences have multiple attributes rather than only one attribute as time series data has. Motions can have different rates and durations, and the resulting data can thus have different lengths. Also, motion data can have noises due to transitions between successive motions. Hence, traditional distance measuring approaches used for time series data (such as Euclidean distances or dynamic time-warped distances) are not suitable for recognition in multi-attribute motion sequences. Hence, we have defined a similarity measure based on the analysis of singular value decomposition (SVD) properties of similar multi-attribute motions. A five-phase algorithm has then been proposed that gives good pruning power by exploiting the proximity of continuous motion data. We experimented this algorithm with data from different sources: 3D motion capture devices, animation motions, and CyberGlove gesture sensing device. These experiments show that our algorithm can segment and recognize long motion streams with high accuracy and in real time without knowing beforehand the number of motions in a stream.
15797731
A Novel Indexing Approach for Efficient and Fast Similarity Search of Captured Motions
Indexing of motion data is important for quickly searching similar motions for sign language recognition and gait analysis and rehabilitation. This paper proposes a simple and efficient tree structure for indexing motion data with dozens of attributes. Feature vectors are extracted for indexing by using singular value decomposition (SVD) properties of motion data matrices. By having similar motions with large variations indexed together, searching for similar motions of a query needs only one node traversal at each tree level, and only one feature needs to be considered at one tree level. Experiments show that the majority of irrelevant motions can be pruned while retrieving all similar motions, and one traversal of the indexing tree takes only several microseconds with the existence of motion variations.
4658514
Structured databases on the web: observations and implications
The Web has been rapidly "deepened" by the prevalence of databases online. With the potentially unlimited information hidden behind their query interfaces, this "deep Web" of searchable databses is clearly an important frontier for data access. This paper surveys this relatively unexplored frontier, measuring characteristics pertinent to both exploring and integrating structured Web sources. On one hand, our "macro" study surveys the deep Web at large, in April 2004, adopting the random IP-sampling approach, with one million samples. (How large is the deep Web? How is it covered by current directory services?) On the other hand, our "micro" study surveys source-specific characteristics over 441 sources in eight representative domains, in December 2002. (How "hidden" are deep-Web sources? How do search engines cover their data? How complex and expressive are query forms?) We report our observations and publish the resulting datasets to the research community. We conclude with several implications (of our own) which, while necessarily subjective, might help shape research directions and solutions.
3427735
RankSQL: Supporting Ranking Queries in Relational Database Management Systems
Ranking queries (or top-k queries) are dominant in many emerging applications, e.g., similarity queries in multimedia databases, searching Web databases, middleware, and data mining. The increasing importance of top-k queries warrants an efficient support of ranking in the relational database management system (RDBMS) and has recently gained the attention of the research community. Top-k queries aim at providing only the top k query results, according to a user-specified ranking function, which in many cases is an aggregate of multiple criteria. The following is an example top-k query.