6794000
Performance Issues in Incremental Warehouse Maintenance
A well-known challenge in data warehousing is the efficient incremental maintenance of warehouse data in the presence of source data updates. In this paper, we identify several critical data representation and algorithmic choices that must be made when developing the machinery of an incrementally maintained data warehouse. For each decision area, we identify various alternatives and evaluate them through extensive experiments. We show that picking the right alternative can lead to dramatic performance gains, and we propose guidelines for making the right decisions under different scenarios. All of the issues addressed in this paper arose in our development of WHIPS, a prototype data warehousing system supporting incremental maintenance.
12872505
Value-Based Notification Conditions in Large-Scale Publish/Subscribe Systems
We address the problem of providing scalable support for subscriptions with personalized value-based notification conditions in wide-area publish/subscribe systems. Notification conditions can be fine-tuned by subscribers, allowing precise and flexible control of when events are delivered to the subscribers. For example, a user may specify that she should be notified if and only if the price of a particular stock moves outside a "radius" around her last notified value. Naive techniques for handling notification conditions are not scalable. It is challenging to share subscription processing and notification dissemination of subscriptions with personalized value-based notification conditions, because two subscriptions may see two completely different sequences of notifications even if they specify the same radius. We develop and experimentally evaluate scalable processing and dissemination techniques for these subscriptions. Our approach uses standard network substrates for notification dissemination, and avoids pushing complex application processing into the network. Compared with other alternatives, our approach generates orders of magnitude lower network traffic, and incurs lower server processing cost.
2070842
Distributed Network Querying with Bounded Approximate Caching
As networks continue to grow in size and complexity, distributed network monitoring and resource querying are becoming increasingly difficult. Our aim is to design, build, and evaluate a scalable infrastructure for answering queries over distributed measurements, at reduced costs (in terms of both network traffic and query latency) while maintaining required precision. In this infrastructure, each network node owns a set of numerical measurements and actively maintains bounds on these values cached at other nodes. We can answer queries approximately, using bounds from nearby caches to avoid contacting the owners directly. We focus on developing efficient and scalable techniques to place, locate, and manage bounded approximate caches across a large network. We have developed two approaches: One uses a recursive partitioning of the network space to place caches in a static, controlled manner, while the other uses a locality-aware distributed hash table to place caches in a dynamic and decentralized manner. In this paper, we focus on the latter approach. Experiments over a large-scale emulated network show that our techniques are very effective in reducing query costs while generating an acceptable amount of background traffic; they are also able to exploit various forms of locality that are naturally present in queries, and adapt to volatility of measurements.
7182462
On the database/network interface in large-scale publish/subscribe systems
The work performed by a publish/subscribe system can conceptually be divided into subscription processing and notification dissemination. Traditionally, research in the database and networking communities has focused on these aspects in isolation. The interface between the database server and the network is often overlooked by previous research. At one extreme, database servers are directly responsible for notifying individual subscribers; at the other extreme, updates are injected directly into the network, and the network is solely responsible for processing subscriptions and forwarding notifications. These extremes are unsuitable for complex and stateful subscription queries. A primary goal of this paper is to explore the design space between the two extremes, and to devise solutions that incorporate both database-side and network-side considerations in order to reduce the communication and server load and maintain system scalability. Our techniques apply to a broad range of stateful query types, and we present solutions for several of them. Our detailed experiments based on real and synthetic workloads with varying characteristics and link-level network simulation show that by exploiting the query semantics and building an appropriate interface between the database and the network, it is possible to achieve orders-of-magnitude savings in network traffic at low server-side processing cost.
2510058
Constraint chaining: on energy-efficient continuous monitoring in sensor networks
Wireless sensor networks have created new opportunities for data collection in a variety of scenarios, such as environmental and industrial, where we expect data to be temporally and spatially correlated. Researchers may want to continuously collect all sensor data from the network for later analysis. Suppression, both temporal and spatial, provides opportunities for reducing the energy cost of sensor data collection. We demonstrate how both types can be combined for maximal benefit. We frame the problem as one of monitoring node and edge constraints. A monitored node triggers a report if its value changes. A monitored edge triggers a report if the difference between its nodes' values changes. The set of reports collected at the base station is used to derive all node values. We fully exploit the potential of this global inference in our algorithm, CONCH, short for constraint chaining. Constraint chaining builds a network of constraints that are maintained locally, but allow a global view of values to be maintained with minimal cost. Network failure complicates the use of suppression, since either causes an absence of reports. We add enhancements to CONCH to build in redundant constraints and provide a method to interpret the resulting reports in case of uncertainty. Using simulation we experimentally evaluate CONCH's effectiveness against competing schemes in a number of interesting scenarios.
14448318
On Suspending and Resuming Dataflows
Consider a long-running, resource-intensive query Q running on a database management system (DBMS). Suppose another task T with much higher priority arrives, and we need to process T as quickly as possible and with all available resources. Ideally, the system should suspend the execution of Q, quickly release all resources held by Q, and start T using all resources. Query Q can be resumed once T finishes execution, ideally without losing any significant fraction of the work that Q had done prior to suspend. Beyond supporting mixed-priority DBMS workloads, suspend/resume of queries, or dataflows in general, is important in many other settings: Utility and grid: Dataflows now run frequently on computational utilities and grids composed of autonomous resources. When the owner of resources wants to use them, dataflows running on these resources must release control quickly, and migrate to other resources. Software rejuvenation: Benefits of software rejuvenation, the practice of rebooting enterprise computing systems regularly, are now recognized widely. Reboot is critical when performance degrades due to resource exhaustion caused by resource leaks. Suspend and resume is important in this setting because (1) the challenge of predicting completion times accurately makes it difficult to schedule task completion to match a rejuvenation schedule; (2) when performance degrades, it may not be cost-effective to wait for all dataflows to complete before rebooting.
8185115
On the combination and normalization of interval-valued belief structures
This paper investigates the issues of combination and normalization of interval-valued belief structures within the framework of Dempster-Shafer theory (DST) of evidence. Existing approaches are reviewed, examined and critically analysed. They either ignore the normalization or separate it from the combination process, leading to irrational or suboptimal interval-valued belief structures. A new logically correct optimality approach is developed, where the combination and the normalization are optimised together rather than separately. Numerical examples are provided throughout the paper.
5502812
A Continuous Genetic Algorithm for Global Optimization
16976457
Fuzzy linear programming technique for multiattribute group decision making in fuzzy environments
The aim of this paper is to develop a linear programming technique for multidimensional analysis of preferences in multiattribute group decision making under fuzzy environments. Fuzziness is inherent in decision data and group decision making processes, and linguistic variables are well suited to assessing an alternative on qualitative attributes using fuzzy ratings. A crisp decision matrix can be converted into a fuzzy decision matrix once the decision makers' fuzzy ratings have been extracted. In this paper, we first define group consistency and inconsistency indices based on preferences to alternatives given by decision makers and construct a linear programming decision model based on the distance of each alternative to a fuzzy positive ideal solution which is unknown. Then the fuzzy positive ideal solution and the weights of attributes are estimated using the new decision model based on the group consistency and inconsistency indices. Finally, the distance of each alternative to the fuzzy positive ideal solution is calculated to determine the ranking order of all alternatives. A numerical example is examined to demonstrate the implementation process of the technique.
5715843
A Cautionary Tale
Recently we have been investigating various methods for improving the retrieval of documents. One frequently used context for such studies is to work with a known database and set of queries, where sufficient work has previously been done to establish an "official" set of relevant documents for each query. Working within such a context, we have discovered at least one instance wherein the combination of the document descriptions in the database, the query, and the set of documents deemed relevant foredooms our attempts at retrieval improvement to failure. This note is to caution other researchers against such possibilities, and to recommend that the database in question, that of the National Physical Laboratory (NPL), as represented on Virginia Disk 1, be retired from use.
6709166
Harmonium Models for Semantic Video Representation and Classification
Accurate and efficient video classification demands the fusion of multimodal information and the use of intermediate representations. Combining the two ideas into the same framework, we propose a probabilistic approach for video classification using intermediate semantic representations derived from the multi-modal features. Based on a class of bipartite undirected graphical models named harmonium, our approach represents video data as latent semantic topics derived by jointly modeling the transcript keywords and color-histogram features, and perform classification using these latent topics under a unified framework. We show satisfactory classification performance of our approach on a benchmark dataset, and some interesting insights of the data provided by this approach.