15810340
Discovering the set of fundamental rule changes
The world around us changes constantly. Knowing what has changed is an important part of our lives. For businesses, recognizing changes is also crucial. It allows businesses to adapt themselves to the changing market needs. In this paper, we study changes of association rules from one time period to another. One approach is to compare the supports and/or confidences of each rule in the two time periods and report the differences. This technique, however, is too simplistic as it tends to report a huge number of rule changes, and many of them are, in fact, simply the snowball effect of a small subset of fundamental changes. Here, we present a technique to highlight the small subset of fundamental changes. A change is fundamental if it cannot be explained by some other changes. The proposed technique has been applied to a number of real-life datasets. Experiments results show that the number of rules whose changes are unexplainable is quite small (about 20% of the total number of changes discovered), and many of these unexplainable changes reflect some fundamental shifts in the application domain.
5096508
Non-Almost-Derivable Frequent Itemsets Mining
The number of frequent itemsets is often too large to handle, so it is very necessary to work out a condensed representation of the collection of all frequent itemsets. In this paper, we propose a new condensed representation called frequent non-almost-derivable itemsets. This representation is a subset of the original collection of frequent itemsets. For any removed itemset X(which is called an frequent almost-derivable itemset), we can derive a lower and an upper bound of its support from this representation, and the lower bound and the upper bound is close enough(can be controlled by a userdefined parameter). We also propose an Apriori-like algorithm, which can extract all frequent nonderivable itemsets. Extensive empirical results on real datasets show the compactness and good approximation of this representation.
12241191
Finding Actionable Knowledge via Automated Comparison
The problem of finding interesting and actionable patterns is a major challenge in data mining. It has been studied by many data mining researchers. The issue is that data mining algorithms often generate too many patterns, which make it very hard for the user to find those truly useful ones. Over the years many techniques have been proposed. However, few have made it to real-life applications. At the end of 2005, we built a data mining system for Motorola (called Opportunity Map) to enable the user to explore the space of a large number of rules in order to find actionable knowledge. The approach is based on the concept of rule cubes and operations on rule cubes. A rule cube is similar to a data cube, but stores rules. Since its deployment, some issues have also been identified during the regular use of the system in Motorola. One of the key issues is that although the operations on rule cubes are flexible, each operation is primitive and has to be initiated by the user. Finding a piece of actionable knowledge typically involves many operations and intense visual inspections, which are labor-intensive and time-consuming. From interactions with our users, we identified a generic problem that is crucial for finding actionable knowledge. The problem involves extensive comparison of sub-populations and identification of the cause of their differences. This paper first defines the problem and then proposes an effective method to solve the problem automatically. To the best of our knowledge, there is no reported study of this problem. The new method has been added to the Opportunity Map system and is now in daily use in Motorola.
3219406
Opinion spam and analysis
Evaluative texts on the Web have become a valuable source of opinions on products, services, events, individuals, etc. Recently, many researchers have studied such opinion sources as product reviews, forum posts, and blogs. However, existing research has been focused on classification and summarization of opinions using natural language processing and data mining techniques. An important issue that has been neglected so far is opinion spam or trustworthiness of online opinions. In this paper, we study this issue in the context of product reviews, which are opinion rich and are widely used by consumers and product manufacturers. In the past two years, several startup companies also appeared which aggregate opinions from product reviews. It is thus high time to study spam in reviews. To the best of our knowledge, there is still no published study on this topic, although Web spam and email spam have been investigated extensively. We will see that opinion spam is quite different from Web spam and email spam, and thus requires different detection techniques. Based on the analysis of 5.8 million reviews and 2.14 million reviewers from amazon.com, we show that opinion spam in reviews is widespread. This paper analyzes such spam activities and presents some novel techniques to detect them
11669811
Multi-level organization and summarization of the discovered rules
Many existing data mining techniques often p roduce a large number of rules, which make it very difficult for manual inspection o f the rules to identify those interesting ones. This problem represents a major gap between the results of data mining and the understanding and use of the mining results. In this paper, we a rgue that t he key problem is not with the large number of rules because if there are indeed many rules that exist in data, they should b e discovered. The main p roblem is with ou r inability to organize, summarize and present the rules in such a way that they can b e ea sily analyzed b y the user. In this paper, we propose a technique to intuitively organize a nd summarize the discovered rules. With this organization, the discovered rules can b e presented to the user in the way as we think and talk about knowledge in ou r daily lives. This organization also allows the user to view the discovered rules at different levels of details, and to focus his/her attention on those interesting aspects. This paper presents this technique a nd u ses it t o o rganize, summarize a nd present t he knowledge e mbedded in a decision tree, and a set of association rules. Experiment results and p ractical applications show that the technique is both intuitive and effective.
2182731
Speed-up iterative frequent itemset mining with constraint changes
Mining of frequent itemsets is a fundamental data mining task. Past research has proposed many efficient algorithms for this purpose. Recent work also highlighted the importance of using constraints to focus the mining process to mine only those relevant itemsets. In practice, data mining is often an interactive and iterative process. The user typically changes constraints and runs the mining algorithm many times before being satisfied with the final results. This interactive process is very time consuming. Existing mining algorithms are unable to take advantage of this iterative process to use previous mining results to speed up the current mining process. This results in an enormous waste of time and computation. In this paper, we propose an efficient technique to utilize previous mining results to improve the efficiency of current mining when constraints are changed. We first introduce the concept of tree boundary to summarize useful information available from previous mining. We then show that the tree boundary provides an effective and efficient framework for the new mining. The proposed technique has been implemented in the context of two existing frequent itemset mining algorithms, FP-tree and tree projection. Experiment results on both synthetic and real-life datasets show that the proposed approach achieves a dramatic saving of computation.
13551397
A Blind Watermarking of 3D Triangular Meshes Using Geometry Image
26305172
Hybrid Particle Swarm Optimization for Flow Shop Scheduling with Stochastic Processing Time
The stochastic flow shop scheduling with uncertain processing time is a typical NP-hard combinatorial optimization problem and represents an important area in production scheduling, which is difficult because of inaccurate objective estimation, huge search space, and multiple local minima. As a novel evolutionary technique, particle swarm optimization (PSO) has gained much attention and wide applications for both function and combinatorial problems, but there is no research on PSO for stochastic scheduling cases. In this paper, a class of PSO approach with simulated annealing (SA) and hypothesis test (HT), namely PSOSAHT is proposed for stochastic flow shop scheduling with uncertain processing time with respect to the makespan criterion (i.e. minimizing the maximum completion time). Simulation results demonstrate the feasibility, effectiveness and robustness of the proposed hybrid algorithm. Meanwhile, the effects of noise magnitude and number of evaluation on searching performances are also investigated.
2750727
Stability Analysis of Social Foraging Swarm with Interaction Time Delays
This paper considers a swarm model with an attraction-repulsion function involving variable communication time lags and an attractant/repellent. It is proved that for quadratic attractant/repellent profiles the members of the swarm with time delays will aggregate and form a cohesive cluster of finite size in a finite time. Moreover, all the swarm members will converge to more favorable areas of the quadratic attractant/repellent profiles under certain conditions in the presence of communication delays.
15435032
Characterizing the dynamic connectivity between genes by variable parameter regression and Kalman filtering based on temporal gene expression data
MOTIVATION
One popular method for analyzing functional connectivity between genes is to cluster genes with similar expression profiles. The most popular metrics measuring the similarity (or dissimilarity) among genes include Pearson's correlation, linear regression coefficient and Euclidean distance. As these metrics only give some constant values, they can only depict a stationary connectivity between genes. However, the functional connectivity between genes usually changes with time. Here, we introduce a novel insight for characterizing the relationship between genes and find out a proper mathematical model, variable parameter regression and Kalman filtering to model it.


RESULTS
We applied our algorithm to some simulated data and two pairs of real gene expression data. The changes of connectivity in simulated data are closely identical with the truth and the results of two pairs of gene expression data show that our method has successfully demonstrated the dynamic connectivity between genes.


CONTACT
jiangtz@nlpr.ia.ac.cn.
39231279
An Intelligent Differential Evolution Algorithm for Designing Trading-Ratio System of Water Market
As a novel optimization technique, neural network based optimization has gained much attention and some applications during the past decade. To enhance the performance of Differential Evolution Algorithm (DEA), which is an evolutionary computation technique through individual improvement plus population cooperation and competition, an intelligent Differential Evolution Algorithm (IDEA) is proposed by incorporating neural network based search behaviors into classic DEA. Firstly, DEA operators are used for exploration by updating individuals so as to maintain the diversity of population and speedup the search process. Secondly, a multi-layer feed-forward neural network is employed for local exploitation to avoid being trapped in local optima and improve the convergence of the IDEA. Simulation results and comparisons based on well-known benchmarks and optimal designing of trading-ratio system for water market demonstrate that the IDEA can effectively enhance the searching efficiency and greatly improve the searching quality.