11669811
Multi-level organization and summarization of the discovered rules
Many existing data mining techniques often p roduce a large number of rules, which make it very difficult for manual inspection o f the rules to identify those interesting ones. This problem represents a major gap between the results of data mining and the understanding and use of the mining results. In this paper, we a rgue that t he key problem is not with the large number of rules because if there are indeed many rules that exist in data, they should b e discovered. The main p roblem is with ou r inability to organize, summarize and present the rules in such a way that they can b e ea sily analyzed b y the user. In this paper, we propose a technique to intuitively organize a nd summarize the discovered rules. With this organization, the discovered rules can b e presented to the user in the way as we think and talk about knowledge in ou r daily lives. This organization also allows the user to view the discovered rules at different levels of details, and to focus his/her attention on those interesting aspects. This paper presents this technique a nd u ses it t o o rganize, summarize a nd present t he knowledge e mbedded in a decision tree, and a set of association rules. Experiment results and p ractical applications show that the technique is both intuitive and effective.
8786674
Entity discovery and assignment for opinion mining applications
Opinion mining became an important topic of study in recent years due to its wide range of applications. There are also many companies offering opinion mining services. One problem that has not been studied so far is the assignment of entities that have been talked about in each sentence. Let us use forum discussions about products as an example to make the problem concrete. In a typical discussion post, the author may give opinions on multiple products and also compare them. The issue is how to detect what products have been talked about in each sentence. If the sentence contains the product names, they need to be identified. We call this problem entity discovery. If the product names are not explicitly mentioned in the sentence but are implied due to the use of pronouns and language conventions, we need to infer the products. We call this problem entity assignment. These problems are important because without knowing what products each sentence talks about the opinion mined from the sentence is of little use. In this paper, we study these problems and propose two effective methods to solve the problems. Entity discovery is based on pattern discovery and entity assignment is based on mining of comparative sentences. Experimental results using a large number of forum posts demonstrate the effectiveness of the technique. Our system has also been successfully tested in a commercial setting.
61155076
Opinion Mining
Описанл процедуру формування інтегрального показника об’єкта згідно з відгуками користувачів під час застосування методів оцінювання опінії текстової інформації у web-документах. Запропоновано використання лінгвістичних змінних та застосування вагових коефіцієнтів для достовірнішого результату оцінювання емоційного забарвлення текстової інформації. Ключові слова: опінія, емоційне забарвлення, об’єкт, інтегральний показник, лінгвістична змінна, ваговий коефіцієнт.
5096508
Non-Almost-Derivable Frequent Itemsets Mining
The number of frequent itemsets is often too large to handle, so it is very necessary to work out a condensed representation of the collection of all frequent itemsets. In this paper, we propose a new condensed representation called frequent non-almost-derivable itemsets. This representation is a subset of the original collection of frequent itemsets. For any removed itemset X(which is called an frequent almost-derivable itemset), we can derive a lower and an upper bound of its support from this representation, and the lower bound and the upper bound is close enough(can be controlled by a userdefined parameter). We also propose an Apriori-like algorithm, which can extract all frequent nonderivable itemsets. Extensive empirical results on real datasets show the compactness and good approximation of this representation.
6193650
Dealing with different distributions in learning from
In the problem of learning with positive and unlabeled examples, existing research all assumes that positive examples P and the hidden positive examples in the unlabeled set U are generated from the same distribution. This assumption may be violated in practice. In such cases, existing methods perform poorly. This paper proposes a novel technique A-EM to deal with the problem. Experimental results with product page classification demonstrate the effectiveness of the proposed technique.
12442299
A holistic lexicon-based approach to opinion mining
One of the important types of information on the Web is the opinions expressed in the user generated content, e.g., customer reviews of products, forum posts, and blogs. In this paper, we focus on customer reviews of products. In particular, we study the problem of determining the semantic orientations (positive, negative or neutral) of opinions expressed on product features in reviews. This problem has many applications, e.g., opinion mining, summarization and search. Most existing techniques utilize a list of opinion (bearing) words (also called opinion lexicon) for the purpose. Opinion words are words that express desirable (e.g., great, amazing, etc.) or undesirable (e.g., bad, poor, etc) states. These approaches, however, all have some major shortcomings. In this paper, we propose a holistic lexicon-based approach to solving the problem by exploiting external evidences and linguistic conventions of natural language expressions. This approach allows the system to handle opinion words that are context dependent, which cause major difficulties for existing algorithms. It also deals with many special words, phrases and language constructs which have impacts on opinions based on their linguistic patterns. It also has an effective function for aggregating multiple conflicting opinion words in a sentence. A system, called Opinion Observer, based on the proposed technique has been implemented. Experimental results using a benchmark product review data set and some additional reviews show that the proposed technique is highly effective. It outperforms existing methods significantly
29971030
Measuring the meaning in time series clustering of text search queries
We use a combination of proven methods from time series analysis and machine learning to explore the relationship between temporal and semantic similarity in web query logs; we discover that the combination of correlation and cycles is a good, but not perfect, sign of semantic relationship.
5256099
Intrusion detection system for high-speed network
The increasing network throughput challenges the current Network Intrusion Detection Systems (NIDS) to have compatible high-performance data processing. In this paper, we describe an in-depth research on the related techniques of high-performance network intrusion detection and an implementation of a Rule-based High-performance Network Intrusion Detection System (RHPNIDS) for high-speed networks. By integrating several performance optimizing methods, the performance of RHPNIDS is very impressive compared with the popular open source NIDS Snort.
3413319
Filtering Spam in Social Tagging System with Dynamic Behavior Analysis
Spam in social tagging systems introduced by some malicious participants has become a serious problem for its global popularizing. Some studies which can be deduced to static user data analysis have been presented to combat tag spam, but either they do not give an exact evaluation or the algorithms’ performances are not good enough. In this paper, we proposed a novel method based on analysis of dynamic user behavior data for the notion that users’ behaviors in social tagging system can reflect the quality of tags more accurately. Through modeling the different categories of participants’ behaviors, we extract tag-associated actions which can be used to estimate whether tag is spam, and then present our algorithm that can filter the tag spam in the results of social search. The experiment results show that our method indeed outperforms the existing methods based on static data and effectively defends against the tag spam in various spam attacks.
32302868
Non-linear Correlation Techniques in Educational Data Mining
There is such an increasing interest in data mining and educational systems currently that has made educational data mining as a new growing research community. This paper explores how to develope new methods for discovering knowledge of data from educational context. The non-linear correlation technology was introduced and applied in the mining process in the whole knowledge achieved. Meanwhile, we have applied these methods in the real course management datasets and found correspondent results for the educators.
17725384
BitTube: Case Study of a Web-Based Peer-Assisted Video-on-Demand System
Recent theoretical and simulation-based studies have confirmed the tremendous benefit of peer-to-peer (P2P)communication at reducing the cost of running a VoD service.To date, very limited effort has been paid to validate the concept of peer-assisted VoD service, especially in terms of system implementation and service deployment. In this paper, we present the case study of a peer-assisted video-on-demand (VoD) system. We designed and developed Bit-Tube, a BitTorrent-compliant VoD system. By combining client/server and P2P downloading, it supports seamless transition across the spectrum from pure client-server mode to BitTorrent mode. Within this framework, we experiment with a series of piece picking policies to enhance BitTubepsilas support to video streaming and promote locality-aware P2P downloading. We evaluate our system over PlanetLab,which hosts the user-side component of the BitTube system and emulates the global-scale user requests to the VoD service.