17022495
Improving Web clustering by cluster selection
Web page clustering is a technology that puts semantically related Web pages into groups and is useful for categorizing, organizing, and refining search results. When clustering using only textual information, suffix tree clustering (STC) outperforms other clustering algorithms by making use of phrases and allowing clusters to overlap. One problem of STC and other similar algorithms is how to select a small set of clusters to display to the user from a very large set of generated clusters. The cluster selection method used in STC is flawed in that it does not handle overlapping clusters appropriately. This paper introduces a new cluster scoring function and a new cluster selection algorithm to overcome the problems with overlapping clusters, which are combined with STC to make a new clustering algorithm ESTC. This paper's experiments show that ESTC significantly outperforms STC and that even with less data ESTC performs similarly to a commercial clustering search engine.
1036423
Investigation of Brood Size in GP with Brood Recombination Crossover for Object Recognition
This paper describes an approach to the investigation of brood size in the brood recombination crossover method in genetic programming for object recognition problems. The approach is examined and compared with the standard crossover operator on three object classification problems of increasing difficulty. The results suggest that the brood recombination method outperforms the standard crossover operator for all the problems in terms of the classification accuracy. As the brood size increases, the system effective performance can be improved. When it exceeds a certain point, however, the effective performance will not be improved and the system will become less efficient.
1168192
Approximately Repetitive Structure Detection for Wrapper Induction
In recent years, much work has been invested into automatically learning wrappers for information extraction from HTML tables and lists. Our research has focused on a system that can learn a wrapper from a single unlabelled page. An essential step is to locate the tabular data within the page. This is not trivial when the structures of data tuples are similar but not identical. In this paper we describe an algorithm that can automatically detect approximate repetitive structures within one sequence. The algorithm does not rely on any domain knowledge or HTML heuristics and it can be used in detecting repetitive patterns and hence to learn wrappers from a single unlabeled tabular page.
40240913
Combining Contents and Citations for Scientific Document Classification
This paper introduces a classification system that exploits the content information as well as citation structure for scientific paper classification. The system first applies a content-based statistical classification method which is similar to general text classification. We investigate several classification methods including K-nearest neighbours, nearest centroid, naive Bayes and decision trees. Among those methods, the K-nearest neighbours is found to outperform others while the rest perform comparably. Using phrases in addition to words and a good feature selection strategy such as information gain can improve system accuracy and reduce training time in comparison with using words only. To combine citation links for classification, the system proposes an iterative method to update the labellings of classified instances using citation links. Our results show that, combining contents and citations significantly improves the system performance.
39682007
TreeWrapper: Automatic Data Extraction Based on Tree Representation
This paper introduces a new algorithm that learns to extract data from Web pages with relatively regular data structures. Current existing systems require training on either manually labelled pages or at least two similar unlabelled pages, and they often have difficulties on handling Web pages with complex formats such as nested tables or lists. Our previous system AutoWrapper does not need any training and can automatically extract data from any single page. This paper improves AutoWrapper by handling nested structures and finding multiple regular data areas. The main contributions include a tree-based representation for Web pages, an expressive language for representing information extraction patterns, and a learning algorithm that automatically detects regular data areas by finding similar sub-trees.
40362881
A Knowledge Learning Approach to Information Extraction from Multiple Text Based Web Sites
This paper describes a learning/adaptive approach to automatically building knowledge bases for information extraction from text based web pages. A frame based representation is introduced to represent domain knowledge as knowledge unit frames. A frame learning algorithm is developed to automatically learn knowledge unit frames from training examples. Some training examples can be obtained by automatically parsing a number of tabular web pages in the same domain, which greatly reduced the amount of time consuming manual work. This approach was investigated on ten web sites of real estate advertisements and car advertisements and nearly all the information was successfully extracted with very few false alarms. These results suggest that both the knowledge unit frame representation and the frame learning algorithm work well, domain specific knowledge bases can be learned from training examples, and the domain specific knowledge base can be used for information extraction from flexible text-based semi-structured Web pages on multiple Web sites. The investigation of the knowledge representation on five other domains suggests that this approach can be easily applied to other domains by simply changing the training examples.
13249479
Fair and efficient router congestion control
Congestion is a natural phenomenon in any network queuing system, and is unavoidable if the queuing system is operated near capacity. In this paper we study how to set the rules of a queuing system so that all the users have a self-interest in controlling congestion when it happens.Routers in the internet respond to local congestion by dropping packets. But if packets are dropped indiscriminately, the effect can be to encourage senders to actually increase their transmission rates, worsening the congestion and destabilizing the system. Alternatively, and only slightly more preferably, the effect can be to arbitrarily let a few insistent senders take over most of the router capacity.We approach this problem from first principles: a router packet-dropping protocol is a mechanism that sets up a game between the senders, who are in turn competing for link capacity. Our task is to design this mechanism so that the game equilibrium is desirable: high total rate is achieved and is shared widely among all senders. In addition, equilibrium should be reestablished quickly in response to changes in transmission rates. Our solution is based upon auction theory: in principle, although not always in practice, we drop packets of the highest-rate sender, in case of congestion. We will prove the game-theoretic merits of our method. We'll also describe a variant of the method with some further advantages that will be supported by network simulations.
31240676
A Novel Clustering Algorithm Based on Immune Network with Limited Resource
20577453
A Novel Clustering Method Based on SVM
For the problem of cluster analysis, the objective function based algorithms are popular and widely used methods. However, the performance of these algorithms depends upon the priori information about cluster number and cluster prototypes. Moreover, it is only effective for analyzing data set with the same type of cluster prototypes. For this end, this paper presents a novel algorithm based on support vector machine (SVM) for realizing fully unsupervised clustering. The experimental results with various test data sets illustrate the effectiveness of the proposed novel clustering algorithm based on SVM.
206603960
FCM-Based Clustering Algorithm Ensemble for Large Data Sets
In the field of cluster analysis, most of the available algorithms were designed for small data sets, which cannot efficiently deal with large scale data set encountered in data mining. However, some sampling-based clustering algorithms for large scale data set cannot achieve ideal result. For this purpose, a FCM-based clustering ensemble algorithm is proposed. Firstly, it performs the atom clustering algorithm on the large data set. Then, randomly select a sample from each atom as representative to reduce the data amount. And the ensemble learning technique is used to improve the clustering performance. For the complex large data sets, the new algorithm has high classification speed and robustness. The experimental results illustrate the effectiveness of the proposed clustering algorithm.
30258061
A Speculative Strategy
In this contribution we propose a strategy which focuses on the game as well as on the opponent. Preference is given to the thoughts of the opponent, so that the strategy might be speculative. We describe a generalization of OM search, called (D, d)-OM search, where D stands for the depth of search by the player and d for the opponent's depth of search. A difference in search depth can be exploited by deliberately chosing a suboptimal move in order to gain a larger advantage than when playing the optimal best move. The idea is that the opponent does not see the variant in sufficiently deep detail. Simulations using a game-tree model including an opponent model as well as experiments in the domain of Othello confirm the effectiveness of the proposed strategy.