14110256
Query Directed Web Page Clustering
Web page clustering methods categorize and organize search results into semantically meaningful clusters that assist users with search refinement; but finding clusters that are semantically meaningful to users is difficult. In this paper, we describe a new Web page clustering algorithm, QDC, which uses the user's query as part of a reliable measure of cluster quality. The new algorithm has five key innovations: a new query directed cluster quality guide that uses the relationship between clusters and the query, an improved cluster merging method that generates semantically coherent clusters by using cluster description similarity in additional to cluster overlap, a new cluster splitting method that fixes the cluster chaining or cluster drifting problem, an improved heuristic for cluster selection that uses the query directed cluster quality guide, and a new method of improving clusters by ranking the pages by relevance to the cluster. We evaluate QDC by comparing its clustering performance against that of four other algorithms on eight data sets (four use full text data and four use snippet data) by using eleven different external evaluation measurements. We also evaluate QDC by informally analysing its real world usability and performance through comparison with six other algorithms on four data sets. QDC provides a substantial performance improvement over other Web page clustering algorithms
17022495
Improving Web clustering by cluster selection
Web page clustering is a technology that puts semantically related Web pages into groups and is useful for categorizing, organizing, and refining search results. When clustering using only textual information, suffix tree clustering (STC) outperforms other clustering algorithms by making use of phrases and allowing clusters to overlap. One problem of STC and other similar algorithms is how to select a small set of clusters to display to the user from a very large set of generated clusters. The cluster selection method used in STC is flawed in that it does not handle overlapping clusters appropriately. This paper introduces a new cluster scoring function and a new cluster selection algorithm to overcome the problems with overlapping clusters, which are combined with STC to make a new clustering algorithm ESTC. This paper's experiments show that ESTC significantly outperforms STC and that even with less data ESTC performs similarly to a commercial clustering search engine.
40362881
A Knowledge Learning Approach to Information Extraction from Multiple Text Based Web Sites
This paper describes a learning/adaptive approach to automatically building knowledge bases for information extraction from text based web pages. A frame based representation is introduced to represent domain knowledge as knowledge unit frames. A frame learning algorithm is developed to automatically learn knowledge unit frames from training examples. Some training examples can be obtained by automatically parsing a number of tabular web pages in the same domain, which greatly reduced the amount of time consuming manual work. This approach was investigated on ten web sites of real estate advertisements and car advertisements and nearly all the information was successfully extracted with very few false alarms. These results suggest that both the knowledge unit frame representation and the frame learning algorithm work well, domain specific knowledge bases can be learned from training examples, and the domain specific knowledge base can be used for information extraction from flexible text-based semi-structured Web pages on multiple Web sites. The investigation of the knowledge representation on five other domains suggests that this approach can be easily applied to other domains by simply changing the training examples.
40240913
Combining Contents and Citations for Scientific Document Classification
This paper introduces a classification system that exploits the content information as well as citation structure for scientific paper classification. The system first applies a content-based statistical classification method which is similar to general text classification. We investigate several classification methods including K-nearest neighbours, nearest centroid, naive Bayes and decision trees. Among those methods, the K-nearest neighbours is found to outperform others while the rest perform comparably. Using phrases in addition to words and a good feature selection strategy such as information gain can improve system accuracy and reduce training time in comparison with using words only. To combine citation links for classification, the system proposes an iterative method to update the labellings of classified instances using citation links. Our results show that, combining contents and citations significantly improves the system performance.
39682007
TreeWrapper: Automatic Data Extraction Based on Tree Representation
This paper introduces a new algorithm that learns to extract data from Web pages with relatively regular data structures. Current existing systems require training on either manually labelled pages or at least two similar unlabelled pages, and they often have difficulties on handling Web pages with complex formats such as nested tables or lists. Our previous system AutoWrapper does not need any training and can automatically extract data from any single page. This paper improves AutoWrapper by handling nested structures and finding multiple regular data areas. The main contributions include a tree-based representation for Web pages, an expressive language for representing information extraction patterns, and a learning algorithm that automatically detects regular data areas by finding similar sub-trees.
1168192
Approximately Repetitive Structure Detection for Wrapper Induction
In recent years, much work has been invested into automatically learning wrappers for information extraction from HTML tables and lists. Our research has focused on a system that can learn a wrapper from a single unlabelled page. An essential step is to locate the tabular data within the page. This is not trivial when the structures of data tuples are similar but not identical. In this paper we describe an algorithm that can automatically detect approximate repetitive structures within one sequence. The algorithm does not rely on any domain knowledge or HTML heuristics and it can be used in detecting repetitive patterns and hence to learn wrappers from a single unlabeled tabular page.
31240676
A Novel Clustering Algorithm Based on Immune Network with Limited Resource
43629891
A Content-Based Image Quality Metric
image quality assessment plays an important role in relevant fields of image processing. The traditional image quality metric, such as PSNR, cannot reflect the visual perception to the image effectively. For this purpose, based on the fuzzy Sugeno integral a novel image quality assessment measure, called content-based metric (CBM), is proposed in this paper. It fuses the amount and local information into the similarity of the image structural information and gives a comprehensive evaluation for the quality of the specified image. The experimental results illustrate that the proposed metric has a good correlation with the human subjective perception, and can reflect the image quality effectively.
20577453
A Novel Clustering Method Based on SVM
For the problem of cluster analysis, the objective function based algorithms are popular and widely used methods. However, the performance of these algorithms depends upon the priori information about cluster number and cluster prototypes. Moreover, it is only effective for analyzing data set with the same type of cluster prototypes. For this end, this paper presents a novel algorithm based on support vector machine (SVM) for realizing fully unsupervised clustering. The experimental results with various test data sets illustrate the effectiveness of the proposed novel clustering algorithm based on SVM.
19644682
A New Cluster Validity Function Based on the Modified Partition Fuzzy Degree
The cluster validity is an important topic of cluster analysis, which is often converted into the determination of the optimal cluster number. Most of the available cluster validity functions are limited for the analysis of numeric data set and ineffective for the categorical data set. For this purpose, a new cluster validity function is presented in this paper, namely the modified partition fuzzy degree. By combining the partition entropy and the partition fuzzy degree, the new cluster validity can be applied to any data set with numeric attributes or categorical attributes. The experimental results illustrate the effectiveness of the proposed cluster validity function.
1210426
De-interlacing Algorithm Based on Motion Objects
A novel de-interlacing algorithm based on motion objects is presented in this paper. In this algorithm, natural motion objects, not contrived blocks, are considered as the processing cells, which are accurately detected by a new scheme, and whose matching objects are quickly searched by the immune clonal selection algorithm. This novel algorithm integrates many other de-interlacing methods, so it is more adaptive to various complex video sequences. Moreover, it can perform the motion compensation for objects with the translation, rotation as well as the scaling transform. The experimental results illustrate that compared with the block matching method with full search, the proposed algorithm greatly improve the efficiency and performance.