25781895
Web page clustering enhanced by summarization
Traditional Web page clustering algorithms use the full-text in the documents to generate feature vectors. Such methods often produce unsatisfactory results because there is much noisy information, such as decoration, interaction, and advertisement, in Web pages. The varying-length problem of the Web pages is also a significant negative factor affecting the performance. In this paper, we investigate the use of several summarization techniques to tackle these issues when clustering Web pages. Compared with the full-text representation of the Web pages, our experimental results indicate that our proposed approach effectively solves the problems of noisy information and varying-length, and thus significantly boosts the clustering performance.
14828051
MRSSA: an iterative algorithm for similarity spreading over interrelated objects
We introduce the Multiple Relationship Similarity Spreading Algorithm (MRSSA) to enhance IR effectiveness. This method has similarity computed in an iterative "spreading" fashion for multiple object types, combining both inter- and intra-object relationships. We demonstrate the value of this approach in the context of the WWW, where the key objects are web pages and queries, Relationships considered are derived from hyperlinks (in- and out-links) and click-through logs.
432736
ReCoM: reinforcement clustering of multi-type interrelated data objects
Most existing clustering algorithms cluster highly related data objects such as Web pages and Web users separately. The interrelation among different types of data objects is either not considered, or represented by a static feature space and treated in the same ways as other attributes of the objects. In this paper, we propose a novel clustering approach for clustering multi-type interrelated data objects, ReCoM (Reinforcement Clustering of Multi-type Interrelated data objects). Under this approach, relationships among data objects are used to improve the cluster quality of interrelated data objects through an iterative reinforcement clustering process. At the same time, the link structure derived from relationships of the interrelated data objects is used to differentiate the importance of objects and the learned importance is also used in the clustering process to further improve the clustering results. Experimental results show that the proposed approach not only effectively overcomes the problem of data sparseness caused by the high dimensional relationship space but also significantly improves the clustering accuracy.
6802371
Improving web search results using affinity graph
In this paper, we propose a novel ranking scheme named Affinity Ranking (AR) to re-rank search results by optimizing two metrics: (1) diversity -- which indicates the variance of topics in a group of documents; (2) information richness -- which measures the coverage of a single document to its topic. Both of the two metrics are calculated from a directed link graph named Affinity Graph (AG). AG models the structure of a group of documents based on the asymmetric content similarities between each pair of documents. Experimental results in Yahoo! Directory, ODP Data, and Newsgroup data demonstrate that our proposed ranking algorithm significantly improves the search performance. Specifically, the algorithm achieves 31% improvement in diversity and 12% improvement in information richness relatively within the top 10 search results.
459986
A New Statistical Approach To Chinese Pinyin Input
Chinese input is one of the key challenges for Chinese PC users. This paper proposes a statistical approach to Pinyin-based Chinese input. This approach uses a trigram-based language model and a statistically based segmentation. Also, to deal with real input, it also includes a typing model which enables spelling correction in sentence-based Pinyin input, and a spelling model for English which enables modeless Pinyin input.
17711119
IRC: an iterative reinforcement categorization algorithm for interrelated Web objects
Most existing categorization algorithms deal with homogeneous Web data objects, and consider interrelated objects as additional features when taking the interrelationships with other types of objects into account. However, focusing on any single aspects of these interrelationships and objects does not fully reveal their true categories. In this paper, we propose a categorization algorithm, the iterative reinforcement categorization algorithm (IRC), to exploit the full interrelationships between the heterogeneous objects on the Web. IRC attempts to classify the interrelated Web objects by iterative reinforcement between individual classification results of different types via the interrelationships. Experiments on a clickthrough log dataset from MSN search engine show that, with the Fl measures, IRC achieves a 26.4% improvement over a pure content-based classification method, a 21% improvement over a query metadata-based method, and a 16.4% improvement over a virtual document-based method. Furthermore, our experiments show that IRC converges rapidly.
17921413
A self-learning worm using importance scanning
The use of side information by an attacker can help a worm speed up the propagation. This philosophy has been the basis for advanced worm scanning mechanisms such as hitlist scanning, routable scanning, and importance scanning. Some of these scanning methods use information on vulnerable hosts. Such information, however, may not be easy to collect before a worm is released. Questions then arise whether and how a worm can self-learn and use such information while propagating, and how virulent the resulting worm may be. In this paper, we design a self-learning worm using importance scanning. An optimal yet practical importance-scanning strategy is derived based on a new metric. A self-learning worm is demonstrated to have the ability to accurately estimate the underlying vulnerable-host distribution if a sufficient number of infected hosts are observed. Experimental results based on parameters chosen from Code Red show that after accurately estimating the distribution of vulnerable hosts, a self-learning worm can spread much faster than a random-scanning worm, a permutation-scanning worm, and a Class A routing worm. Some guidelines for detecting and defending against such self learning worms are also discussed.
1853937
Numerically Stable Real Number Codes Based on Random Matrices
Error correction codes defined over real-number field have been studied and recognized as useful in many applications. However, most real-number codes in literature are quite suspect in their numerical stability. In this paper, we introduce a class of real-number codes based on random generator matrices over real-number fields. Codes over complex-number field are also discussed. Experiment results demonstrate our codes are numerically much more stable than existing codes in literature.
7582784
Hierarchical Clustering-Merging for Multidimensional Index Structures
The R-tree family index structures are among the most common index structures used in multidimensional databases. To improve the search performance it is very important to reduce the overlap between bounding regions in the R-tree. However the arbitrary insertion order in the tree construction procedure might result in tree structures inefficient in the search operations. In this paper we propose a new technique called Hierarchical Clustering-Merging (HCM) to improve the tree construction procedure of the R-tree family index structures. With this technique we can take advantage of the data distribution information in the data set to achieve an optimized tree structure and improve the search performance.
52859343
Towards rate-distortion tradeoff in real-time color video coding
In this paper, we address to the key problem in real-time video coding, the rate-distortion tradeoff. We found that the distribution of the integer transform coefficients in H.264 color video coding can be approximated by Laplacian distribution but the rate-distortion characteristics should be modelled for the luminance and chrominance components separately. So we propose separable rate-distortion models for color video coding. The feedback from the encoder buffer is analyzed by a control-theoretic adaptation approach. To achieve smooth video quality and satisfy the delay constraints in real-time applications, a novel rate distortion tradeoff controller is employed. Simulations results demonstrate that the proposed approach can achieve smooth video quality without sacrificing overall quality while efficiently utilize the bandwidth without buffer overflow or underflow.
13035041
Searching the Web with Queries
In this paper we study the problem of searching the Web with online learning algorithms. We consider that Web documents can be represented by vectors of n boolean attributes. A search engine is viewed as a learner, and a user is viewed as a teacher. We investigate the number of queries a search engine needs from the user to search for a collection of Web documents. We design several efficient learning algorithms to search for any collection of documents represented by a disjunction (or a conjunction) of relevant attributes with the help of membership queries or equivalence queries.