461312
Dependence language model for information retrieval
This paper presents a new dependence language modeling approach to information retrieval. The approach extends the basic language modeling approach based on unigram by relaxing the independence assumption. We integrate the linkage of a query as a hidden variable, which expresses the term dependencies within the query as an acyclic, planar, undirected graph. We then assume that a query is generated from a document in two stages: the linkage is generated first, and then each term is generated in turn depending on other related terms according to the linkage. We also present a smoothing method for model parameter estimation and an approach to learning the linkage of a sentence in an unsupervised manner. The new approach is compared to the classical probabilistic retrieval model and the previously proposed language models with and without taking into account term dependencies. Results show that our model achieves substantial and significant improvements on TREC collections.
11423197
Unsupervised Learning of Dependency Structure for Language Modeling
This paper presents a dependency language model (DLM) that captures linguistic constraints via a dependency structure, i.e., a set of probabilistic dependencies that express the relations between headwords of each phrase in a sentence by an acyclic, planar, undirected graph. Our contributions are three-fold. First, we incorporate the dependency structure into an n-gram language model to capture long distance word dependency. Second, we present an unsupervised learning method that discovers the dependency structure of a sentence using a bootstrapping procedure. Finally, we evaluate the proposed models on a realistic application (Japanese Kana-Kanji conversion). Experiments show that the best DLM achieves an 11.3% error rate reduction over the word trigram model.
12419149
On the use of words and n-grams for Chinese information retrieval
In the processing of Chinese documents and queries in information retrieval (IR), one has to identify the units that are used as indexes. Words and n-grams have been used as indexes in several previous studies, which showed that both kinds of indexes lead to comparable IR performances. In this study, we carry out more experiments on different ways to segment documents and queries, and to combine words with n-grams. Our experiments show that a combination of the longest-matching algorithm with single characters is the best choice.
14630821
A New Semi-Supervised Subspace Clustering Algorithm on Fitting Mixture Models
We propose a new subspace clustering algorithm (EPSCMIX), which is based on the feature saliency measure that is obtained by using both the Emerging Patterns algorithm and the EM algorithm, for the analysis of microarray data. For the model selection, it employs a novel agglomerative step together with MDL criterion. And, we present the result of comparative experiments between AIC, MDL and minimum message length (MML) used to determine a criterion for our algorithm. The robustness of using emerging patterns based on mixture models, as well as using the Gaussian mixture model for subspace clustering, was demonstrated on both synthetic and real data sets. In experiments, it also certified that a new agglomerative method that merges mostly correlated components with MDL consistently worked better than the one that removes weak weight components.
42656979
Geometric spanner for routing in mobile networks
15261652
Rendezvous Points-Based Scalable Content Discovery with Load Balancing
ABSTRACT A Content Dis overy System (CDS) enables ontent onsumers to dis over ontent published by ontent providers via a set of ontent resolvers. Existing solutions to CDS systems have diAE ulties in a hieving both ri h fun tionality and s alability. In this paper, we present a distributed and s alable approa h based on Rendezvous Points (RPs) that eAE iently supports exible sear h of dynami ontents. Our system deploys load balan ing matri es (LBMs) to dynami ally balan e registration and query load a ross resolver nodes to maximize system utilization. Our system utilizes existing hash-based algorithms for ontent and query distribution within the overlay resolver network. We use simulation to validate our design.
2327805
Detecting Wormhole Attacks in Wireless Networks Using Connectivity Information
We propose a novel algorithm for detecting worm-hole attacks in wireless multi-hop networks. The algorithm uses only connectivity information to look for forbidden substructures in the connectivity graph. The proposed approach is completely localized and, unlike many techniques proposed in literature, does not use any special hardware artifact or location information, making the technique universally applicable. The algorithm is independent of wireless communication models. However, knowledge of the model and node distribution helps estimate a parameter used in the algorithm. We present simulation results for three different communication models and two different node distributions, and show that the algorithm is able to detect wormhole attacks with a 100% detection and 0% false alarm probabilities whenever the network is connected with high probability. Even for very low density networks where chances of disconnection is very high, the detection probability remains very high.
207794
Trade-Offs between Stretch Factor and Load-Balancing Ratio in Routing on Growth-Restricted Graphs
An unweighted graph has density rho and growth rate k if the number of nodes in every ball with radius r is bounded by rhork. The communication graphs of wireless networks and peer-to-peer networks often have constant bounded density and small growth rate. In this paper, we study the trade-off between two quality measures for routing in growth-restricted graphs. The two measures we consider are the stretch factor, which measures the lengths of the routing paths, and the load-balancing ratio, which measures the evenness of the traffic distribution. We show that if the routing algorithm is required to use paths with stretch factor c, then its load-balancing ratio is bounded by O(rho1/k(n/c)1-1/k), and the bound is tight in the worst case. We show the application and extension of the trade-off to the wireless network routing and VLSI layout design. We also present a load-balanced routing algorithm with the stretch factor constraint in an online setting, in which the routing requests come one by one.