9663679
The Use of SVM for Chinese New Word Identification
We present a study of new word identification (NWI) to improve the performance of a Chinese word segmenter. In this paper the distribution and types of new words are discussed empirically. In particular, we focus on the new words of two surface patterns, which account for more than 80% of new words in our data sets: NW11 (two-character new word) and NW21 (a bi-character word followed with a single character). NWI is defined as a problem of binary classification. A statistical learning approach based on a SVM classifier is used. Different features for NWI are explored, including in-word probability of a character (IWP), the analogy between new words and lexicon words, anti-word list, and frequency in documents. The experiments show that these features are useful for NWI. The F-scores of NWI we achieved are 64.4% and 54.7% for NW11 and NW21, respectively. The overall performance of the Chinese word segmenter could be improved by Roov 24.5% and F-score 6.5% in PK-close test of the 1st SIGHAN bakeoff. This achieves the performance of state-of-the-art word segmenters.
12419149
On the use of words and n-grams for Chinese information retrieval
In the processing of Chinese documents and queries in information retrieval (IR), one has to identify the units that are used as indexes. Words and n-grams have been used as indexes in several previous studies, which showed that both kinds of indexes lead to comparable IR performances. In this study, we carry out more experiments on different ways to segment documents and queries, and to combine words with n-grams. Our experiments show that a combination of the longest-matching algorithm with single characters is the best choice.
461312
Dependence language model for information retrieval
This paper presents a new dependence language modeling approach to information retrieval. The approach extends the basic language modeling approach based on unigram by relaxing the independence assumption. We integrate the linkage of a query as a hidden variable, which expresses the term dependencies within the query as an acyclic, planar, undirected graph. We then assume that a query is generated from a document in two stages: the linkage is generated first, and then each term is generated in turn depending on other related terms according to the linkage. We also present a smoothing method for model parameter estimation and an approach to learning the linkage of a sentence in an unsupervised manner. The new approach is compared to the classical probabilistic retrieval model and the previously proposed language models with and without taking into account term dependencies. Results show that our model achieves substantial and significant improvements on TREC collections.
8204298
Improved Source-Channel Models for Chinese Word Segmentation
This paper presents a Chinese word segmentation system that uses improved source-channel models of Chinese sentence generation. Chinese words are defined as one of the following four types: lexicon words, morphologically derived words, factoids, and named entities. Our system provides a unified approach to the four fundamental features of word-level Chinese language processing: (1) word segmentation, (2) morphological analysis, (3) factoid detection, and (4) named entity recognition. The performance of the system is evaluated on a manually annotated test set, and is also compared with several state-of-the-art systems, taking into account the fact that the definition of Chinese words often varies from system to system.
11423197
Unsupervised Learning of Dependency Structure for Language Modeling
This paper presents a dependency language model (DLM) that captures linguistic constraints via a dependency structure, i.e., a set of probabilistic dependencies that express the relations between headwords of each phrase in a sentence by an acyclic, planar, undirected graph. Our contributions are three-fold. First, we incorporate the dependency structure into an n-gram language model to capture long distance word dependency. Second, we present an unsupervised learning method that discovers the dependency structure of a sentence using a bootstrapping procedure. Finally, we evaluate the proposed models on a realistic application (Japanese Kana-Kanji conversion). Experiments show that the best DLM achieves an 11.3% error rate reduction over the word trigram model.
11590179
GLIDER: gradient landmark-based distributed routing for sensor networks
We present gradient landmark-based distributed routing (GLIDER), a novel naming/addressing scheme and associated routing algorithm, for a network of wireless communicating nodes. We assume that the nodes are fixed (though their geographic locations are not necessarily known), and that each node can communicate wirelessly with some of its geographic neighbors - a common scenario in sensor networks. We develop a protocol which in a preprocessing phase discovers the global topology of the sensor field and, as a byproduct, partitions the nodes into routable tiles - regions where the node placement is sufficiently dense and regular that local greedy methods can work well. Such global topology includes not just connectivity but also higher order topological features, such as the presence of holes. We address each node by the name of the tile containing it and a set of local coordinates derived from connectivity graph distances between the node and certain landmark nodes associated with its own and neighboring tiles. We use the tile adjacency graph for global route planning and the local coordinates for realizing actual inter- and intra-tile routes. We show that efficient load-balanced global routing can be implemented quite simply using such a scheme.
40124869
Image Pattern Recognition Based on Examples - A Combined Statistical and Structural-Syntactic Approach
An application of combined statistical and structural-syntactic approach in Chinese character recognition is presented. The algorithm adopts a structural representation for Chinese characters, but in the classification and training process, the structural matching and parameter adjustment is conducted in a statistical way. Different from the conventional structural approaches, in this system, only a few predefined "knowledge" is required. In most cases, knowledge acquisition is simplified to "memorization" of examples, and the parameters for classification can be refined using statistical training. In this way it avoids the main difficulties inherent in the implementation of classification systems based on structural features. Compared with conventional statistical algorithms, the algorithm is based on a structural model of image patterns, so it has approximately all the advantages of structural pattern recognition algorithms. A prototype system has been realized based on this strategy, and the effectiveness of the method is verified by the experimental results.
17381912
Hydrogen-bonding interactions in the active site of a low molecular weight protein-tyrosine phosphatase
Molecular dynamics simulation of the Michaelis complex, phospho-enzyme intermediate, and the wild-type and C12S mutant have been carried out to examine hydrogen-bonding interactions in the active site of the bovine low molecular weight protein-tyrosine phosphatase (BPTP). It was found that the Sγ atom of the nucleophilic residue Cys-12 is ideally located at a position opposite from the phenylphosphate dianion for an inline nucleophilic substitution reaction. In addition, electrostatic and hydrogen-bonding interactions from the backbone amide groups of the phosphate-binding loop strongly stabilize the thiolate anion, making Cys-12 ionized in the active site. In the phospho-enzyme intermediate, three water molecules are found to form strong hydrogen bonds with the phosphate group. In addition, another water molecule can be identified to form bridging hydrogen bonds between the phosphate group and Asp-129, which may act as the nucleophile in the subsequent phosphate hydrolysis reaction, with Asp-129 serving as a general base. The structural difference at the active site between the wild-type and C12S mutant has been examined. It was found that the alkoxide anion is significantly shifted toward one side of the phosphate binding loop, away from the optimal position enjoyed by the thiolate anion of the wild-type enzyme in an SN2 process. This, coupled with the high pKa value of an alcoholic residue, makes the C12S mutant catalytically inactive. These molecular dynamics simulations provided details of hydrogen bonding interactions in the active site of BPTP, and a structural basis for further studies using combined quantum mechanical and molecular mechanical potential to model the entire dephosphorylation reaction by BPTP. c © 2000 John Wiley & Sons, Inc. J Comput Chem 21: 1192–1203, 2000
14630821
A New Semi-Supervised Subspace Clustering Algorithm on Fitting Mixture Models
We propose a new subspace clustering algorithm (EPSCMIX), which is based on the feature saliency measure that is obtained by using both the Emerging Patterns algorithm and the EM algorithm, for the analysis of microarray data. For the model selection, it employs a novel agglomerative step together with MDL criterion. And, we present the result of comparative experiments between AIC, MDL and minimum message length (MML) used to determine a criterion for our algorithm. The robustness of using emerging patterns based on mixture models, as well as using the Gaussian mixture model for subspace clustering, was demonstrated on both synthetic and real data sets. In experiments, it also certified that a new agglomerative method that merges mostly correlated components with MDL consistently worked better than the one that removes weak weight components.
2327805
Detecting Wormhole Attacks in Wireless Networks Using Connectivity Information
We propose a novel algorithm for detecting worm-hole attacks in wireless multi-hop networks. The algorithm uses only connectivity information to look for forbidden substructures in the connectivity graph. The proposed approach is completely localized and, unlike many techniques proposed in literature, does not use any special hardware artifact or location information, making the technique universally applicable. The algorithm is independent of wireless communication models. However, knowledge of the model and node distribution helps estimate a parameter used in the algorithm. We present simulation results for three different communication models and two different node distributions, and show that the algorithm is able to detect wormhole attacks with a 100% detection and 0% false alarm probabilities whenever the network is connected with high probability. Even for very low density networks where chances of disconnection is very high, the detection probability remains very high.