16831415
Automatic construction of English/Chinese parallel corpora
As the demand for global information increases significantly, multilingual corpora has become a valuable linguistic resource for applications to cross-lingual information retrieval and natural language processing. In order to cross the boundaries that exist between different languages, dictionaries are the most typical tools. However, the general-purpose dictionary is less sensitive in both genre and domain. It is also impractical to manually construct tailored bilingual dictionaries or sophisticated multilingual thesauri for large applications. Corpus-based approaches, which do not have the limitation of dictionaries, provide a statistical translation model with which to cross the language boundary. There are many domain-specific parallel or comparable corpora that are employed in machine translation and cross-lingual information retrieval. Most of these are corpora between Indo-European languages, such as English/French and English/Spanish. The Asian/Indo-European corpus, especially English/Chinese corpus, is relatively sparse. The objective of the present research is to construct English/ Chinese parallel corpus automatically from the World Wide Web. In this paper, an alignment method is presented which is based on dynamic programming to identify the one-to-one Chinese and English title pairs. The method includes alignment at title level, word level and character level. The longest common subsequence (LCS) is applied to find the most reliable Chinese translation of an English word. As one word for a language may translate into two or more words repetitively in another language, the edit operation, deletion, is used to resolve redundancy. A score function is then proposed to determine the optimal title pairs. Experiments have been conducted to investigate the performance of the proposed method using the daily press release articles by the Hong Kong SAR government as the test bed. The precision of the result is 0.998 while the recall is 0.806. The release articles and speech articles, published by Hongkong & Shanghai Banking Corporation Limited, are also used to test our method, the precision is 1.00, and the recall is 0.948.
17665082
Generating Cross-lingual Concept Space from Parallel Corpora on the Web
The information available in languages other than English on the World Wide Web is increasing significantly. To cross language boundaries between different languages, dictionaries are the most typical tools. However, the general-purpose dictionary is less sensitive in genre and domain and it is impractical to manually construct tailored bilingual dictionaries or sophisticated multilingual thesauri for large applications. Corpus-based approaches, which do not have the limitation of dictionaries, provide a statistical translation model to cross the language boundary. The objective of this research work is to mine English/Chinese parallel documents automatically from the World Wide Web and generate a crosslingual concept space automatically for cross-lingual information retrieval. The alignment method is developed based on dynamic programming to identify the one-to-one Chinese and English title pairs for building parallel corpus. The Hopfield network is then employed to generate the cross-lingual concept space based on the statistical correlation analysis of the semantics (knowledge) embedded in the bilingual press release corpus. The research output consisted of a thesaurus-like, semantic network knowledge base, which can aid in semantics-based cross-lingual information management and retrieval.
16166999
Automatic Construction of Cross-Lingual Networks of Concepts from the Hong Kong SAR Police Department
The tragic event of September 11 has prompted the rapid growth of attention of national security and criminal analysis. In the national security world, very large volumes of data and information are generated and gathered. Much of this data and information written in different languages and stored in different locations may be seemingly unconnected. Therefore, cross-lingual semantic interoperability is a major challenge to generate an overview of this disparate data and information so that it can be analysed, searched. The traditional information retrieval (IR) approaches normally require a document to share some keywords with the query. In reality, the users may use some keywords that are different from what used in the documents. There are then two different term spaces, one for the users, and another for the documents. The problem can be viewed as the creation of a thesaurus. The creation of such relationships would allow the system to match queries with relevant documents, even though they contain different terms. Apart from this, terrorists and criminals may communicate through letters, e-mails and faxes in languages other than English. The translation ambiguity significantly exacerbates the retrieval problem. To facilitate cross-lingual information retrieval, a corpusbased approach uses the term co-occurrence statistics in parallel or comparable corpora to construct a statistical translation model to cross the language boundary. However, collecting parallel corpora between European language and Oriental language is not an easy task due to the unique linguistics and grammar structures of oriental languages. In this paper, the text-based approach to align English/Chinese Hong Kong Police press release documents from the Web is first presented. This article then reports an algorithmic approach to generate a robust knowledge base based on statistical correlation analysis of the semantics (knowledge) embedded in the bilingual press release corpus. The research output consisted of a thesaurus-like, semantic network knowledge base, which can aid in semantics-based cross-lingual information management and retrieval.
15408418
CatRelate: A New Hierarchical Document Category Integration Algorithm by Learning Category Relationships
We address the problem of integrating documents from a source catalog into a master catalog. Current technologies for solving the problem deem it as a flat category integration problem without considering the useful hierarchy information in the catalog, or deal with it hierarchically but without a rigorous model. In contrast, our method is based on correctly identifying relationships among categories, such as Match, Disjoint, SubConcept, SuperConcept, and Overlap, which come from the relations of sets in Set theory. Compared with traditional Match/NotMatch relationship in literature, our approach is more expressive in defining the relationship. The relationships among categories are first learned in a probabilistic way, and then refined by considering the hierarchy context. Our preliminary experiments show that it can help to correctly identify category relationships, and thus increase the accuracy of document integration.
15657957
Automatic Summarization of Chinese and English
As a result of the rapid growth in Internet access, significantly more information has become available online in real time. However, there is not sufficient time for users to read large volumes of information and make decisions accordingly. The problem of information-overloading can be resolved through the application of automatic summarization. Many summarization systems for documents in different languages have been implemented. However, the performance of summarization system on documents in different languages has not yet been investigated. In this paper, we compare the result of fractal summarization technique on parallel documents in Chinese and English. The grammatical and lexical differences between Chinese and English have significant effect on the summarization processes. Their impact on the performances of the summarization for the Chinese and English parallel documents is compared.
2566456
Interval constraint networks for tolerance analysis and synthesis
This paper proposes interval constraint networks and interval propagation techniques for automatic tolerance design. To develop an intelligent automatic tolerance design mechanism, the following interrelated problems are addressed: (1) representing the relationships between the tolerances of the entities, attributes, and the functional requirements of a design component by a constraint network, (2) developing tolerance propagation techniques in the network that will satisfy the constraints between the entities, attributes and functional requirement. For the purpose of tolerance design, we propose new definitions for consistency of constraints and consistency of a constraint network. New forward and backward propagation techniques achieve worst-case tolerance analysis and synthesis, respectively. A tank design problem illustrates the use of the procedures for tolerance analysis and synthesis. The discussion and examples compare the new constraint network and propagation techniques with previous works.
2733911
A Hybrid Parallel Loop Scheduling Scheme on Heterogeneous PC Clusters
Traditional loop-scheduling schemes are designed for homogeneous computing environments, and are probably not suitable for emerging heterogeneous PC clusters. This paper applies a two-phased method, named HPLS (Hybrid Parallel Loop Scheduling), to heterogeneous PC-cluster environments. The key idea is to distribute most of the workload to each node according to its performance, which is modeled and estimated in advance. Experiments on our testbed PCcluster showed that, in most cases, our method could reduce the execution time of application programs more than previous schemes could.
16872458
Decision tree construction for data mining on grid computing environments
In this paper, the authors presented the grid-based decision tree architecture, with the intention of applying it to both parallel and sequential algorithms. Also, it is shown that, based on the scope and model of data mining applied in the grid environment as well as user equivalent perspective, grid roles can be categorized into three types. It was aimed, through these definitions, to help software developers define clear system processes and differentiate the application scope for software applications. To fulfill the architecture, an existing parallel decision tree algorithm was first applied (the SPRINT algorithm) to the grid environment. The performance and differences in many other areas are compared using datasets of different sizes. The experimental results will be used for future reference and further development.
36980746
Decision tree construction for data mining on grid computing
Decision tree is one of the frequently used methods in data mining for searching prediction information. Due to its characteristics which are suitable for parallelism, it has been widely adopted in high performance field and developed into various parallel decision tree algorithms to deal with huge data and complex computation. Following the development of other technology fields, grid computing is regarded as the extension of PC cluster and therefore it future research development is highly valued. This new wave of Internet application is the 3rd generation of Internet applications following the traditional Internet and Web application. We have presented a grid-based decision tree architecture, and hope it can be applied on both parallel and sequential algorithms for the decision tree applications. Also, based on the scope and model of data mining applied in grid environment as well as user equivalent perspective, grid roles can be categorized into three types. We are hoping that through these definitions, software developers can define clear system processes and differentiate the application scope for software applications. To fulfil our architecture, we first apply an existing parallel decision tree algorithm-SPRINT algorithm in the grid environment. The performance and differences in many other areas are compared using different sizes of dataset. The experimental results are used for future reference and further development.
53783846
Translating–transliterating named entities for multilingual information access
Named entities are major constituents of a document but are usually unknown words. This work proposes a systematic way of dealing with formulation, transformation, translation, and transliteration of multilingual-named entities. The rules and similarity matrices for translation and transliteration are learned automatically from parallel-named-entity corpora. The results are applied in cross-language access to collections of images with captions. Experimental results demonstrate that the similarity-based transliteration of named entities is effective, and runs in which transliteration is considered outperform the runs in which it is neglected. © 2006 Wiley Periodicals, Inc.
18251933
A Information Monitoring and Job Scheduling System for Multiple Linux PC Clusters
Managing and monitoring a cluster is both a tedious and challenging task, since each computing node is designed as a stand-alone system rather than a part of a parallel architecture. In this paper, a software system that allows the centralized administration of a generic Beowulf cluster is proposed. This system also provides Web services and applications to monitor multiple PC clusters with job submission and scheduling