14151705
An adaptive k-nearest neighbor text categorization strategy
k is the most important parameter in a text categorization system based on the k-nearest neighbor algorithm (kNN). To classify a new document, the k-nearest documents in the training set are determined first. The prediction of categories for this document can then be made according to the category distribution among the k nearest neighbors. Generally speaking, the class distribution in a training set is not even; some classes may have more samples than others. The system's performance is very sensitive to the choice of the parameter k. And it is very likely that a fixed k value will result in a bias for large categories, and will not make full use of the information in the training set. To deal with these problems, an improved kNN strategy, in which different numbers of nearest neighbors for different categories are used instead of a fixed number across all categories, is proposed in this article. More samples (nearest neighbors) will be used to decide whether a test document should be classified in a category that has more samples in the training set. The numbers of nearest neighbors selected for different categories are adaptive to their sample size in the training set. Experiments on two different datasets show that our methods are less sensitive to the parameter k than the traditional ones, and can properly classify documents belonging to smaller classes with a large k. The strategy is especially applicable and promising for cases where estimating the parameter k via cross-validation is not possible and the class distribution of a training set is skewed.
1260337
An Improved k-Nearest Neighbor Algorithm for Text Categorization
k is the most important parameter in a text categorization system based on k-Nearest Neighbor algorithm (kNN).In the classification process, k nearest documents to the test one in the training set are determined firstly. Then, the predication can be made according to the category distribution among these k nearest neighbors. Generally speaking, the class distribution in the training set is uneven. Some classes may have more samples than others. Therefore, the system performance is very sensitive to the choice of the parameter k. And it is very likely that a fixed k value will result in a bias on large categories. To deal with these problems, we propose an improved kNN algorithm, which uses different numbers of nearest neighbors for different categories, rather than a fixed number across all categories. More samples (nearest neighbors) will be used for deciding whether a test document should be classified to a category, which has more samples in the training set. Preliminary experiments on Chinese text categorization show that our method is less sensitive to the parameter k than the traditional one, and it can properly classify documents belonging to smaller classes with a large k. The method is promising for some cases, where estimating the parameter k via cross-validation is not allowed.
195606600
CTEMP: a chinese temporal parser for extracting and normalizing temporal information
Temporal information is useful in many NLP applications, such as information extraction, question answering and summarization. In this paper, we present a temporal parser for extracting and normalizing temporal expressions from Chinese texts. An integrated temporal framework is proposed, which includes basic temporal concepts and the classification of temporal expressions. The identification of temporal expressions is fulfilled by powerful chart-parsing based on grammar rules and constraint rules. We evaluated the system on a substantial corpus and obtained promising results.
71018
Experimental Study on Representing Units in Chinese Text Categorization
This paper is a comparative study on representing units in Chinese text categorization. Several kinds of representing units, including byte 3-gram, Chinese character, Chinese word, and Chinese word with part of speech tag, were investigated. Empirical evidence shows that when the size of training data is large enough, representations of higher-level or with larger feature spaces result in better performance than those of lower level or with smaller feature spaces, whereas when the training data is limited the conclusion may be the reverse. In general, representations of higher-level or with larger feature spaces need more training data to reach the best performance. But, as to a specific representation, the size of training data and the categorization performance are not always positively correlated.
17742402
A Unicode-based Adaptive Segmenter
This paper presents a Unicode based Chinese word segmentor. It can handle Chinese text in Simplified, Traditional, or mixed mode. The system uses the strategy of divide-and-conquer to handle the recognition of personal names, numbers, time and numerical values, etc in the preprocessing stage. The segmentor further uses tagging information to work on disambiguation. Adopting a modular design approach, different functional parts are separately implemented using different modules and each module tackles one problem at a time providing more flexibility and extensibility. Results show that with added pre-processing modules and accessorial modules, the accuracy of the segmentor is increased and the system is easily adaptive to different applications.
34801552
Profile-based event tracking
In this research, we focus on tracking topics that originate and evolve from a specific event. Intuitively, a few key elements of a target event, such as date, location, and persons involved, would be enough for making a decision on whether a test story is on-topic. Consequently, a profile-based event tracking method is proposed. We attempt to build an event profile from the given on-topic stories by robust information retrieval technologies. A feature selection metric and a recognized event clause are utilized to determine most (if not all) key semantic elements of the target event. Preliminary experiments on the TDT2 mandarin corpus show that this profile-based event tracking method is promising.
10610312
Semantic sports video analysis: approaches and new applications
In this paper, we show that event-based semantic analysis of sports video can be achieved through two major classes of methods: approaches using deterministic reasoning and approaches depending on probabilistic inference. Deterministic approaches depend on an explicit description of an event and an explicit representation of the event in terms of low-level aural-visual features that can be automatically detected. Probabilistic approaches are based on some proper state/transition models whose parameters are learnt through labeled training sequences. We first discuss the pros and cons of the two classes of approaches. Next, we present algorithms of these two types for a new application, automatic analysis of coaching video (which is different from sports broadcast programs that have been subject of most prior work).
15806898
Market-driven bandwidth allocation in selfish overlay networks
Selfish overlay networks consist of autonomous nodes that develop their own strategies by optimizing towards their local objectives and self-interests, rather than following prescribed protocols. It is thus important to regulate the behavior of selfish nodes, so that system-wide properties are optimized. In this paper, we investigate the problem of bandwidth allocation in overlay networks, and propose to use a market-driven approach to regulate the behavior of selfish nodes that either provide or consume services. In such markets, consumers of services select the best service providers, taking into account both the performance and the price of the service. On the other hand, service providers are encouraged to strategically decide their respective prices in a pricing game, in order to maximize their economic revenues and minimize losses in the long run. In order to overcome the limitations of previous models towards similar objectives, we design a decentralized algorithm that uses reinforcement learning to help selfish nodes to incrementally adapt to the local market, and to make optimized strategic decisions based on past experiences. We have simulated our proposed algorithm in randomly generated overlay networks, and have shown that the behavior of selfish nodes converges to their optimal strategies, and resource allocations in the entire overlay are near-optimal, and efficiently adapts to the dynamics of overlay networks.
7321726
A gateway-assisted approach toward QoS adaptations
We present a gateway-assisted QoS adaptation framework which satisfies high-level QoS application guarantees. We validate this framework via a distributed tracking system and show provisioning of critical QoS guarantees despite the best effort underlying resource management.
46149042
Data reconciliation for real-time optimization of an industrial coke-oven-gas purification process
Online optimization is more and more used in the chemical industry to run a process near its optimum operating condition by providing real-time computed optimal set-points to the distributed control system. Process measurements are necessary for these applications to determine the actual state of the process and to increase the accuracy of the model with parameter estimation techniques. However, these measurements usually contain random as well as gross errors which have to be identified and eliminated before the measurements are used for online optimization. In this contribution, a data reconciliation approach was integrated into an online optimization framework for the ammonia hydrogen sulfide circulation scrubbing, a common industrial coke-oven-gas purification process. We used a rigorous rate-based model to describe this reactive absorption and desorption process. To increase the accuracy of the model, we estimated several process parameters using a sequential parameter estimation approach. Data reconciliation was performed based on simple component balances to achieve model-consistent data and to identify measurement biases. The model was then validated online on a pilot plant by connecting the estimation package through the process control system. Based on the online measured data, operating cost minimization was carried out and the computed optimal set-points realized real-time. A satisfactory agreement between measured data and optimization was achieved.
5536365
Simulation of Diabetic Retinopathy Neovascularization in Color Digital Fundus Images
Diabetic retinopathy (DR) has been identified as a leading cause of blindness. One type of lesion, neovascularization (NV), indicates that the disease has entered a vision-threatening phase. Early detection of NV is thus clinically significant. Efforts have been devoted to use computer-aided analyses of digital retina images to detect DR. However, developing reliable NV detection algorithms requires large numbers of digital retinal images to test and refine approaches. Computer simulation of NV offers the potential of developing lesion detection algorithms without the need for large image databases of real pathology. In this paper, we propose a systematic approach to simulating NV. Specifically, we propose two algorithms based on fractal models to simulate the main structure of NV and an adaptive color generation method to assign photorealistic pixel values to the structure. Moreover, we develop an interactive system that provides instant visual feedback to support NV simulation guided by an ophthalmologist. This enables us to combine the low level algorithms with high-level human feedback to simulate realistic lesions. Experiments suggest that our method is able to produce simulated NVs that are indistinguishable from real lesions.