18349973
Bayesian Ying-Yang machine, clustering and number of clusters
Abstract It is shown that a particular case of the Bayesian Ying–Yang learning system and theory reduces to the maximum likelihood learning of a finite mixture, from which we have obtained not only the EM algorithm for its parameter estimation and its various approximate but fast algorithms for clustering in general cases (including Mahalanobis distance clustering or elliptic clustering), but also criteria for the selection of the number of densities in a mixture, and the number k in the conventional Mean Square Error clustering. Moreover, a Re-weighted EM algorithm is also proposed and shown to be more robust in learning. Finally, experimental results are provided.
8090511
Bayesian Kullback Ying-Yang dependence reduction theory
Bayesian Kullback Ying—Yang dependence reduction system and theory is presented. Via stochastic approximation, implementable algorithms and criteria are given for parameter learning and model selection, respectively. Three typical architectures are further studied on several special cases. The forward one is a general information theoretic dependence reduction model that maps an observation x into a representation y of k independent components, with k detectable by criteria. For the special cases of invertible map xPy, a general adaptive algorithm is obtained, which not only is applicable to nonlinear or post-nonlinear mixtures, but also provides an adaptive EM algorithm that implements the previously proposed learned parametric mixture method for independent component analysis (ICA) on linear mixtures. The backward architecture provides a maximum likelihood independent factor model for modeling observations from unknown number of independent factors via a linear or nonlinear system under noisy situations. For the special cases of linear or post-nonlinear mixture under Gaussian noise, the simplified adaptive algorithm and the criterion for detecting k are given, with an approximately optimal linear mapping xPy suggested. Moreover, if the independent factors are assumed to be standard Gaussians, we are further led to the conventional factor analysis, but with a new adaptive algorithm for its estimation and a criterion for deciding the number of factors. The bi-directional architecture combines the advantages of backward and forward ones. A mean field approximation is presented, with a simplified adaptive parameter learning algorithm and an approximate k-selection criterion. Moreover, its special cases lead to the existing least mean square error reconstruction learning and the one hidden layer deterministic Helmholtz machine, with new findings. Also, a specific degenerate case of bi-directional architecture results in a non-invertible but adaptively implementable forward on to mapping for ICA. Experiments on binary sources are demonstrated with successes. ( 1998 Elsevier Science B.V. All rights reserved.
6364266
BYY harmony learning, structural RPCL, and topological self-organizing on mixture models
The Bayesian Ying-Yang (BYY) harmony learning acts as a general statistical learning framework, featured by not only new regularization techniques for parameter learning but also a new mechanism that implements model selection either automatically during parameter learning or via a new class of model selection criteria used after parameter learning. In this paper, further advances on BYY harmony learning by considering modular inner representations are presented in three parts. One consists of results on unsupervisedmixture models, ranging from Gaussian mixture based Mean Square Error (MSE) clustering, elliptic clustering, subspace clustering to NonGaussian mixture based clustering not only with each cluster represented via either Bernoulli-Gaussian mixtures or independent real factor models, but also with independent component analysis implicitly made on each cluster. The second consists of results on supervised mixture-of-experts (ME) models, including Gaussian ME, Radial Basis Function nets, and Kernel regressions. The third consists of two strategies for extending the above structural mixtures into self-organized topological maps. All these advances are introduced with details on three issues, namely, (a) adaptive learning algorithms, especially elliptic, subspace, and structural rival penalized competitive learning algorithms, with model selection made automatically during learning; (b) model selection criteria for being used after parameter learning, and (c) how these learning algorithms and criteria are obtained from typical special cases of BYY harmony learning.
16709558
One-Bit-Matching Conjecture for Independent Component Analysis
The one-bit-matching conjecture for independent component analysis (ICA) could be understood from different perspectives but is basically stated as all the sources can be separated as long as there is a one-toone same-sign-correspondence between the kurtosis signs of all source probability density functions (pdf's) and the kurtosis signs of all model pdf's (Xu, Cheung, & Amari, 1998a). This conjecture has been widely believed in the ICA community and implicitly supported by many ICA studies, such as the Extended Infomax (Lee, Girolami, & Sejnowski, 1999) and the soft switching algorithm (Welling & Weber, 2001). However, there is no mathematical proof to confirm the conjecture theoretically. In this article, only skewness and kurtosis are considered, and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes. Moreover, empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks. As a by-product, we also show that the kurtosis maximization criterion (Moreau & Macchi, 1996) is actually a special case of the minimum mutual information criterion for ICA.
14770227
Data smoothing regularization, multi-sets-learning, and problem solving strategies
First, we briefly introduce the basic idea of data smoothing regularization, which was firstly proposed by Xu [Brain-like computing and intelligent information systems (1997) 241] for parameter learning in a way similar to Tikhonov regularization but with an easy solution to the difficulty of determining an appropriate hyper-parameter. Also, the roles of this regularization are demonstrated on Gaussian-mixture via smoothed versions of the EM algorithm, the BYY model selection criterion, adaptive harmony algorithm as well as its related Rival penalized competitive learning. Second, these studies are extended to a mixture of reconstruction errors of Gaussian types, which provides a new probabilistic formulation for the multi-sets learning approach [Proc. IEEE ICNN94 I (1994) 315] that learns multiple objects in typical geometrical structures such as points, lines, hyperplanes, circles, ellipses, and templates of given shapes. Finally, insights are provided on three problem solving strategies, namely the competition-penalty adaptation based learning, the global evidence accumulation based selection, and the guess-test based decision, with a general problem solving paradigm suggested.
37879675
A new curve detection method: Randomized Hough transform (RHT)
Abstract A new method is proposed for curve detection. For a curve with n parameters, instead of transforming one pixel into a hypersurface of the n -D parameter space as the HT and its variants do, we randomly pick n pixels and map them into one point in the parameter space. In comparison with the HT and its variants, our new method has the advantages of small storage, high speed, infinite parameter space and arbitrarily high resolution. The preliminary experiments have shown that the new method is quite effective.
39559674
A framework for Web applications testing
Web application testing is concerned with numerous and complicated testing objects, methods and processes. So a testing framework fitting for the properties of Web application is needed to guide and organize all the testing tasks. Based on the analysis for Web application characters and traditional software testing process, the process for Web application testing is modeled, which describes a series of testing flows such as the testing requirement analysis, test cases generation and selection, testing execution, and testing results analysis and measurement. Furthermore, the realization techniques are also investigated so as to integrate each testing step and implement the whole testing process harmoniously and effectively. Thus the framework is suitable for the Internet environment and can guide the Web application testing actively and availably.
32570723
Parallel algorithm for mining fuzzy association rules
The principle and steps of the algorithm for mining fuzzy association rules is studied, and the parallel algorithm for mining fuzzy association rules is presented. In this parallel mining algorithm, quantitative attributes are partitioned into several fuzzy sets by the parallel fuzzy c-means algorithm, and fuzzy sets are applied to soften the partition boundary of the attributes. Then, the parallel algorithm for mining Boolean association rules is improved to discover frequent fuzzy attributes. Last, the fuzzy association rules with at least fuzzy confidence are generated on all processors. The parallel mining algorithm is implemented on the distributed linked PC/workstation. The experiment results show that the parallel mining algorithm has fine scaleup, sizeup and speedup.
7919733
Discovering direct and indirect matches for schema elements
Automating schema matching is challenging. Previous approaches to automating schema matching focus on computing direct element matches between two schemas. Schemas, however rarely match directly. Thus, to complete the task of schema matching, we must also compute indirect element matches. In this paper we present a framework for generating direct as well as many indirect element matches between a source schema and a target schema. Recognizing expected data values associated with schema elements and applying schema-structure heuristics are the key ideas to computing indirect matches. Experiments we have conducted over several real-world application domains show encouraging results, yielding over 90% precision and recall for both direct and indirect element matches.
3234740
Stochastic Ordering for Internet Congestion Control and its Applications
Window growth function for congestion control is a strong determinant of protocol behaviors, especially its second and higher-order behaviors associated with the distribution of transmission rates, its variances, and protocol stability. This paper presents a new stochastic tool, called convex ordering, that provides an ordering of any convex function of transmission rates of two protocols and valuable insights into high order behaviors of protocols. As the ordering determined by this tool is consistent with any convex function of rates, it can be applied to any unknown metric for protocol performance that consists of some high-order moments of transmission rates, as well as those already known such as rate variance. Using the tool, it is analyzed that a protocol with a growth function that starts off with a concave function and then switches to a convex function (e.g., an odd order function such as x3 and x5) around the maximum window size in the previous loss epoch, gives the smallest rate variation under a variety of network conditions. Among existing protocols, BIC and CUBIC have this window growth function. Experimental and simulation results confirm the analytical findings.
8691766
Media Streaming via TFRC: An Analytical Study of the Impact of TFRC on User-Perceived Media Quality
TCP-Friendly Rate Control (TFRC) is being adopted in Internet standards for congestion control of various streaming media applications. In this paper, we consider the transmission of pre-recorded media from a server to a client by using TFRC, and analytically study the impact of TFRC on user-perceived media quality, which is roughly measured by calculating the rebuffering probability. A rebuffering probability is defined to be the probability that the total duration of all rebuffering events experienced by a user is longer than a certain threshold. Two approaches are presented to help an application determine an appropriate initial buffering delay and media playback rate in order to achieve a certain rebuffering probability under a given network condition. First, we derive a closed- form expression to approximate the average TFRC sending rate, which could be used as the maximum allowed playback rate of a media stream. Second, we develop a queueing model for a TFRC client buffer with the traffic described by a Markov-Renewal- Modulated Deterministic Process (MRMDP), and present an iterative method to calculate the rebuffering probability.