5844132
Convergence results for the EM approach to mixtures of experts architectures
The Expectation-Maximization (EM) algorithm is an iterative approach to maximum likelihood parameter estimation. Jordan and Jacobs (1993) recently proposed an EM algorithm for the mixture of experts architecture of Jacobs, Jordan, Nowlan and Hinton (1991) and the hierarchical mixture of experts architecture of Jordan and Jacobs (1992). They showed empirically that the EM algorithm for these architectures yields significantly faster convergence than gradient ascent. In the current paper we provide a theoretical analysis of this algorithm. We show that the algorithm can be regarded as a variable metric algorithm with its searching direction having a positive projection on the gradient of the log likelihood. We also analyze the convergence of the algorithm and provide an explicit expression for the convergence rate. In addition, we describe an acceleration technique that yields a significant speedup in simulation experiments.
6958967
Dual multivariate auto-regressive modeling in state space for temporal signal separation
Many existing independent component analysis (ICA) approaches result in deteriorated performance in temporal source separation because they have not taken into consideration of the underlying temporal structure of sources. In this paper, we model temporal sources as a general multivariate auto-regressive (AR) process whereby an underlying multivariate AR process in observation space is obtained. In this dual AR modeling, the mixing process from temporal sources to observations is the same as the mixture from the nontemporal residuals of the source AR (SAR) process to that of the observation AR (OAR) process. We can therefore avoid the source temporal effects in performing ICA by learning the demixing system on the independently distributed OAR residuals rather than the time-correlated observations. Particularly, we implement this approach by modeling each source signal as a finite mixture of generalized autoregressive conditional heteroskedastic (GARCH) process. The adaptive algorithms are proposed to extract the OAR residuals appropriately online, together with learning the demixing system via a nontemporal ICA algorithm. The experiments have shown its superior performance on temporal source separation.
16709558
One-Bit-Matching Conjecture for Independent Component Analysis
The one-bit-matching conjecture for independent component analysis (ICA) could be understood from different perspectives but is basically stated as all the sources can be separated as long as there is a one-toone same-sign-correspondence between the kurtosis signs of all source probability density functions (pdf's) and the kurtosis signs of all model pdf's (Xu, Cheung, & Amari, 1998a). This conjecture has been widely believed in the ICA community and implicitly supported by many ICA studies, such as the Extended Infomax (Lee, Girolami, & Sejnowski, 1999) and the soft switching algorithm (Welling & Weber, 2001). However, there is no mathematical proof to confirm the conjecture theoretically. In this article, only skewness and kurtosis are considered, and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes. Moreover, empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks. As a by-product, we also show that the kurtosis maximization criterion (Moreau & Macchi, 1996) is actually a special case of the minimum mutual information criterion for ICA.
11164646
Strip line detection and thinning by RPCL-based local PCA
We solve the tasks of strip line detection and thinning in image processing and pattern recognition with the help of a statistical learning technique called rival penalized competitive learning based local principal component analysis. Due to its model selection and noise resistance ability, the technique is experimentally shown to outperform conventional Hough transform and thinning algorithms.
37879675
A new curve detection method: Randomized Hough transform (RHT)
Abstract A new method is proposed for curve detection. For a curve with n parameters, instead of transforming one pixel into a hypersurface of the n -D parameter space as the HT and its variants do, we randomly pick n pixels and map them into one point in the parameter space. In comparison with the HT and its variants, our new method has the advantages of small storage, high speed, infinite parameter space and arbitrarily high resolution. The preliminary experiments have shown that the new method is quite effective.
15052539
A Further Result on the ICA One-Bit-Matching Conjecture
The one-bit-matching conjecture for independent component analysis (ICA) has been widely believed in the ICA community. Theoretically, it has been proved that under the assumption of zero skewness for the model probability density functions, the global maximum of a cost function derived from the typical objective function on the ICA problem with the one-bit-matching condition corresponds to a feasible solution of the ICA problem. In this note, we further prove that all the local maximums of the cost function correspond to the feasible solutions of the ICA problem in the two-source case under the same assumption. That is, as long as the one-bit-matching condition is satisfied, the two-source ICA problem can be successfully solved using any local descent algorithm of the typical objective function with the assumption of zero skewness for all the model probability density functions.
7919733
Discovering direct and indirect matches for schema elements
Automating schema matching is challenging. Previous approaches to automating schema matching focus on computing direct element matches between two schemas. Schemas, however rarely match directly. Thus, to complete the task of schema matching, we must also compute indirect element matches. In this paper we present a framework for generating direct as well as many indirect element matches between a source schema and a target schema. Recognizing expected data values associated with schema elements and applying schema-structure heuristics are the key ideas to computing indirect matches. Experiments we have conducted over several real-world application domains show encouraging results, yielding over 90% precision and recall for both direct and indirect element matches.
303543
Limitations of Equation-Based Congestion Control
We study limitations of an equation-based congestion control protocol, called TCP-friendly rate control (TFRC). It examines how the three main factors that determine TFRC throughput, namely, the TCP-friendly equation, loss event rate estimation, and delay estimation, can influence the long-term throughput imbalance between TFRC and TCP. Especially, we show that different sending rates of competing flows cause these flows to experience different loss event rates. There are several fundamental reasons why TFRC and TCP flows have different average sending rates, from the first place. Earlier work shows that the convexity of the TCP-friendly equation used in TFRC causes the sending rate difference. We report two additional reasons in this paper: 1) the convexity of where is a loss event period and 2) different retransmission timeout period (RTO) estimations of TCP and TFRC. These factors can be the reasons for TCP and TFRC to experience initially different sending rates. But we find that the loss event rate difference due to the differing sending rates greatly amplifies the initial throughput difference; in some extreme cases, TFRC uses around 20 times more, or sometimes 10 times less, bandwidth than TCP. Despite these factors influencing the throughput difference, we also find that simple heuristics can greatly mitigate the problem.
3234740
Stochastic Ordering for Internet Congestion Control and its Applications
Window growth function for congestion control is a strong determinant of protocol behaviors, especially its second and higher-order behaviors associated with the distribution of transmission rates, its variances, and protocol stability. This paper presents a new stochastic tool, called convex ordering, that provides an ordering of any convex function of transmission rates of two protocols and valuable insights into high order behaviors of protocols. As the ordering determined by this tool is consistent with any convex function of rates, it can be applied to any unknown metric for protocol performance that consists of some high-order moments of transmission rates, as well as those already known such as rate variance. Using the tool, it is analyzed that a protocol with a growth function that starts off with a concave function and then switches to a convex function (e.g., an odd order function such as x3 and x5) around the maximum window size in the previous loss epoch, gives the smallest rate variation under a variety of network conditions. Among existing protocols, BIC and CUBIC have this window growth function. Experimental and simulation results confirm the analytical findings.
39559674
A framework for Web applications testing
Web application testing is concerned with numerous and complicated testing objects, methods and processes. So a testing framework fitting for the properties of Web application is needed to guide and organize all the testing tasks. Based on the analysis for Web application characters and traditional software testing process, the process for Web application testing is modeled, which describes a series of testing flows such as the testing requirement analysis, test cases generation and selection, testing execution, and testing results analysis and measurement. Furthermore, the realization techniques are also investigated so as to integrate each testing step and implement the whole testing process harmoniously and effectively. Thus the framework is suitable for the Internet environment and can guide the Web application testing actively and availably.
32570723
Parallel algorithm for mining fuzzy association rules
The principle and steps of the algorithm for mining fuzzy association rules is studied, and the parallel algorithm for mining fuzzy association rules is presented. In this parallel mining algorithm, quantitative attributes are partitioned into several fuzzy sets by the parallel fuzzy c-means algorithm, and fuzzy sets are applied to soften the partition boundary of the attributes. Then, the parallel algorithm for mining Boolean association rules is improved to discover frequent fuzzy attributes. Last, the fuzzy association rules with at least fuzzy confidence are generated on all processors. The parallel mining algorithm is implemented on the distributed linked PC/workstation. The experiment results show that the parallel mining algorithm has fine scaleup, sizeup and speedup.