There are many algorithms to estimate the density of probability distributions. I am looking for one that is not sensitive to the initial condition. For instance, Expectation–maximization algorithm starts by initializing the parameters theta to some random values and then it goes from there to other steps. So for different theta one might obtain two different distributions estimate. Are there algorithms that do not have this dependency? in other words, it only depends on the samples from which we are trying to estimate the density?



a probability density algorithm that is not sensitive to the initial condition
The result obtained by a local numerical optimization solver applied to a non-convex optimization problem is sensitive to the starting value used by the optimizer. Expectation-maximization as applied to find a local optimum of a non-convex maximum likelihood estimation problem is one such example of a local optimization solver.


If you use a "rigorous" (for example, based on branch and bound) numerical global optimization solver, and the problem is solved to global optimality (which may require non-trivial run-time and maybe memory for problems with many parameters to estimate), then the optimal objective value (measure of fit being optimized), will be independent of starting value. The optimal parameter values will also be independent of starting value provided that there is a unique global optimum. In reality, problems are solved to within some specified relative or absolute tolerance gap on the objective value, and the problem is easier to 'solve", the larger the specified gap (non-dependence on starting value only then applies relative to getting some solution within the specified gap, but the solution obtained and the actual gap achieved may depend on the starting value). In contrast, a local optimization solver provides no information as to how close the solution found is in objective value to the global optimum.


BARON is one such rigorous numerical global optimization optimizer (it is not totally rigorous, given that it can still be fooled by very badly numerically behaved functions which overtax its internal precision). The main issue with it is that run-time can be very long on some problems - that's why local optimizers are still used.


