42772317
Density estimation by stochastic complexity
The results by P. Hall and E.J. Hannan (1988) on optimization of histogram density estimators with equal bin widths by minimization of the stochastic complexity are extended and sharpened in two separate ways. As the first contribution, two generalized histogram estimators are constructed. The first has unequal bin widths which, together with the number of the bins, are determined by minimization of the stochastic complexity using dynamic programming. The other estimator consists of a mixture of equal bin width estimators, each of which is defined by the associated stochastic complexity. As the main contribution in the present work, two theorems are proved, which together extend the universal coding theorems to a large class of data generating densities. The first gives an asymptotic upper bound for the code redundancy in the order of magnitude, achieved with a special predictive type of histogram estimator, which sharpens a related bound. The second theorem states that this bound cannot be improved upon by any code whatsoever. >
39243731
A rate of convergence result for a universal D-semifaithful code
The problem of optimal rate universal coding is considered in the context of rate-distortion theory. A D-semifaithful universal coding scheme for discrete memoryless sources is given. The main result is a refined covering lemma based on the random coding argument and the method of types. The average codelength of the code is shown to approach its lower bound, the rate-distortion function, at a rate O(n/sup -1/log n), and this is conjectured to be optimal based on a result of A.J. Pilc (1968). Issues of constructiveness and universality are addressed. >
2174351
On Model Selection Consistency of Lasso
Sparsity or parsimony of statistical models is crucial for their proper interpretations, as in sciences and social sciences. Model selection is a commonly used method to find such models, but usually involves a computationally heavy combinatorial search. Lasso (Tibshirani, 1996) is now being used as a computationally feasible alternative to model selection. Therefore it is important to study Lasso for model selection purposes. 
 
In this paper, we prove that a single condition, which we call the Irrepresentable Condition, is almost necessary and sufficient for Lasso to select the true model both in the classical fixed p setting and in the large p setting as the sample size n gets large. Based on these results, sufficient conditions that are verifiable in practice are given to relate to previous works and help applications of Lasso for feature selection and sparse representation. 
 
This Irrepresentable Condition, which depends mainly on the covariance of the predictor variables, states that Lasso selects the true model consistently if and (almost) only if the predictors that are not in the true model are "irrepresentable" (in a sense to be clarified) by predictors that are in the true model. Furthermore, simulations are carried out to provide insights and understanding of this result.
1825087
The Minimum Description Length Principle in Coding and Modeling
We review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon's basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. We assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference. Context tree modeling, density estimation, and model selection in Gaussian linear regression serve as examples.
15189341
Wavelet thresholding for multiple noisy image copies
This correspondence addresses the recovery of an image from its multiple noisy copies. The standard method is to compute the weighted average of these copies. Since the wavelet thresholding technique has been shown to effectively denoise a single noisy copy, we consider in this paper combining the two operations of averaging and thresholding. Because thresholding is a nonlinear technique, averaging then thresholding or thresholding then averaging produce different estimators. By modeling the signal wavelet coefficients as Laplacian distributed and the noise as Gaussian, our investigation finds the optimal ordering to depend on the number of available copies and on the signal-to-noise ratio. We then propose thresholds that are nearly optimal under the assumed model for each ordering. With the optimal and near-optimal thresholds, the two methods yield similar performance, and both show considerable improvement over merely averaging.
1767373
Wavelet thresholding via MDL for natural images
We study the application of Rissanen's (1989) principle of minimum description length (MDL) to the problem of wavelet denoising and compression for natural images. After making a connection between thresholding and model selection, we derive an MDL criterion based on a Laplacian model for noiseless wavelet coefficients. We find that this approach leads to an adaptive thresholding rule. While achieving mean-squared-error performance comparable with other popular thresholding schemes, the MDL procedure tends to keep far fewer coefficients. From this property, we demonstrate that our method is an excellent tool for simultaneous denoising and compression. We make this claim precise by analyzing MDL thresholding in two optimality frameworks; one in which we measure rate and distortion based on quantized coefficients and one in which we do not quantize, but instead record rate simply as the number of nonzero coefficients.
33239543
Implementing KDB-trees to support high-dimensional data
The problem of retrieving large volumes of high dimensional data is an important and timely issue in the area of database management. The guiding idea of the paper is to develop a general-purpose point access method that attacks the limitations of KDB-trees in high-dimensional spaces, while preserving their relatively good performance in low-dimensional situations. The proposed structure, called high-dimensional KDB-tree, eliminates downward propagation of splits associated with the original KDB-tree structure, which results in low storage utilization and rapid deterioration of the retrieval performance. Additional improvements in the storage and retrieval performance are achieved by removing certain redundant information from the interior nodes. Experimental results show that, in high-dimensional spaces, the proposed structure outperforms the original KDB-trees by a significant margin, while incurring no loss of performance in low-dimensional spaces. The structure also outperforms two other variants of KDB-trees investigated in the paper.
33931538
Dynamic Splitting Policies of the Adaptive 3DR-Tree for Indexing Continuously Moving Objects
Moving-objects databases need a spatio-temporal indexing scheme for moving objects to efficiently process queries over continuously changing locations of the objects. A simple extension of the R-tree that employs time as the third dimension of the data universe shows low space utilization and poor search performance because of overlapping index regions. In this paper, we propose a variant of the 3-dimensional R-tree called the Adaptive 3DR-tree. The dynamic splitting policies of the Adaptive 3DR-tree significantly reduce the overlap rate, and this, in turn, results in improved query performance. The results of our extensive experiments show that the Adaptive 3DR-tree outperforms the original 3D R-tree and the TB-tree typically by a big margin.
17844386
A retrieval technique for high-dimensional data and partially specified queries
While the persistent data of many advanced database applications, such as OLAP and scientific studies, are characterized by very high dimensionality, typical queries posed on these data appeal to a small number of relevant dimensions. Unfortunately, the multi-dimensional access methods designed for high-dimensional data perform rather poorly for these partially specified queries. The retrieval technique proposed in this paper uses a combination of two complementary measures to support efficient processing of partial queries over high-dimensional data. First, an elaborate storage organization, called the inverted space, allows the system administrator to control the size of individual indexes in order to avoid the negative impact of extremely high data dimensionality on the retrieval performance. Second, a new indexing structure, which is designed to support the inverted-space storage organization, enables efficient query processing in projected spaces with moderate dimensionality. This indexing mechanism is a general-purpose point access method that effectively attacks the limitations of KDB-trees in spaces with many dimensions, while preserving the simplicity and relatively good performance of the later structure in low-dimensional spaces. The analytical and experimental results show that the new indexing scheme outperforms two other variants of KDB-trees investigated in the paper for both fully and partially specified queries.
1656491
An integrated token-based algorithm for scalable coordination
Efficient coordination among large numbers of heterogeneous agents promises to revolutionize the way in which some complex tasks, such as responding to urban disasters can be performed. However, state of the art coordination algorithms are not capable of achieving efficient and effective coordination when a team is very large. Building on recent successful token-based algorithms for task allocation and information sharing, we have developed an integrated and efficient approach to effective coordination of large scale teams. We use tokens to encapsulate anything that needs to be shared by the team, including information, tasks and resources. The tokens are efficiently routed through the team via the use of local decision theoretic models. Each token is used to improve the routing of other tokens leading to a dramatic performance improvement when the algorithms work together. We present results from an implementation of this approach which demonstrates its ability to coordinate large teams.
43033576
The node multiterminal cut polyhedron