10174533
Cost-Sensitive Decision Tree Learning for Forensic Classification
In some learning settings, the cost of acquiring features for classification must be paid up front, before the classifier is evaluated. In this paper, we introduce the forensic classification problem and present a new algorithm for building decision trees that maximizes classification accuracy while minimizing total feature costs. By expressing the ID3 decision tree algorithm in an information theoretic context, we derive our algorithm from a well-formulated problem objective. We evaluate our algorithm across several datasets and show that, for a given level of accuracy, our algorithm builds cheaper trees than existing methods. Finally, we apply our algorithm to a real-world system, Clarify. Clarify classifies unknown or unexpected program errors by collecting statistics during program runtime which are then used for decision tree classification after an error has occurred. We demonstrate that if the classifier used by the Clarify system is trained with our algorithm, the computational overhead (equivalently, total feature costs) can decrease by many orders of magnitude with only a slight (<1%) reduction in classification accuracy.
1681184
Improved error reporting for software that uses black-box components
An error occurs when software cannot complete a requested action as a result of some problem with its input, configuration, or environment. A high-quality error report allows a user to understand and correct the problem. Unfortunately, the quality of error reports has been decreasing as software becomes more complex and layered. End-users take the cryptic error messages given to them by programsand struggle to fix their problems using search engines and support websites. Developers cannot improve their error messages when they receive an ambiguous or otherwise insufficient error indicator from a black-box software component.
 We introduce Clarify, a system that improves error reporting by classifying application behavior. Clarify uses minimally invasive monitoring to generate a behavior profile, which is a summary of the program's execution history. A machine learning classifier uses the behavior profile to classify the application's behavior, thereby enabling a more precise error report than the output of the application itself.
 We evaluate a prototype Clarify system on ambiguous error messages generated by large, modern applications like gcc, La-TeX, and the Linux kernel. For a performance cost of less than 1% on user applications and 4.7% on the Linux kernel, the proto type correctly disambiguates at least 85% of application behaviors that result in ambiguous error reports. This accuracy does not degrade significantly with more behaviors: a Clarify classifier for 81 La-TeX error messages is at most 2.5% less accurate than a classifier for 27 LaTeX error messages. Finally, we show that without any human effort to build a classifier, Clarify can provide nearest-neighbor software support, where users who experience a problem are told about 5 other users who might have had the same problem. On average 2.3 of the 5 users that Clarify identifies have experienced the same problem.
2738544
Estimating the global pagerank of web communities
Localized search engines are small-scale systems that index a particular community on the web. They offer several benefits over their large-scale counterparts in that they are relatively inexpensive to build, and can provide more precise and complete search capability over their relevant domains. One disadvantage such systems have over large-scale search engines is the lack of global PageRank values. Such information is needed to assess the value of pages in the localized search domain within the context of the web as a whole. In this paper, we present well-motivated algorithms to estimate the global PageRank values of a local domain. The algorithms are all highly scalable in that, given a local domain of size n, they use O(n) resources that include computation time, bandwidth, and storage. We test our methods across a variety of localized domains, including site-specific domains and topic-specific domains. We demonstrate that by crawling as few as n or 2n additional pages, our methods can give excellent global PageRank estimates.
2294366
The Correlated Correspondence Algorithm for Unsupervised Registration of Nonrigid Surfaces
We present an unsupervised algorithm for registering 3D surface scans of an object undergoing significant deformations. Our algorithm does not need markers, nor does it assume prior knowledge about object shape, the dynamics of its deformation, or scan alignment. The algorithm registers two meshes by optimizing a joint probabilistic model over all point-to-point correspondences between them. This model enforces preservation of local mesh geometry, as well as more global constraints that capture the preservation of geodesic distance between corresponding point pairs. The algorithm applies even when one of the meshes is an incomplete range scan; thus, it can be used to automatically fill in the remaining surfaces for this partial scan, even if those surfaces were previously only seen in a different configuration. We evaluate the algorithm on several real-world datasets, where we demonstrate good results in the presence of significant movement of articulated parts and non-rigid surface deformation. Finally, we show that the output of the algorithm can be used for compelling computer graphics tasks such as interpolation between two scans of a non-rigid object and automatic recovery of articulated object models.
3198534
Understanding Information Technology Complementarities Using an Augmented Endogenous Growth Framework
The complex relationships between information technology investments and organizational practices have been the focus of intensive research in recent years. The research focus appears to have shifted to investigating the effects of various organizational practices and their interactions with information technology capital. There also is emerging evidence of recent emphasis on organizational factors and a greater shift toward IT complementarities in which value addition is linked to combining complementary organizational practices with IT investments. In this paper we focus on the contributions of IT use and organizational practices and their interactions. We use firm-level data to extend previous studies in three ways: (1) by using a large sample covering 3,299 firms spanning 11 industries, (2) by focusing on a large number of organizational practices, and (3) by augmenting the endogenous growth accounting framework with a data mining technique. Our findings indicate that the set of interrelated organizational practices that complement positively to IT use is different from the set of practices hindering IT use. We conclude that IT complementary factors that positively affect productivity growth do not necessarily have a reverse effect when they are reduced or removed. It appears that some organizational factors have asymmetrical effects on growth.
16674358
Change of Representation for Statistical Relational Learning
Statistical relational learning (SRL) algorithms learn statistical models from relational data, such as that stored in a relational database. We previously introduced view learning for SRL, in which the view of a relational database can be automatically modified, yielding more accurate statistical models. The present paper presents SAYU-VISTA, an algorithm which advances beyond the initial view learning approach in three ways. First, it learns views that introduce new relational tables, rather than merely new fields for an existing table of the database. Second, new tables or new fields are not limited to being approximations to some target concept; instead, the new approach performs a type of predicate invention. The new approach avoids the classical problem with predicate invention, of learning many useless predicates, by keeping only new fields or tables (i.e., new predicates) that immediately improve the performance of the statistical model. Third, retained fields or tables can then be used in the definitions of further new fields or tables. We evaluate the new view learning approach on three relational classification tasks.
31090559
The SSP: an example of high-assurance systems engineering
The SSP is a high assurance systems engineering effort spanning both hardware and software. Extensive design review, first principle design, n-version programming, program transformation, verification, and consistency checking are the techniques used to provide assurance in the correctness of the resulting system.
4850818
Analysis of Weblog Link Structure - A Community Perspective
In this paper, we report a two level study on weblog link structures. At the micro level, we carried out an in-depth investigation of individual weblogs. Our goal was to obtain some preliminary understanding of the different types of links that might indicate underlying communities of bloggers. Complete and detailed link data was collected from eight weblogs followed by a variety of analyses. The result shows that both incoming and outgoings follow Zipf like distribution in terms of the sources of those links. These suggest clustering patterns (communities) within the whole blogspace. We also examine the temporal aspects of weblogs. The average life span of a weblog entry is fairly long in most of our sample cases. In addition, analysis on individual comment authors shows that in average, active comment authors maintain a rather long relationship with a certain weblog. It provides evidence that historical data may be useful in understanding weblog communities. On a larger scale, we developed a program to collect complete link data from large number of interconnected weblogs and performed cluster analysis on it. Communities with common topics are successfully extracted using those link data.