652801
The general inefficiency of batch training for gradient descent learning
Gradient descent training of neural networks can be done in either a batch or on-line manner. A widely held myth in the neural network community is that batch training is as fast or faster and/or more 'correct' than on-line training because it supposedly uses a better approximation of the true gradient for its weight updates. This paper explains why batch training is almost always slower than on-line training-often orders of magnitude slower-especially on large training sets. The main reason is due to the ability of on-line training to follow curves in the error surface throughout each epoch, which allows it to safely use a larger learning rate and thus converge with less iterations through the training data. Empirical results on a large (20,000-instance) speech recognition task and on 26 other learning tasks demonstrate that convergence can be reached significantly faster using on-line training than batch training, with no apparent difference in accuracy.
5584468
Improved Heterogeneous Distance Functions
Instance-based learning techniques typically handle continuous and linear input values well, but often do not handle nominal input attributes appropriately. The Value Difference Metric (VDM) was designed to find reasonable distance values between nominal attribute values, but it largely ignores continuous attributes, requiring discretization to map continuous values into nominal values. This paper proposes three new heterogeneous distance functions, called the Heterogeneous Value Difference Metric (HVDM), the Interpolated Value Difference Metric (IVDM), and the Windowed Value Difference Metric (WVDM). These new distance functions are designed to handle applications with nominal attributes, continuous attributes, or both. In experiments on 48 applications the new distance metrics achieve higher classification accuracy on average than three previous distance functions on those datasets that have both nominal and continuous attributes.
5596125
Preserving recommender accuracy and diversity in sparse datasets
Recommender systems combine research from user profiling, information filtering and artificial intelligence to provide users with more intelligent information access. They have proven to be useful in a range of Internet and e-commerce applications. Recent research has shown that a content-based (or case-based) perspective on collaborative filtering for recommendation can provide significant benefits in decision support accuracy over traditional collaborative techniques, particularly as dataset sparsity increases. These benefits derive both from the use of more sophisticated case-based similarity metrics and from the proactive maintenance of item similarity knowledge using data mining. This article presents a natural next step in this ongoing research to improve the quality of recommender systems by validating these findings in the context of more complex models of collaborative filtering, as well as by demonstrating that such techniques also preserve recommendation diversity, one of the key issues affecting traditional recommender systems.
815646
In-Depth Analysis of Similarity Knowledge and Metric Contributions to Recommender Performance
Initial successes in the area of recommender systems have led to considerable early optimism. However as a research community, we are still in the early days of our understanding of recommender systems. Evaluation metrics continue to be refined but we still need to account for the relative contributions of the various knowledge elements that play a part in the recommendation process. In this paper, we make a fine-grained analysis of a successful approach in the area of casebased recommendation, providing an ablation study of similarity knowledge and similarity metric contributions to improved system performance. We gauge the strengths and weaknesses of knowledge components and discuss future work as well as implications for re-
26290427
Knowledge Capture and Reuse for Geo-spatial Imagery Tasks
The continuously increasing amount and availability of geospatial image data is giving rise to problems of information overload in organisations that rely on digital geo-spatial imagery. Intelligent support for relevant image retrieval is needed in order to help manage large geospatial image libraries. Moreover, managing the knowledge implicit in using geo-spatial imagery to address particular tasks is crucial for capturing and making the most effective use of organisational knowledge assets. We are developing case-based knowledge-management support for large geo-spatial image repositories, which incorporates sketch-based querying for image retrieval; image manipulation and annotation tools for highlighting and composing relevant aspects of task-relevant imagery; and automatic context-based querying for retrieving relevant previous task experiences. This paper describes our approach to knowledge capture and reuse through task-based image annotation, and it introduces the environment we are developing for capture and reuse of task knowledge involving geo-spatial imagery.
11169287
Acquiring Case Adaptation Knowledge: A Hybrid Approach
The ability of case-based reasoning (CBR) systems to apply cases to novel situations depends on their case adaptation knowledge. However, endowing CBR systems with adequate adaptation knowledge has proven to be a very difficult task. This paper describes a hybrid method for performing case adaptation, using a combination of rule-based and case-based reasoning. It shows how this approach provides a framework for acquiring flexible adaptation knowledge from experiences with autonomous adaptation and suggests its potential as a basis for acquisition of adaptation knowledge from interactive user guidance. It also presents initial experimental results examining the benefits of the approach and comparing the relative contributions of case learning and adaptation learning to reasoning performance.
9824069
Data Mining Support for Case-Based Collaborative Recommendation
This paper describes ongoing research which aims to enhance collaborative recommendation techniques in the context of PTV, an applied recommender system for the TV listings domain. We have developed a case-based perspective on PTV's collaborative recommendation component, viewing the sparsity problem in collaborative filtering as one of updating and maintaining similarity knowledge for case-based systems. Our approach applies data mining techniques to extract relationships between program items that can be used to address the sparsity/ maintenance problem, as well as employing recommendation ranking that combines user similarities and item similarities to deliver more effective recommendation orderings.