random forests PID
52876193
Collective behavior recognition using compact descriptors
This paper presents a novel hierarchical approach for collective behavior recognition based solely on ground plane trajectories. In the first layer of our classifier, we introduce a novel feature called Personal Interaction Descriptor (PID) which combines the spatial distribution of a pair of pedestrians within a temporal window with a pyramidal representation of the relative speed to detect pairwise interactions. These interactions are then combined with higher level features related to the mean speed and shape formed by the pedestrians in the scene, generating a Collective Behavior Descriptor (CBD) that is used to identify collective behaviors in a second stage. In both layers, Random Forests were used as classifiers, since they allow features of different natures to be combined seamlessly. Our experimental results indicate that the proposed method achieves results on par with state of the art techniques with a better balance of class errors. Moreover, we show that our method can generalize well across different camera setups through cross dataset experiments.
ArXiv
Gustavo Fuhr Claudio Rosito Jung
208748184
Random Forests machine learning applied to gas chromatography Mass spectrometry derived average mass spectrum data sets for classification and characterisation of essential oils.
Differences in chemical profiles of various essential oils (EOs) come from the fact that each plant species and chemotype has a distinctive secondary metabolism. Therefore, these differences can be used as the chemical markers for EO classification and determination of their quality. Herein, the Random Forests (RF) machine learning algorithm was applied to the classification of 20 different EOs. From three way raw gas chromatography mass spectra data, total chromatogram average mass spectra (TCAMS) and segment average mass spectra (SAMS) were created. TCAMS was generated by averaging response of each m/z over the whole chromatogram and SAMS by averaging the response of each fragment across a certain time segment within the chromatogram. The RF model was applied to the two data sets and optimised through the evaluation of pre processed data, number of trees, and number of variables used in each node split. The performance of the model was evaluated through a cross validation process, repeated 50 times by dividing the whole sample set into training and validation subsets. The calculated average out of bag error (OOBE) over 50 different training TCAMS data sets was 3.22 1.29% while for SAMS it was found to be 2.28 1.33% The minimal number of variables necessary for EO classification was determined by a nested cross validation process. The amount of reduced variables in each step was 10% It was shown that the TCAMS data set with 6 variables had similar prediction power as the SAMS with 30 variables. OOBE for classification of 20 EOs was 2.89 1.44% and 3.70 1.73% for TCAMS and SAMS, respectively. Proximity between samples was used to evaluate their qualities. Samples with greater intra class proximity had good similarity, while the lower ones indicated greater variations in the chemical profiles. The SAMS data set showed superior potential for quality assurance, compared with TCAMS.
Talanta
Leo Lebanov Laura Tedone Ali Reza Ghiasvand Brett Paull
41899688
Evaluation of variable selection methods for random forests and omics data sets
Abstract Machine learning methods and in particular random forests are promising approaches for prediction based on high dimensional omics data sets. They provide variable importance measures to rank predictors according to their predictive power. If building a prediction model is the main goal of a study, often a minimal set of variables with good prediction performance is selected. However, if the objective is the identification of involved variables to find active networks and pathways, approaches that aim to select all relevant variables should be preferred. We evaluated several variable selection procedures based on simulated data as well as publicly available experimental methylation and gene expression data. Our comparison included the Boruta algorithm, the Vita method, recurrent relative variable importance, a permutation approach and its parametric variant (Altmann) as well as recursive feature elimination (RFE) In our simulation studies, Boruta was the most powerful approach, followed closely by the Vita method. Both approaches demonstrated similar stability in variable selection, while Vita was the most robust approach under a pure null model without any predictor variables related to the outcome. In the analysis of the different experimental data sets, Vita demonstrated slightly better stability in variable selection and was less computationally intensive than Boruta. In conclusion, we recommend the Boruta and Vita approaches for the analysis of high dimensional data sets. Vita is considerably faster than Boruta and thus more suitable for large data sets, but only Boruta can also be applied in low dimensional settings.
Briefings Bioinform.
Frauke Degenhardt Stephan Seifert Silke Szymczak
155980866
A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources
Random forests (RF) is a supervised machine learning algorithm, which has recently started to gain prominence in water resources applications. However, existing applications are generally restricted to the implementation of Breiman's original algorithm for regression and classification problems, while numerous developments could be also useful in solving diverse practical problems in the water sector. Here we popularize RF and their variants for the practicing water scientist, and discuss related concepts and techniques, which have received less attention from the water science and hydrologic communities. In doing so, we review RF applications in water resources, highlight the potential of the original algorithm and its variants, and assess the degree of RF exploitation in a diverse range of applications. Relevant implementations of random forests, as well as related concepts and techniques in the R programming language, are also covered.
Water
Hristos Tyralis Georgia Papacharalampous Andreas Langousis
52170645
iForest: Interpreting Random Forests via Visual Analytics
As an ensemble model that consists of many independent decision trees, random forests generate predictions by feeding the input to internal trees and summarizing their outputs. The ensemble nature of the model helps random forests outperform any individual decision tree. However, it also leads to a poor model interpretability, which significantly hinders the model from being used in fields that require transparent and explainable predictions, such as medical diagnosis and financial fraud detection. The interpretation challenges stem from the variety and complexity of the contained decision trees. Each decision tree has its unique structure and properties, such as the features used in the tree and the feature threshold in each tree node. Thus, a data input may lead to a variety of decision paths. To understand how a final prediction is achieved, it is desired to understand and compare all decision paths in the context of all tree structures, which is a huge challenge for any users. In this paper, we propose a visual analytic system aiming at interpreting random forest models and predictions. In addition to providing users with all the tree information, we summarize the decision paths in random forests, which eventually reflects the working mechanism of the model and reduces users' mental burden of interpretation. To demonstrate the effectiveness of our system, two usage scenarios and a qualitative user study are conducted.
IEEE Transactions on Visualization and Computer Graphics
Xun Zhao Yanhong Wu Weiwei Cui
53501734
ABC random forests for Bayesian parameter inference
MOTIVATION Approximate Bayesian computation (ABC) has grown into a standard methodology that manages Bayesian inference for models associated with intractable likelihood functions. Most ABC implementations require the preliminary selection of a vector of informative statistics summarizing raw data. Furthermore, in almost all existing implementations, the tolerance level that separates acceptance from rejection of simulated parameter values needs to be calibrated. RESULTS We propose to conduct likelihood free Bayesian inferences about parameters with no prior selection of the relevant components of the summary statistics and bypassing the derivation of the associated tolerance level. The approach relies on the random forest (RF) methodology of Breiman (2001) applied in a (non parametric) regression setting. We advocate the derivation of a new RF for each component of the parameter vector of interest. When compared with earlier ABC solutions, this method offers significant gains in terms of robustness to the choice of the summary statistics, does not depend on any type of tolerance level, and is a good trade off in term of quality of point estimator precision and credible interval estimations for a given computing time. We illustrate the performance of our methodological proposal and compare it with earlier ABC methods on a Normal toy example and a population genetics example dealing with human population evolution. AVAILABILITY AND IMPLEMENTATION All methods designed here have been incorporated in the R package abcrf (version 1.7.1) available on CRAN. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.
Bioinform.
Louis Raynal Jean-Michel Marin Pierre Pudlo Mathieu Ribatet Christian P Robert Arnaud Estoup
196009937
Decision Trees and Random Forests
The capability to model unkown complex interactions between variables made machine learning a pervasive tool in bioinformatics and computational biology. Among others, decision trees and random forests have been used within the field for several successful applications achieveing high levels of performance. Furthermore, decision trees and random forests offer the possibility of inspecting the decision rules and to investigate the relevance of each variable, as well as, the dependencies among them. Here we review the theoretical foundations of both decision trees and random forests and illustrate basic case studies as well as applications from recent literature.
Encyclopedia of Bioinformatics and Computational Biology
Michele Fratello Roberto Tagliaferri
134381051
Geographical random forests: a spatial extension of the random forest algorithm to address spatial heterogeneity in remote sensing and population modelling
Abstract Machine learning algorithms such as Random Forest (RF) are being increasingly applied on traditionally geographical topics such as population estimation. Even though RF is a well performing and generalizable algorithm, the vast majority of its implementations is still 'aspatial' and may not address spatial heterogenous processes. At the same time, remote sensing (RS) data which are commonly used to model population can be highly spatially heterogeneous. From this scope, we present a novel geographical implementation of RF, named Geographical Random Forest (GRF) as both a predictive and exploratory tool to model population as a function of RS covariates. GRF is a disaggregation of RF into geographical space in the form of local sub models. From the first empirical results, we conclude that GRF can be more predictive when an appropriate spatial scale is selected to model the data, with reduced residual autocorrelation and lower Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) values. Finally, and of equal importance, GRF can be used as an effective exploratory tool to visualize the relationship between dependent and independent variables, highlighting interesting local variations and allowing for a better understanding of the processes that may be causing the observed spatial heterogeneity.
Geocarto International
Stefanos Georganos Tais Grippa Assane Niang Gadiaga Catherine Linard Moritz Lennert Sabine Vanhuysse Nicholus Mboga Eleonore Wolff Stamatis Kalogirou
89141
Random Forests
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, 148 156) but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
Machine Learning
Leo Breiman
61987452
ranger: A Fast Implementation of Random Forests for High Dimensional Data in C+ and R
We introduce the C+ application and R package ranger. The software is a fast implementation of random forests for high dimensional data. Ensembles of classification, regression and survival trees are supported. We describe the implementation, provide examples, validate the package with a reference implementation, and compare runtime and memory usage with other implementations. The new software proves to scale best with the number of features, samples, trees, and features tried for splitting. Finally, we show that ranger is the fastest and most memory efficient implementation of random forests to analyze data on the scale of a genome wide association study.
Marvin N Wright Andreas Ziegler