tokens: descent, that is good for learning acceleration, but can lead to missing the loss minimum), and max_depth (greater values correspond to a large complexity of the trees, and can lead to overfitting). Moreover, GB algorithms usually allow the use of a stop criterion, responsible for the termination of the learning when an overfitting occurs (the so-called, early_stopping parameter). It is expressed via the number of constructed trees, after which the quality of the metric no longer increases. Usually this parameter ranges between 10 and 1000 trees, depending on learning_rate . If the early stopping criterion is met, the GB algorithm accepts the number of trees, satisfied to the best score. Paying particular attention to the early_stopping parameter is the best way to avoid as much overfitting as possible. To quantify, how the quality of the classification changes over the iterations, for XGBoost and CatBoost, it is necessary to define a quality metric. Defining this metric allows one to express the changing of the classification quality against the complexity of the GBDT algorithm and can be easily used to control the overfitting. Widely used quality metrics are accuracy, precision, recall, and F1-score; however, these metrics are sensitive to the imbalance in number of training sources among different classes. Therefore, we decided to use Matthews correlation coefficient (MCC, Matthews 1975 ) which is instead insensitive to this imbalance. Finally, a good way to decrease the number of stars and galaxies classified by the algorithm as quasars is to apply a weight to the loss function of these two classes. In fact, this trick, applied to CatBoost, helped us in the paper to improve the final purity of the quasar selection with a minimal decrease in the completeness of the training set. In particular, we weighted the loss function for the STAR and GALAXY samples in the following way: where w i = 1 if source is a QSO and w i = 4 if source is STAR or GALAXY . A.2. Random Forest Another method of ensemble learning with decision trees is based on the use of an RF ( Breiman 2001 ) algorithm. This was the choice adopted in KiDS DR3 by N19. The basic idea of RF is that a set of decision trees can fit a robust classifier by averaging their decisions. From the performance side, the RF constructs a large number of decision trees and then uses the majority vote among them. The main pipeline of the single decision tree includes the following steps: (1) generation of different samples from the training set with the same size, but using random subsets of all the objects. The repetition of some objects among different subsamples is required to make the sample complete (so-called, random subsample with replacement); (2) let the decision tree learn on the “random subsample with replacement”, generated at step (1), but using randomly selected features. The RF therefore consists of many learning processes, each performed on a single decision tree using different
ner_tags: O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model I-Model O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model I-Model O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model O B-Model O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model I-Model O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Citation I-Citation O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Formula I-Formula I-Formula I-Formula O O O O O O B-Formula I-Formula I-Formula I-Formula O O O O O O O O B-Model I-Model O O O O O O O O O O O O O O O B-Model O B-Citation I-Citation O O O O O O O O B-Dataset I-Dataset O B-Citation O O O O B-Model O O O O O O O O O O O O O O O O O O O O O B-Model O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B-Model O O O O O O O O O O O O O O O