% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/autoplot.BenchmarkAggr.R
\name{autoplot.BenchmarkAggr}
\alias{autoplot.BenchmarkAggr}
\title{Plots for BenchmarkAggr}
\usage{
\method{autoplot}{BenchmarkAggr}(
  object,
  type = c("mean", "box", "fn", "cd"),
  meas = NULL,
  level = 0.95,
  p.value = 0.05,
  minimize = TRUE,
  test = "nem",
  baseline = NULL,
  style = 1L,
  ratio = 1/7,
  col = "red",
  friedman_global = TRUE,
  ...
)
}
\arguments{
\item{object}{(\link{BenchmarkAggr})\cr
The benchmark aggregation object.}

\item{type}{\code{(character(1))} \cr Type of plot, see description.}

\item{meas}{\code{(character(1))} \cr Measure to plot, should be in \code{obj$measures}, can be \code{NULL} if
only one measure is in \code{obj}.}

\item{level}{\code{(numeric(1))} \cr Confidence level for error bars for \code{type = "mean"}}

\item{p.value}{\code{(numeric(1))} \cr What value should be considered significant for
\code{type = "cd"} and \code{type = "fn"}.}

\item{minimize}{\code{(logical(1))} \cr
For \code{type = "cd"}, indicates if the measure is optimally minimized. Default is \code{TRUE}.}

\item{test}{(\verb{character(1))}) \cr
For \code{type = "cd"}, critical differences are either computed between all learners
(\code{test = "nemenyi"}), or to a baseline (\code{test = "bd"}). Bonferroni-Dunn usually yields higher
power than Nemenyi as it only compares algorithms to one baseline. Default is \code{"nemenyi"}.}

\item{baseline}{\code{(character(1))} \cr
For \code{type = "cd"} and \code{test = "bd"} a baseline learner to compare the other learners to,
should be in \verb{$learners}, if \code{NULL} then differences are compared to the best performing
learner.}

\item{style}{\code{(integer(1))} \cr
For \code{type = "cd"} two ggplot styles are shipped with the package (\code{style = 1} or \code{style = 2}),
otherwise the data can be accessed via the returned ggplot.}

\item{ratio}{(\code{numeric(1)}) \cr
For \code{type = "cd"} and \code{style = 1}, passed to \code{\link[ggplot2:coord_fixed]{ggplot2::coord_fixed()}}, useful for quickly
specifying the aspect ratio of the plot, best used with \code{\link[=ggsave]{ggsave()}}.}

\item{col}{(\code{character(1)})\cr
For \code{type = "fn"}, specifies color to fill significant tiles, default is \code{"red"}.}

\item{friedman_global}{(\code{logical(1)})\cr
Should a friedman global test be performed for\code{type = "cd"} and \code{type = "fn"}?
If \code{FALSE}, a warning is issued in case the corresponding friedman posthoc test fails instead of an error.
Default is \code{TRUE} (raises an error if global test fails).}

\item{...}{\code{ANY} \cr Additional arguments, currently unused.}
}
\value{
The generated plot.
}
\description{
Generates plots for \link{BenchmarkAggr}, all assume that there are multiple, independent, tasks.
Choices depending on the argument \code{type}:
\itemize{
\item \code{"mean"} (default): Assumes there are at least two independent tasks. Plots the sample mean
of the measure for all learners with error bars computed with the standard error of the mean.
\item \code{"box"}: Boxplots for each learner calculated over all tasks for a given measure.
\item \code{"fn"}: Plots post-hoc Friedman-Nemenyi by first calling \link{BenchmarkAggr}\verb{$friedman_posthoc}
and plotting significant pairs in coloured squares and leaving non-significant pairs blank,
useful for simply visualising pair-wise comparisons.
\item \code{"cd"}: Critical difference plots (Demsar, 2006). Learners are drawn on the x-axis according
to their average rank with the best performing on the left and decreasing performance going
right. Any learners not connected by a horizontal bar are significantly different in performance.
Critical differences are calculated as:
\deqn{CD = q_{\alpha} \sqrt{\left(\frac{k(k+1)}{6N}\right)}}{CD = q_alpha sqrt(k(k+1)/(6N))}
Where \eqn{q_\alpha} is based on the studentized range statistic.
See references for further details.
It's recommended to crop white space using external tools, or function \code{image_trim()} from package \CRANpkg{magick}.
}
}
\examples{
if (requireNamespaces(c("mlr3learners", "mlr3", "rpart", "xgboost"))) {
library(mlr3)
library(mlr3learners)
library(ggplot2)

set.seed(1)
task = tsks(c("iris", "sonar", "wine", "zoo"))
learns = lrns(c("classif.featureless", "classif.rpart", "classif.xgboost"))
bm = benchmark(benchmark_grid(task, learns, rsmp("cv", folds = 3)))
obj = as_benchmark_aggr(bm)

# mean and error bars
autoplot(obj, type = "mean", level = 0.95)

if (requireNamespace("PMCMRplus", quietly = TRUE)) {
  # critical differences
  autoplot(obj, type = "cd",style = 1)
  autoplot(obj, type = "cd",style = 2)

  # post-hoc friedman-nemenyi
  autoplot(obj, type = "fn")
}

}

}
\references{
Dem≈°ar J (2006).
\dQuote{Statistical Comparisons of Classifiers over Multiple Data Sets.}
\emph{Journal of Machine Learning Research}, \bold{7}(1), 1-30.
\url{https://jmlr.org/papers/v7/demsar06a.html}.
}
