"Or first detector is the Kernel MMD drift detector. As you may have guessed from the name, it uses a kernel to define a metric on the space of distributions on the feature-space (see our [note on the intuition behind MMD](./note_on_mmd.ipynb). TorchDrift implements a few kernels in the `detectors.mmd` module, the `GaussianKernel` (also known as squared exponential) is the default, `ExpKernel` (aka Laplacian Kernel) and `RationalQuadraticKernel` are also available.\n",

"Finally we use the Untrained Autoencoder. This is a bit of a funny name because it really half an autoencode, so we might as well call it a untrained or randomly initialized feature extractor. This performed reasonably well in _Failing Loudly_, so it appears relatively frequently.\n",

"For drift detection, we need a feature extractor (of course, we had one above, too, but let's play along and pretend we got the `model` from our colleague."

"These are not exclusive, e.g. when the class balance changes without the class-conditional distributions changing, you would observe both covarite and label shift.\n",

"There are several things we would want to do that are related to drift:\n",

"- In _domain adaptation_ we try to make models which can cope with the new distribution $P$.\n",

"- In _drift detection_ (here) we are interested in detecting whether drift has happened. It will be hard to deal with concept drift because we typically do not have access to $Y$ from $P(Y)$, but we will see what can be done about the others.\n",

"There are two main ways to do do the dimension reduction.\n",

"If $X$ at the very top is not a good candidate, so we might use any other - the features, the scores, the class probabiities or (at least we might conceptually) the predicted class.\n",

"As the p-value gives us the expected rate of \"false alarms\" if the null hypotheses _no drift_ remains valid, just leaning on it does leave something to be desired in terms of calibrating detection rates. In many practical applications, we may not have samples of drifted data, so we have to make to with the p-value only. Typically we would choose rather low p-values. If we do have access to (potentially fabricated) samples from the drifted set, we can extend our analysis to consider the [receiver operating characteristic (ROC) curve](https://en.wikipedia.org/wiki/Receiver_operating_characteristic), so we can trade off the false positive rate against the statistical sensitivity of the detector. Examples of this analysis are provided in [drift detection on images](./drift_detection_on_images.ipynb).\n",

"Another approach to the dirft detection problem can be to try to classify samples into coming from $P_{ref}$ or $P$, respectively. If we cannot train a classifier that works better than random, we may conclude that $P_{ref}$ and $P$ are indistinguishable. This may sound very familiar, because it also is at the core of the Generative Adversarial Network\n",

"The other route is to try to give a threshold for a given confidence level or equivalently to convert $\\widehat{MMD}$ values (given the sample sizes) into $p$-values. We could try to derived expressions for this (and earlier papers of the same group of authors do, see the link above), but the conceptually easiest way is to sample from the distribution $\\widehat{MMD}$ under the null-hypothesis using bootstrapping. In this technique, we approximate sampling from the null-hypothesis by shuffeling between the $X_\\cdot$ and $Y_\\cdot$, so that both the $x$ and $y$ argument come from the same distribution. If one sample is sufficiently large (e.g. the training sample), we might compute the thresholds just on that, too.\n",