The KRLS Tracker

In the previous section we offered a Bayesian interpretation of fixed-budget KRLS. Data was assumed to be stationary, i.e., $f({\boldsymbol{\mathbf{x}}})$ did not change over time. However, in a time-varying scenario, only recent samples have relevant information, whereas the information contained in older samples is actually misleading. In such a case, we would be interested in having a KRLS tracker that is able to forget past information and track changes in the target latent function.

In this section we are interested in developing a forgetting strategy and assess its effect throughout the whole input space, so we will work with complete GPs. We briefly remind the reader the GP notation: GPs are stochastic processes that are defined through a mean function $m({\boldsymbol{\mathbf{x}}})$ and a covariance function $c({\boldsymbol{\mathbf{x}}},{\boldsymbol{\mathbf{x}}}')$. To denote that $f({\boldsymbol{\mathbf{x}}})$ is a stochastic function drawn from a GP, we will use the notation $f({\boldsymbol{\mathbf{x}}})\sim\mathcal{GP}(m({\boldsymbol{\mathbf{x}}}), c({\boldsymbol{\mathbf{x}}},{\boldsymbol{\mathbf{x}}}'))$. Loosely speaking, one can think of a GP as a Gaussian distribution over an infinite set of points (evaluating $f({\boldsymbol{\mathbf{x}}})$ at every possible ${\boldsymbol{\mathbf{x}}}$ and thus building an infinitely long vector), with a corresponding infinitely long mean (given by the evaluation of $m({\boldsymbol{\mathbf{x}}})$ at every possible point) and an infinitely big covariance matrix (given by evaluating $c({\boldsymbol{\mathbf{x}}},{\boldsymbol{\mathbf{x}}}')$ at each possible pair of points). A complete background on GPs can be found in [11].