Standard KRLS (with evergrowing dictionary)

Assume a set of ordered input-output pairs ${\cal D}_t \equiv \{{\boldsymbol{\mathbf{x}}}_i, y_i\}_{i=1}^t$, where ${\boldsymbol{\mathbf{x}}}_i \in \mathbb{R}^D$ are $D$-dimensional input vectors and $y_i\in\mathbb{R}$ are scalar outputs. Data pairs are made available on a one-at-a-time basis, i.e., $({\boldsymbol{\mathbf{x}}}_t, y_t)$ is made available at time $t$. Our objective is to infer the predictive distribution of a new, unseen output $y_{t+1}$ given the corresponding input ${\boldsymbol{\mathbf{x}}}_{t+1}$ and data available up to time $t$, ${\cal D}_t$.