Measures Against Overfitting

For most useful kernel functions, the dimension of the feature space, $m_k'$, will be much higher than the number of available data points, $N$. In such cases the kernel matrices $\textbf{K}_k$ do not have full rank and Eq. (6) could have an infinite number of solutions, representing an overfit problem. Various techniques to handle this overfitting have been presented.