com.linkedin.photon.ml.hyperparameter.estimators
kernels
Base trait for covariance kernel functions
Implements the Matérn 5/2 covariance kernel.
Implements the Matérn 5/2 covariance kernel.
The Matern kernel is a generalization of the RBF kernel with an additional parameter $\nu$ that allows controlling smoothness. At $\nu = \infty$, the Matern kernel is equivalent to RBF. At $\nu = 0.5$, it's equivalent to the absolute exponential kernel. It's noted in the literature that $\nu = 2.5$ allows the kernel to closely approximate hyperparameter spaces where the smoothness of RBF causes issues (see PBO). Here we hard-code to the 5/2 value because the computation is much simpler than allowing a user-defined $\nu$.
$K(x,x') = \big(\sqrt{5r2(x,x')} + \frac{5}{3} r2(x,x') + 1\big) \exp(-\sqrt{5r^2(x,x')})$
Where $r(x,x')$ is the Euclidean distance between $x$ and $x'$.
"Practical Bayesian Optimization of Machine Learning Algorithms" (PBO), https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf
"Gaussian Processes for Machine Learning" (GPML), http://www.gaussianprocess.org/gpml/, Chapter 4
Implements the radial basis function (RBF) kernel.
Implements the radial basis function (RBF) kernel.
$K(x,x') = \exp(-\frac{1}{2} r(x,x')^2)$
Where $r(x,x')$ is the Euclidean distance between $x$ and $x'$.
Base trait for stationary covariance kernel functions
Base trait for stationary covariance kernel functions
Stationary kernels depend on the relative positions of points (e.g. distance), rather than on their absolute positions.
Base trait for covariance kernel functions
In Gaussian processes estimators and models, the covariance kernel determines the similarity between points in the space. We assume that similarity in domain entails similarity in range, hence the kernel also encodes our prior assumptions about how the function behaves.
"Gaussian Processes for Machine Learning" (GPML), http://www.gaussianprocess.org/gpml/, Chapter 4