com.linkedin.photon.ml.hyperparameter
search
Performs a guided random search of the given ranges, where the search is guided by a Gaussian Process estimated from evaluations of the actual evaluation function.
Performs a random search of the bounded space.
Performs a guided random search of the given ranges, where the search is guided by a Gaussian Process estimated from evaluations of the actual evaluation function. Since we assume that the evaluation function is very costly (as it often is for doing a full train / cycle evaluation of a machine learning model), it makes sense to spend time doing what would otherwise be considered an expensive computation to reduce the number of times we need to evaluate the function.
At a high level, the search routine proceeds as follows:
1) Assume a uniform prior over the evaluation function 2) Receive a new observation, and use it along with any previous observations to train a new Gaussian Process regression model for the evaluation function. This approximation is the new posterior over the evaluation function. 3) Sample candidates uniformly, evaluate the posterior for each, and select the candidate with the highest predicted evaluation. 4) Evaluate the best candidate with the actual evaluation function to acquire a new observation. 5) Repeat from step 2.