gpp_mock_optimization_objective_functions

Contents:

gpp_mock_optimization_objective_functions.hpp

This file contains mock objective functions interfaces for use with optimization routines. This is solely for unit testing. Instead of testing gradient descent against log marginal likelihood with some random set of data (which is an integration test), we would instead like to be able to test gradient descent on something easier to understand, e.g., z = -x^2 - y^2. These simpler functions have analytic optima which makes testing optimizers (e.g., gradient descent, newton) much easier.

See the header comments in gpp_optimization.hpp, Section 3a), for further details on what it means for an Evaluator class to be “optimizable.”

Since perfomance is irrelevant for these test functions, we will define pure abstract Evaluator classes that can be used to test optimizers in gpp_optimization.hpp. Generally the usage should be to subclass say SimpleObjectiveFunctionEvaluator and override the pure virtuals. Then we only end up compiling one version of the [templated] optimization code for running tests. See gpp_optimization_test.cpp for examples.

Following the style laid out in gpp_common.hpp (file comments, item 5), we currently define:

  • class SimpleObjectiveFunctionEvaluator;
  • struct SimpleObjectiveFunctionState;

SimpleObjectiveFunctionEvaluator defines a pure abstract base class with interface consistent with the interface that all .*Evaluator classes must provide (e.g., ExpectedImprovementEvaluator, LogMarginalLikelihoodEvaluator).

SimpleObjectiveFunctionState is simple: it’s just a container class that holds a point at which to evaluate the polynomial.

namespace optimal_learning

Macro to allow restrict as a keyword for C++ compilation and CUDA/nvcc compilation. See related entry in gpp_common.hpp for more details.

class SimpleObjectiveFunctionEvaluator

Class to evaluate the function f(x_1,...,x_{dim}) = -\sum_i (x_i - s_i)^2, i = 1..dim. This is a simple quadratic form with maxima at (s_1, ..., s_{dim}).

Public Type

typedef SimpleObjectiveFunctionState StateType

Public Functions

~SimpleObjectiveFunctionEvaluator()

int dim()

double GetOptimumValue()

Helpful for testing so we know what the optimum is. This value should be the result of:

GetOptimumPoint(point);
state.SetCurrentPoint(point);
optimum_value = ComputeObjectiveFunction(state);

Then optimum_value == GetOptimumValue().

Returns:
the optimum value of the polynomial.

void GetOptimumPoint(double *restrict point)

Helpful for testing so we know where the optimum (value returned by GetOptimumValue) occurs.

Note

if the optimal point is not unique, this function may return one arbitrarily.

input
point[dim]:space to write the output
OUTPUTS:
point[dim]:the point at which the polynomial obtains its optimum value

double ComputeObjectiveFunction(StateType * quadratic_dummy_state)

Compute the quadratic objective function: f(x_1,...,x_{dim}) = -\sum_i (x_i - s_i)^2.

Parameters:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType (e.g., SimpleObjectiveFunctionState)
Outputs:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType; only temporary state may be mutated
Returns:
the value of the objective at quadratic_dummy_state.GetCurrentPoint()

void ComputeGradObjectiveFunction(StateType * quadratic_dummy_state, double *restrict grad_polynomial)

Compute the gradient of the objective function: f'(x_1,...,x_{dim})_i = -2 * (x_i - s_i).

Parameters:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType (e.g., SimpleObjectiveFunctionState)
Outputs:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType; only temporary state may be mutated grad_polynomial[dim]: gradient of the objective

void ComputeHessianObjectiveFunction(StateType * quadratic_dummy_state, double *restrict hessian_polynomial)

Compute the gradient of the objective function: f''(x_1,...,x_{dim})_{i,j} = -2 * \delta_{i,j}.

Parameters:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType (e.g., SimpleObjectiveFunctionState)
Outputs:
quadratic_dummy_state[1]: ptr to a FULLY CONFIGURED StateType; only temporary state may be mutated hessian_polynomial[dim][dim]: hessian of the objective

OL_DISALLOW_COPY_AND_ASSIGN(SimpleObjectiveFunctionEvaluator)

Protected Functions

SimpleObjectiveFunctionEvaluator()

class SimpleObjectiveFunctionState

Public Type

typedef SimpleObjectiveFunctionEvaluator EvaluatorType

Public Functions

SimpleObjectiveFunctionState(const EvaluatorType & quadratic_eval, double const *restrict current_point_in)

SimpleObjectiveFunctionState(SimpleObjectiveFunctionState && OL_UNUSED)

int GetProblemSize()

void GetCurrentPoint(double *restrict current_point_in)

void SetCurrentPoint(const EvaluatorType & OL_UNUSED, double const *restrict current_point_in)

OL_DISALLOW_DEFAULT_AND_COPY_AND_ASSIGN(SimpleObjectiveFunctionState)

Public Members

const int dim

spatial dimension (e.g., entries per point of points_sampled)

std::vector< double > current_point

the point at which to evaluate the associated objective