gpp_optimization_test

Contents:

gpp_optimization_test.hpp

Functions to test the optimization algorithms in gpp_optimization.hpp.

Calls optimizers with simple analytic functions (e.g., polynomials) with known optima and verifies that the optimizers can find the solutions.

namespace optimal_learning

Macro to allow restrict as a keyword for C++ compilation and CUDA/nvcc compilation. See related entry in gpp_common.hpp for more details.

gpp_optimization_test.cpp

Unit tests for the optimization algorithms in gpp_optimization.hpp. Currently we have tests for:

  1. restarted gradient descent (which uses gradient descent)
  2. newton

And each optimizer is tested against:

  1. a quadratic objective function

performing:

  1. Unconstrained optimization: optimal point is set away from domain boundaries
  2. Constrained optimization: optimal point is set outside the boundaries

on:

  1. tensor product domains

Generally the tests verify that:

  1. The expected optimum is actually an optimum (gradients are small).
  2. When started from the optimum value, the optimizer does not move away from it
  3. When started away from the optimum value, the optimizer finds it.

Where “finds it” is checked by seeing whether the point returned by the optimizer is within tolerance of the true optimum and whether the gradients are within tolerance of 0.

This is a little tricky for the constrained case b/c we are currently assuming we can compute the location of the true optimum directly... which may not always be possible.

TODO(GH-146): we should add some more complex objective functions (higher order polynomials and/or transcendentals) and simplex domains. We also need a more general way of testing constrained optimization since it is not always possible to directly compute the location of the optima.

TODO(GH-146): we have quite a bit of code duplication here. For the most part, the only difference is how we set up the optimizers in the beginning of each test function. This duplication could be reduced by encapsulating the optimizers in classes and then templating the testing functions on the optimizer type.

Defines

OL_TEMP_ERROR_PRINT

OL_TEMP_WARNING_PRINT

OL_ERROR_PRINT

OL_WARNING_PRINT

Variables

std::string exception_condition

“x == 1.0” to throw when current_point, aka x == 1.0 and “x != 1.0” to throw when x != 1.0

int dim_

std::vector< double > maxima_point_

namespace optimal_learning

Macro to allow restrict as a keyword for C++ compilation and CUDA/nvcc compilation. See related entry in gpp_common.hpp for more details.

Functions

int RunOptimizationTests()

Checks that the following optimizers are working correctly with simple objectives:

  • kGradientDescent
  • kNewton

by checking unconstrained and constrained optimization against polynomial objective function(s).

Also checks that MultistartOptimizer::MultistartOptimize() handles exceptions correctly and without crashing.

Returns:
number of test failures: 0 if optimizer is working properly