moe_examples package

Submodules

moe_examples.bandit_example module

An example for accessing the all the types in moe/easy_interface/bandit_simple_endpoint.

The following function is used:

See Multi-Armed Bandits for more details on multi-armed bandits. The function requires some historical information to inform bandit.

We compute arm allocations for all bandit type and subtypes with the simple example of Bernoulli arms.

moe_examples.bandit_example.run_example(verbose=True, testapp=None, bandit_bla_kwargs=None, bandit_epsilon_kwargs=None, bandit_ucb_kwargs=None, **kwargs)[source]

Run the bandit example.

Parameters:
  • verbose (bool) – Whether to print information to the screen [True]
  • testapp (Pyramid test application) – Whether to use a supplied test pyramid application or a rest server [None]
  • bandit_bla_kwargs (dict) – Optional kwargs to pass to bandit_bla endpoint
  • bandit_epsilon_kwargs (dict) – Optional kwargs to pass to bandit_epsilon endpoint
  • bandit_ucb_kwargs (dict) – Optional kwargs to pass to bandit_ucb endpoint
  • kwargs (dict) – Optional kwargs to pass to all endpoints

moe_examples.blog_post_example_ab_testing module

An example of using MOE to optimize a simulated A/B testing framework.

Blog post: http://engineeringblog.yelp.com/2014/10/using-moe-the-metric-optimization-engine-to-optimize-an-ab-testing-experiment-framework.html

moe_examples.blog_post_example_ab_testing.COVARIANCE_INFO = {'hyperparameters': [0.3, 0.1]}

Set as a constant for clarity of example, see mod:moe_examples.combined_example for hyperparameter updating

moe_examples.blog_post_example_ab_testing.EPSILON_BANDIT_PARAMETER = 0.15

Our bandit policy moe.bandit.epsilon.epsilon_greedy requires an epsilon parameter This is the fraction of allocations that will only explore, the rest will only exploit

moe_examples.blog_post_example_ab_testing.EXPERIMENT_DOMAIN = [[0.0, 1.0]]

domain over which our parameter x can vary in optimizing CTR(x)

moe_examples.blog_post_example_ab_testing.EXPERIMENT_ITERATIONS = 5

Number of times to have MOE update cohorts

moe_examples.blog_post_example_ab_testing.GLOBAL_OPTIMAL_PARAMETER = array([ 0.71428711])

Global optima of CTR function

moe_examples.blog_post_example_ab_testing.LOCAL_OPTIMAL_PARAMETER = array([ 0.14289063])

Local optima of CTR function

moe_examples.blog_post_example_ab_testing.NUMBER_OF_ACTIVE_COHORTS = 3

Number of parameters to simulate each day

moe_examples.blog_post_example_ab_testing.STATUS_QUO_PARAMETER = array([ 0.2])

Initial parameter value

moe_examples.blog_post_example_ab_testing.TRAFFIC_PER_DAY = 1000000

Simulated traffic volume

moe_examples.blog_post_example_ab_testing.bernoulli_mean_and_var(observed_clicks, samples)[source]

Return the mean and variance of a Bernoulli Distribution with a given number of successes (clicks) and trials (impressions).

Parameters:
  • observed_clicks (int >= 0) – number of successes aka clicks
  • samples – number of trials aka impressions
Returns:

(mean, variance) of the Bernoulli Distribution producing the observed clicks/samples
Return type:

tuple
moe_examples.blog_post_example_ab_testing.calculate_system_ctr(arms)[source]

Calculate the average CTR for the entire system, given a set of arms.

This is different than :func:~`moe_examples.blog_post_example_ab_testing.objective_function` which computes observed mean and variance for a single arm. This function is a summary statistic for the entire experiment.

Parameters:arms (list of moe.bandit.data_containers.SampleArm) – arms currently being evaluated in the system
Returns:calculated system-wide CTR
Type:float
moe_examples.blog_post_example_ab_testing.find_new_points_to_sample(experiment, num_points=1, verbose=False)[source]

Find the optimal next point(s) to sample using expected improvement (via MOE).

Parameters:
  • experiment (moe.easy_interface.experiment.Experiment) – an Experiment object containing the historical data and metadata MOE needs to optimize
  • num_points (int >= 1) – number of new points (experiments) that we want MOE to suggest
  • verbose (bool) – whether to print status messages to stdout
Returns:

the next point(s) to sample
Return type:

list of length num_points of coordinates (list of length dim)
moe_examples.blog_post_example_ab_testing.generate_initial_traffic()[source]

Generate initial traffic allocations.

active_arms: list of coordinate-tuples corresponding to arms/cohorts currently being sampled

sample_arms: all arms from prev and current cohorts, keyed by coordinate-tuples.

Arm refers specifically to a moe.bandit.data_containers.SampleArm

Returns:(active_arms, sample_arms) tuple as described above
Return type:tuple
moe_examples.blog_post_example_ab_testing.generate_new_arms(active_arms, sample_arms, verbose=False)[source]

Find optimal new parameters to sample to get active_arms up to NUMBER_OF_ACTIVE_COHORTS.

This is done in the following way:
  1. Find the initial allocations of all active parameters
  2. MOE suggests new, optimal arms/parameters to sample, given all previous samples, using Bayesian Global Optimization
  3. The new parameters are allocated traffic according to a Multi-Armed Bandit policy for all active_arms
  4. If the objective function for a given parameter is too low (with high statistical significance), it is turned off
  5. Steps 2-4 are repeated until there are NUMBER_OF_ACTIVE_COHORTS parameters being sampled with non-zero traffic
Parameters:
  • active_arms (list of tuple) – list of coordinate-tuples corresponding to arms/cohorts currently being sampled
  • sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
  • verbose (bool) – whether to print status messages to stdout
Returns:

(allocations, active_arms, sample_arms) describing the next round of experiments to run

allocations: dict of traffic allocations, indexed by the active_arms return value

active_arms: new active arms to run in the next round of our experiment; same format as the input

sample_arms: the sample arms input updated with the newly generated active arms; same format as the input
Return type:

tuple
moe_examples.blog_post_example_ab_testing.get_allocations(active_arms, sample_arms, verbose=False)[source]

Return the allocation for each of active_arms using the epsilon greedy multi-armed bandit strategy.

Parameters:
  • active_arms (list of tuple) – list of coordinate-tuples corresponding to arms/cohorts currently being sampled
  • sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
  • verbose (bool) – whether to print status messages to stdout
Returns:

traffic allocations for the active_arms, one for each tuple in active_arms
Return type:

dict
moe_examples.blog_post_example_ab_testing.moe_experiment_from_sample_arms(sample_arms)[source]

Make a MOE experiment with all historical data (i.e., parameter value, CTR, noise variance triples).

Parameters:sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
Returns:MOE Experiment object usable with GP endpoints like gp_mean_var and gp_next_points
Return type:moe.easy_interface.experiment.Experiment
moe_examples.blog_post_example_ab_testing.objective_function(observed_sample_arm, observed_status_quo_sample_arm)[source]

The observed objective function MOE is attempting to maximize; returns the mean (relative to status quo) and variance.

Lower values are better (minimization). Our objective is relative to the CTR of the initial parameter value, STATUS_QUO_PARAMETER.

\[\Phi(\vec{x}) = \frac{\mathtt{CTR}(\vec{x})}{\mathtt{CTR}(\vec{SQ})}\]
Parameters:
Returns:

(relative CTR, variance) observed mean and variance of the objective function
Return type:

tuple
moe_examples.blog_post_example_ab_testing.plot_sample_arms(active_arms, sample_arms, iteration_number)[source]

Plot the underlying Gaussian Process, showing samples, GP mean/variance, and the true CTR.

Plot is written to ab_plot_%.2d.pdf where %.2d is the two-digit iteration_number. This makes it convenient to sequence the plots later for animation.

The shows the current state of the Gaussian Process. Arms currently being sampled are highlighted with red Xs and previous arms are marked with blue Xs. The GP mean line is drawn and the variance region is shaded. The true CTR curve (that the GP approximates) is also shown.

Parameters:
  • active_arms (list of tuple) – list of coordinate-tuples corresponding to arms/cohorts currently being sampled
  • sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
  • iteration_number (int >= 0) – the index of the current iteration/round being tested in our experiment
moe_examples.blog_post_example_ab_testing.plot_system_ctr(system_ctr)[source]

Plot the system ctr time series and write it to ctr_plot.pdf.

Time is “fake” so we index it 0, 1, .. len(system_ctr) - 1; i.e., time is just iteration/round number in our experiment.

Parameters:system_ctr (list of float) – ctr values to plot, ordered by “time”
moe_examples.blog_post_example_ab_testing.prune_arms(active_arms, sample_arms, verbose=False)[source]

Remove all arms from active_arms that have an allocation less than two standard deviations below the current best arm.

Parameters:
  • active_arms (list of tuple) – list of coordinate-tuples corresponding to arms/cohorts currently being sampled
  • sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
  • verbose (bool) – whether to print status messages to stdout
Returns:

list of coordinate-tuples that are the well-performing members of active_arms length is at least 1 and at most len(active_arms)
Return type:

list of tuple
moe_examples.blog_post_example_ab_testing.run_example(verbose=False)[source]

Run the example experiment framework.

This is done in the following way:

  1. Generate initial data

  2. Run the experiment EXPERIMENT_ITERATIONS times
    1. Removing underperforming arms/parameters
    2. Generate new parameters to sample (aka new arms to replace those removed)
    3. Sample the new arms/parameters
    4. Plot the updated GP from the new parameters
    5. Repeat

  3. Plot out summary metrics

Note

Below, active_arms is a list of tuples. The tuples are the coordinates of the point represented by active_arms. This is so that members of active_arms can be used as keys. In turn, sample_arms is a dict keyed on those same coordinate tuples. The tuples make it convenient to switch between serializable types and lists/arrays.

In practice, users would likely give their cohorts str names. These names could key both active_arms (which becomes a dict mapping name to coordinates) and sample_arms.

Parameters:verbose (bool) – whether to print status messages to stdout
moe_examples.blog_post_example_ab_testing.run_time_consuming_experiment(allocations, sample_arms, verbose=False)[source]

Run the time consuming or expensive experiment.

Note

Obtaining the value of the objective function is assmumed to be either time consuming or expensive, where every evaluation is precious and we want to find the optimal set of parameters with as few calls as possible.

This experiment runs a simulation of user traffic over various parameters and users. It simulates an A/B testing experiment framework.

For each arm/cohort, we compute the true CTR. From bandits, we know the allocation of traffic for that arm. So the simulation involves running (TRAFFIC_PER_DAY * allocation) Bernoulli trials each with probability = true CTR. Then since we cannot know the true CTR, we only work with the observed CTR and variance (as computed in :func:~`moe_examples.blog_post_example_ab_testing.objective_function`).

Parameters:
  • allocations (dict) – traffic allocations for the active_arms, one for each tuple in active_arms. These are the cohorts for the experiment round–the parameters being tested and on what portion of traffic.
  • sample_arms (dict) – all arms from prev and current cohorts, keyed by coordinate-tuples Arm refers specifically to a moe.bandit.data_containers.SampleArm
  • verbose (bool) – whether to print status messages to stdout
moe_examples.blog_post_example_ab_testing.true_click_through_rate(x)[source]

Return the true underlying Click Through Rate (CTR) with respect to the parameter x.

Higher values are better. There is an local optima at LOCAL_OPTIMAL_PARAMETER (0.14289063) and a global optima at GLOBAL_OPTIMAL_PARAMETER (0.71428711).

\[\begin{split}\texttt{CTR}(x) = \begin{cases} \frac{1}{100}\left( \sin\left( \frac{7\pi}{2} x\right) + 2.5\right) & x \leq 0.6 \\ \frac{1}{100}\left( \sin\left( \frac{7\pi}{2} x\right) + 1.1\right) & x > 0.6 \end{cases}\end{split}\]
Parameters:x (array of float64 with shape (1, )) – The parameter that we are optimizing
Returns:CTR evaluated at x
Return type:array of float64 with shape (1, )

moe_examples.combined_example module

An example for accessing the all the functions in moe/easy_interface/simple_endpoint.

The following functions are used:

The function requires some historical information to inform the Gaussian Process we use an arbitrary function (with noise) function_to_minimize

We first sample [0,0] from the function and then generate and sample 5 optimal points from moe sequentially We then update the hyperparameters of the GP (model selection) This process is repeated until we have sampled 20 points in total We then calculate the posterior mean and variance of the GP at several points

moe_examples.combined_example.function_to_minimize(x)[source]

Calculate an aribitrary 2-d function with some noise.

This function has a minimum near [1, 2.6].

moe_examples.combined_example.run_example(num_to_sample=20, verbose=True, testapp=None, gp_next_points_kwargs=None, gp_hyper_opt_kwargs=None, gp_mean_var_kwargs=None, **kwargs)[source]

Run the combined example.

Parameters:
  • num_to_sample (int > 0) – Number of points for MOE to suggest and then sample [20]
  • verbose (bool) – Whether to print information to the screen [True]
  • testapp (Pyramid test application) – Whether to use a supplied test pyramid application or a rest server [None]
  • gp_next_points_kwargs (dict) – Optional kwargs to pass to gp_next_points endpoint
  • gp_hyper_opt_kwargs (dict) – Optional kwargs to pass to gp_hyper_opt_kwargs endpoint
  • gp_mean_var_kwargs (dict) – Optional kwargs to pass to gp_mean_var_kwargs endpoint
  • kwargs (dict) – Optional kwargs to pass to all endpoints

moe_examples.hyper_opt_of_gp_from_historical_data module

An example for accessing the gp_mean_var simple endpoint.

moe.easy_interface.simple_endpoint.gp_mean_var()

The function requires some historical information to inform the Gaussian Process

The optimal hyperparameters are returned.

moe_examples.hyper_opt_of_gp_from_historical_data.run_example(verbose=True, **kwargs)[source]

Run the example, aksing MOE for optimal hyperparameters given historical data.

moe_examples.mean_and_var_of_gp_from_historic_data module

An example for accessing the gp_mean_var simple endpoint.

moe.easy_interface.simple_endpoint.gp_mean_var()

The function requires some historical information to inform the Gaussian Process and a set of points to calculate the posterior mean and variance at.

The posterior mean and variance is then printed for every point.

moe_examples.mean_and_var_of_gp_from_historic_data.run_example(verbose=True, testapp=None, **kwargs)[source]

Run the example, finding the posterior mean and variance for various poinst from a random GP.

moe_examples.next_point_via_simple_endpoint module

An example for accessing the gp_next_points_* endpoints.

moe.easy_interface.simple_endpoint.gp_next_points()

The function requires some historical information to inform the Gaussian Process

The result is the next best set of point(s) to sample

Note

This file is copied in README.md and docs/index.rst. Any changes here should be made there as well.

moe_examples.next_point_via_simple_endpoint.function_to_minimize(x)[source]

Calculate an aribitrary 2-d function with some noise with minimum near [1, 2.6].

moe_examples.next_point_via_simple_endpoint.run_example(num_points_to_sample=20, verbose=True, **kwargs)[source]

Run the example, aksing MOE for num_points_to_sample optimal points to sample.

Module contents

Examples for using the moe package.