pmlearn.gaussian_process package

Submodules

pmlearn.gaussian_process.gpc module

pmlearn.gaussian_process.gpr module

Gaussian process regression.

class pmlearn.gaussian_process.gpr.GaussianProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

Gaussian Process Regression built using PyMC3.

Fit a Gaussian process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:
  • prior_mean (mean object) – The mean specifying the mean function of the GP. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.
  • kernel (covariance function (kernel)) – The function specifying the covariance of the GP. If None is passed, the kernel “RBF()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import GaussianProcessRegressor
>>> from pmlearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> gpr = GaussianProcessRegressor(kernel=kernel).fit(X, y)
>>> gpr.score(X, y) 
0.3680...
>>> gpr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model.
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None
class pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin[source]

Bases: pmlearn.base.BayesianRegressorMixin

Mixin class for Gaussian Process Regression

predict(X, return_std=False)[source]

Perform Prediction

Predicts values of new data with a trained Gaussian Process Regression model

Parameters:
  • X (numpy array, shape [n_samples, n_features]) –
  • return_std (Boolean) – Whether to return standard deviations with mean values. Defaults to False.
class pmlearn.gaussian_process.gpr.SparseGaussianProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

Sparse Gaussian Process Regression built using PyMC3.

Fit a Sparse Gaussian process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:prior_mean (mean object) – The mean specifying the mean function of the GP. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import SparseGaussianProcessRegressor
>>> from sklearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> sgpr = SparseGaussianProcessRegressor(kernel=kernel).fit(X, y)
>>> sgpr.score(X, y) 
0.3680...
>>> sgpr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None
class pmlearn.gaussian_process.gpr.StudentsTProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

StudentsT Process Regression built using PyMC3.

Fit a StudentsT process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:prior_mean (mean object) – The mean specifying the mean function of the StudentsT process. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import StudentsTProcessRegressor
>>> from sklearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> spr = StudentsTProcessRegressor(kernel=kernel).fit(X, y)
>>> spr.score(X, y) 
0.3680...
>>> spr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None

pmlearn.gaussian_process.kernels module

Kernels for Gaussian process regression and classification.

class pmlearn.gaussian_process.kernels.DotProduct(input_dim, ls=None, ls_inv=None, active_dims=None)[source]

Bases: pymc3.gp.cov.Exponential

Dot-Product kernel from pymc3.gp.cov.Exponential

The DotProduct kernel is non-stationary and can be obtained from linear regression by putting N(0, 1) priors on the coefficients of x_d (d = 1, . . . , D) and a prior of N(0, sigma_0^2) on the bias. The DotProduct kernel is invariant to a rotation of the coordinates about the origin, but not translations. It is parameterized by a parameter sigma_0^2. For sigma_0^2 =0, the kernel is called the homogeneous linear kernel, otherwise it is inhomogeneous.

The kernel is given by k(x_i, x_j) = sigma_0 ^ 2 + x_i cdot x_j

The DotProduct kernel is commonly combined with exponentiation.

class pmlearn.gaussian_process.kernels.RBF(input_dim, ls=None, ls_inv=None, active_dims=None)[source]

Bases: pymc3.gp.cov.ExpQuad

Radial-basis function kernel from pymc3.gp.cov.ExpQuad

The RBF kernel is a stationary kernel. It is also known as the “squared exponential” kernel. It is parameterized by a length-scale parameter length_scale>0, which can either be a scalar (isotropic variant of the kernel) or a vector with the same number of dimensions as the inputs X (anisotropic variant of the kernel).

The kernel is given by:

k(x_i, x_j) = exp(-1 / 2 d(x_i / length_scale, x_j / length_scale)^2)

This kernel is infinitely differentiable, which implies that GPs with this kernel as covariance function have mean square derivatives of all orders, and are thus very smooth.

class pmlearn.gaussian_process.kernels.WhiteKernel(noise_level=1.0)[source]

Bases: pymc3.gp.cov.WhiteNoise

White kernel from ``pymc3.gp.cov.WhiteNoise..

The main use-case of this kernel is as part of a sum-kernel where it explains the noise-component of the signal. Tuning its parameter corresponds to estimating the noise-level.

k(x_1, x_2) = noise_level if x_1 == x_2 else 0

Module contents

The pmlearn.gaussian_process module implements Gaussian Process based regression and classification.

class pmlearn.gaussian_process.GaussianProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

Gaussian Process Regression built using PyMC3.

Fit a Gaussian process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:
  • prior_mean (mean object) – The mean specifying the mean function of the GP. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.
  • kernel (covariance function (kernel)) – The function specifying the covariance of the GP. If None is passed, the kernel “RBF()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import GaussianProcessRegressor
>>> from pmlearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> gpr = GaussianProcessRegressor(kernel=kernel).fit(X, y)
>>> gpr.score(X, y) 
0.3680...
>>> gpr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model.
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None
class pmlearn.gaussian_process.StudentsTProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

StudentsT Process Regression built using PyMC3.

Fit a StudentsT process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:prior_mean (mean object) – The mean specifying the mean function of the StudentsT process. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import StudentsTProcessRegressor
>>> from sklearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> spr = StudentsTProcessRegressor(kernel=kernel).fit(X, y)
>>> spr.score(X, y) 
0.3680...
>>> spr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None
class pmlearn.gaussian_process.SparseGaussianProcessRegressor(prior_mean=None, kernel=None)[source]

Bases: pmlearn.base.BayesianModel, pmlearn.gaussian_process.gpr.GaussianProcessRegressorMixin

Sparse Gaussian Process Regression built using PyMC3.

Fit a Sparse Gaussian process model and estimate model parameters using MCMC algorithms or Variational Inference algorithms

Parameters:prior_mean (mean object) – The mean specifying the mean function of the GP. If None is passed, the mean “pm.gp.mean.Zero()” is used as default.

Examples

>>> from sklearn.datasets import make_friedman2
>>> from pmlearn.gaussian_process import SparseGaussianProcessRegressor
>>> from sklearn.gaussian_process.kernels import DotProduct, WhiteKernel
>>> X, y = make_friedman2(n_samples=500, noise=0, random_state=0)
>>> kernel = DotProduct() + WhiteKernel()
>>> sgpr = SparseGaussianProcessRegressor(kernel=kernel).fit(X, y)
>>> sgpr.score(X, y) 
0.3680...
>>> sgpr.predict(X[:2,:], return_std=True) 
(array([653.0..., 592.1...]), array([316.6..., 316.6...]))

Rasmussen and Williams (2006). Gaussian Processes for Machine Learning.

create_model()[source]

Creates and returns the PyMC3 model.

Note: The size of the shared variables must match the size of the training data. Otherwise, setting the shared variables later will raise an error. See http://docs.pymc.io/advanced_theano.html

Returns:model
Return type:the PyMC3 model
load(file_prefix)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params
Return type:

Dictionary of custom parameters
save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None