pmlearn.naive_bayes package

Submodules

pmlearn.naive_bayes.naive_bayes module

Naive Bayes models.

class pmlearn.naive_bayes.naive_bayes.GaussianNB[source]

Bases: pmlearn.base.BayesianModel, pmlearn.naive_bayes.naive_bayes.GaussianNBClassifierMixin

Gaussian Naive Bayes (GaussianNB) classification built using PyMC3.

The Gaussian Naive Bayes algorithm assumes that the random variables that describe each class and each feature are independent and distributed according to Normal distributions.

Example

>>> import numpy as np
>>> X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
>>> Y = np.array([1, 1, 1, 2, 2, 2])
>>> from pmlearn.naive_bayes import GaussianNB
>>> clf = GaussianNB()
>>> clf.fit(X, Y)
GaussianNB(priors=None, var_smoothing=1e-09)
>>> print(clf.predict([[-0.8, -1]]))
[1]
>>> clf_pf = GaussianNB()
>>> clf_pf.partial_fit(X, Y, np.unique(Y))
GaussianNB(priors=None, var_smoothing=1e-09)
>>> print(clf_pf.predict([[-0.8, -1]]))
[1]

See the documentation of the create_model method for details on the model itself.

create_model()[source]

Creates and returns the PyMC3 model.

We note \(x_{jc}\) the value of the j-th element of the data vector \(x\) conditioned on x belonging to the class \(c\). The Gaussian Naive Bayes algorithm models \(x_{jc}\) as:

\[x_{jc} \sim Normal(\mu_{jc}, \sigma_{jc})\]

While the probability that \(x\) belongs to the class \(c\) is given by the categorical distribution:

\[P(y=c|x_i) = Cat(\pi_1, \dots, \pi_C)\]

where \(\pi_i\) is the probability that a vector belongs to category \(i\).

We assume that the \(\pi_i\) follow a Dirichlet distribution:

\[\pi \sim Dirichlet(\alpha)\]

with hyperparameter \(\alpha = [1, .., 1]\). The \(\mu_{jc}\) are sampled from a Normal distribution centred on \(0\) with variance \(100\), and the \(\sigma_{jc}\) are sampled from a HalfNormal distribuion of variance \(100\):

\[ \begin{align}\begin{aligned}\mu_{jc} \sim Normal(0, 100)\\\sigma_{jc} \sim HalfNormal(100)\end{aligned}\end{align} \]

Note that the Gaussian Naive Bayes model is equivalent to a Gaussian mixture with a diagonal covariance [1].

Returns:
Return type:A PyMC3 model

References

[1]Murphy, K. P. (2012). Machine learning: a probabilistic

perspective.

load(file_profile)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params

Return type:

Dictionary of custom parameters

save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None
class pmlearn.naive_bayes.naive_bayes.GaussianNBClassifierMixin[source]

Bases: pmlearn.base.BayesianClassifierMixin

Mixin class for naive Bayes classifiers

fit(X, y, inference_type='advi', minibatch_size=None, inference_args=None)[source]

Train the Naive Bayes model.

Parameters:
  • X (numpy array, shape [num_training_samples, num_pred]) – Contains the data points.
  • y (numpy array, shape [num_training_samples,]) – Contains the category of the data points.
  • inference_type (string, specifies which inference method to call.) – Default is ‘advi’. Currently, only ‘advi’ and ‘nuts’ are implemented.
  • minibatch_size (int, number of samples to include in each minibatch) – for ADVI. Defaults to None so minibatch is not run by default.
  • inference_args (dict, arguments to be passed to the inference methods.) – Check the PyMC3 documentation.
Returns:

Return type:

The current instance of the GaussianNB class.

normalize(array)[source]

Normalize values in the array to get probabilities. :param array: :type array: numpy array of shape [1,]

Returns:
Return type:A normalized array
predict(X)[source]

Classify new data with a trained Naive Bayes model. The output is the point estimate of the posterior predictive distribution that corresponds to the one-hot loss function.

Parameters:X (numpy array, shape [num_training_samples, num_pred]) – Contains the data to classify.
Returns:
  • A numpy array of shape [num_training_samples,] that contains the
  • predicted class to which the data points belong.
predict_proba(X)[source]

Predicts the probabilities that data points belong to each category.

Given a new data point \(\vec{x}\), we want to estimate the probability that it belongs to a category \(c\). Following the notations in [1], the probability reads:

\[P(y=c|\vec{x}, \mathcal{D}) = P(y=c|\mathcal{D}) \prod_{j=1}^{n_{dims}} \ P(x_j|y=c, \mathcal{D})\]

We previously used the data \(\mathcal{D}\) to estimate the distribution of the parameters \(\vec{\mu}\), \(\vec{\pi}\) and \(\vec{\sigma}\). To compute the above probability, we need to integrate over the values of these parameters:

\[P(y=c|\vec{x}, \mathcal{D}) = \left[\int Cat(y=c|\vec{\pi})P(\vec{\pi}|\ \mathcal{D})\mathrm{d}\vec{\pi}\right] \int P(\vec{x}|\vec{\mu}, \vec{\sigma})\ P(\vec{\mu}|\mathcal{D})\ P(\vec{\sigma}|\mathcal{D})\ \mathrm{d}\vec{\mu}\mathrm{d}\vec{\sigma}\]
Parameters:X (numpy array, shape [num_training_samples, num_pred]) – Contains the points for which we want to predict the class
Returns:
  • A numpy array of shape [num_training_samples, num_cats] that contains
  • the probabilities that each sample belong to each category.

References

[1]Murphy, K. P. (2012). Machine learning: a probabilistic

perspective.

Module contents

The pmlearn.naive_bayes module implements Naive Bayes algorithms. These are supervised learning methods based on applying Bayes’ theorem with strong (naive) feature independence assumptions.

class pmlearn.naive_bayes.GaussianNB[source]

Bases: pmlearn.base.BayesianModel, pmlearn.naive_bayes.naive_bayes.GaussianNBClassifierMixin

Gaussian Naive Bayes (GaussianNB) classification built using PyMC3.

The Gaussian Naive Bayes algorithm assumes that the random variables that describe each class and each feature are independent and distributed according to Normal distributions.

Example

>>> import numpy as np
>>> X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
>>> Y = np.array([1, 1, 1, 2, 2, 2])
>>> from pmlearn.naive_bayes import GaussianNB
>>> clf = GaussianNB()
>>> clf.fit(X, Y)
GaussianNB(priors=None, var_smoothing=1e-09)
>>> print(clf.predict([[-0.8, -1]]))
[1]
>>> clf_pf = GaussianNB()
>>> clf_pf.partial_fit(X, Y, np.unique(Y))
GaussianNB(priors=None, var_smoothing=1e-09)
>>> print(clf_pf.predict([[-0.8, -1]]))
[1]

See the documentation of the create_model method for details on the model itself.

create_model()[source]

Creates and returns the PyMC3 model.

We note \(x_{jc}\) the value of the j-th element of the data vector \(x\) conditioned on x belonging to the class \(c\). The Gaussian Naive Bayes algorithm models \(x_{jc}\) as:

\[x_{jc} \sim Normal(\mu_{jc}, \sigma_{jc})\]

While the probability that \(x\) belongs to the class \(c\) is given by the categorical distribution:

\[P(y=c|x_i) = Cat(\pi_1, \dots, \pi_C)\]

where \(\pi_i\) is the probability that a vector belongs to category \(i\).

We assume that the \(\pi_i\) follow a Dirichlet distribution:

\[\pi \sim Dirichlet(\alpha)\]

with hyperparameter \(\alpha = [1, .., 1]\). The \(\mu_{jc}\) are sampled from a Normal distribution centred on \(0\) with variance \(100\), and the \(\sigma_{jc}\) are sampled from a HalfNormal distribuion of variance \(100\):

\[ \begin{align}\begin{aligned}\mu_{jc} \sim Normal(0, 100)\\\sigma_{jc} \sim HalfNormal(100)\end{aligned}\end{align} \]

Note that the Gaussian Naive Bayes model is equivalent to a Gaussian mixture with a diagonal covariance [1].

Returns:
Return type:A PyMC3 model

References

[1]Murphy, K. P. (2012). Machine learning: a probabilistic

perspective.

load(file_profile)[source]

Loads a saved version of the trace, and custom param files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to load the) –
  • trace for this model. (saved) – Ex: given file_prefix = “path/to/file/” This will attempt to load “path/to/file/trace.pickle”
  • load_custom_params (Boolean flag to indicate whether custom parameters) –
  • be loaded. Defaults to False. (should) –
Returns:

custom_params

Return type:

Dictionary of custom parameters

save(file_prefix)[source]

Saves the trace and custom params to files with the given file_prefix.

Parameters:
  • file_prefix (str, path and prefix used to identify where to save the) –
  • for this model. (trace) – Ex: given file_prefix = “path/to/file/” This will attempt to save to “path/to/file/trace.pickle”
  • custom_params (Dictionary of custom parameters to save.) – Defaults to None