1.1. Generalized Linear Models

The following are a set of methods intended for regression in which the target value is expected to be a linear combination of the input variables. In mathematical notion, if \(\hat{y}\) is the predicted value.

\[\hat{y}(\beta, x) = \beta_0 + \beta_1 x_1 + ... + \beta_p x_p\]

Where \(\beta = (\beta_1, ..., \beta_p)\) are the coefficients and \(\beta_0\) is the y-intercept.

To perform classification with generalized linear models, see Bayesian Logistic regression.

1.1.1. Bayesian Linear Regression

To obtain a fully probabilistic model, the output \(y\) is assumed to be Gaussian distributed around \(X w\):

\[p(y|X,w,\alpha) = \mathcal{N}(y|X w,\alpha)\]

Alpha is again treated as a random variable that is to be estimated from the data.

References

• A good introduction to Bayesian methods is given in C. Bishop: Pattern Recognition and Machine learning
• Original Algorithm is detailed in the book Bayesian learning for neural networks by Radford M. Neal

1.1.2. Bayesian Logistic regression

Bayesian Logistic regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.

The implementation of logistic regression in pymc-learn can be accessed from class LogisticRegression.