Basis FunctionsΒΆ

This module contains basis function objects that can be used with the machine learning algorithms in revrand. It also implements basis concatenation (kernel addition) using the Basis and BasisCat classes. See the Quickstart Guide for an overview of the usage of these objects.

Note

When calling transform or grad on concatenated bases while using arguments, be careful of ordering. For instance,

>>> from revrand.basis_functions import RandomRBF, RandomMatern52
>>> base = RandomRBF(Xdim=D, nbases=n) \
...     + RandomMatern52(Xdim=D, nbases=n)
>>> base.transform(X)

This call to transform just uses the default value for the length scales in both bases.

>>> base.transform(X, 1)

Will pass a length scale of 1 to just the first basis object (RandomRBF), and the second (RandomMatern52) will use its default.

>>> base.transform(X, None, 2)

Will make the first basis object use its default, and make the second use a value of 2.

>>> base.transform(X, 1, 2)

Will make the first use a value of 1, and the second, 2.

Basis([regularizer]) The base Basis class.
BiasBasis([offset, regularizer]) Bias Basis for adding a bias term to a regressor.
LinearBasis([onescol, regularizer]) Linear basis class, basically this just prepends a columns of ones onto X.
PolynomialBasis(order[, include_bias, ...]) Polynomial basis class.
RadialBasis(centres[, lenscale, bounds, ...]) Radial basis class.
SigmoidalBasis(centres[, lenscale, bounds, ...]) Sigmoidal Basis.
RandomRBF(nbases, Xdim[, lenscale, bounds, ...]) Random RBF Basis – Approximates an RBF kernel function.
RandomLaplace(nbases, Xdim[, lenscale, ...]) Random Laplace Basis – Approximates a Laplace kernel function.
RandomCauchy(nbases, Xdim[, lenscale, ...]) Random Cauchy Basis – Approximates a Cauchy kernel function.
RandomMatern32(nbases, Xdim[, lenscale, ...]) Random Matern 3/2 Basis – Approximates a Matern 3/2 kernel function.
RandomMatern52(nbases, Xdim[, lenscale, ...]) Random Matern 5/2 Basis – Approximates a Matern 5/2 kernel function.
OrthogonalRBF(nbases, Xdim[, lenscale, ...]) Orthogonal Random RBF Basis – Approximates an RBF kernel function.
FastFoodRBF(nbases, Xdim[, lenscale, ...]) Fast Food radial basis function.
FastFoodGM(nbases, Xdim[, mean, bounds, ...]) A mixture component from a Gaussian spectral mixture kernel approximation.

Various basis function objects specialised for parameter learning.

To make a new basis object, see the documentation of the Basis class.

class revrand.basis_functions.Basis(regularizer=None)

The base Basis class.

To make other basis classes, make sure they are subclasses of this class to enable concatenation and operation with the machine learning algorithms.

Parameters:regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).

Example

Basis concatentation works as follows if you subclass this class:

>>> base = MyBasis1(properties1) + MyBasis2(properties2)
get_dim(X)

Get the output dimensionality of this basis.

This makes a cheap call to transform with the initial parameter values to ascertain the dimensionality of the output features.

Parameters:X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
Returns:The dimensionality of the basis.
Return type:int
grad(X)

Return the gradient of the basis function for each parameter.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • params (optional) – parameter aguments, these can be scalars or arrays.
Returns:

this will be a list of ndarrays if there are multiple parameters, or just an ndarray if there is a single parameter. The ndarrays can have more than two dimensions (i.e. tensors of rank > 2), depending on the dimensions of the basis function parameters. If there are no parameters, [] is returned.
Return type:

list or ndarray
params

Get this basis’ Parameter types.

params_values()

Get a list of the Parameter values if they have a value.

This does not include the basis regularizer.

regularizer

Get the Parameter value of this basis’ regularizer.

regularizer_diagonal(X, regularizer=None)

Get the diagonal of the prior variance on the weights (regularizer).

This also returns a slice into the regression weights to which this regularizer applies.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • regularizer (float, optional) – use this value instead of the value passed into the constructor for the regularizer.
Returns:

  • ndarray – an array of shape (D,) that is the diagonal of the prior variance of the weights, \(\boldsymbol\Lambda\) in \(\mathbf{w} \sim \mathcal{N}(\mathbf{0}, \boldsymbol\Lambda)\).
  • slice – the slice into the regression weights that this regularizer applies to.

Note

You should not have to modify this method if you inherit from Basis unless you are doing something very interesting...

transform(X)

Return the basis function applied to X.

I.e. Phi(X, params), where params can also optionally be used and learned.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • params (optional) – parameter aguments, these can be scalars or arrays.
Returns:

of shape (N, D) where D is the number of basis functions.
Return type:

ndarray
class revrand.basis_functions.BasisCat(basis_list)

A class that implements concatenation of bases.

Parameters:basis_list (list) – a list of Basis objects to concatenate
get_dim(X)

Get the output dimensionality of this basis.

This makes a cheap call to transform with the initial parameter values to ascertain the dimensionality of the output features.

Parameters:X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
Returns:The dimensionality of the basis.
Return type:int
grad(X, *params)

Return the gradient of the basis function for each parameter.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • *params (optional) – parameter aguments, these are the parameters of the concatenated bases in the order they were concatenated.
Returns:

this will be a list of ndarrays if there are multiple parameters, or just an ndarray if there is a single parameter. The ndarrays can have more than two dimensions (i.e. tensors of rank > 2), depending on the dimensions of the basis function parameters. If there are no parameters, [] is returned.
Return type:

list or ndarray
params

Return a list of all of the Parameter objects.

Or a just a single Parameter is there is only one, and single empty Parameter if there are no parameters.

params_values()

Get a list of the Parameter values if they have a value.

This does not include the basis regularizers.

regularizer

Return a list of concatenated bases regularizers.

regularizer_diagonal(X, *regularizer)

Get the diagonal of the prior variance on the weights (regularizer).

This also returns a slice into the regression weights to which all of the concatenated bases’ regularizers applies.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • *regularizer – use these values instead of the values passed into the Basis constructors for each regularizer. These are applied to each basis in the concatentation in the order they were concatenated.
Returns:

  • ndarray – an array of shape (D,) that is the diagonal of the prior variance of the weights, \(\boldsymbol\Lambda\) in \(\mathbf{w} \sim \mathcal{N}(\mathbf{0}, \boldsymbol\Lambda)\).
  • list of slices – the slices into the regression weights that each regularizer applies to.
transform(X, *params)

Return the basis function applied to X.

I.e. Phi(X, params), where params can also optionally be used and learned.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • *params (optional) – parameter aguments, these are the parameters of the concatenated bases in the order they were concatenated.
Returns:

of shape (N, D) where D is the number of basis functions.
Return type:

ndarray
class revrand.basis_functions.BiasBasis(offset=1.0, regularizer=None)

Bias Basis for adding a bias term to a regressor.

This just returns a column of a constant value so a bias term can be learned by a regressor.

\[\phi(\mathbf{X}) = \mathbf{1} \times \text{const}\]
Parameters:
  • offset (float, optional) – A scalar value to give the bias column. By default this is one.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
transform(X)

Return this basis applied to X.

Parameters:X (ndarray) – of shape (N, d) of observations where N is the number of samples, and d is the dimensionality of X.
Returns:of shape (N, 1) of ones * self.offset.
Return type:ndarray
class revrand.basis_functions.FastFoodGM(nbases, Xdim, mean=Parameter(value=0.0, bounds=Bound(lower=None, upper=None), shape=()), lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

A mixture component from a Gaussian spectral mixture kernel approximation.

This implements a GM basis component from “A la Carte - Learning Fast Kernels”. This essentially learns the form of a kernel function, and so has no explicit kernel representation!

To fully implement a Gaussian spectral mixture, concatenate as many of these objects as desired (one per mixture component). Also remember to initialize all of the bases with different means.

Parameters:
  • nbases (int) – a scalar for how many (unique) random bases to create approximately, this actually will be to the nearest larger two power.
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • mean (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the component frequency means for optimization. This will always initialise (d,) means if a scalr bound is given, it is applied to all means.
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. This will always initialise ARD length scales, if a scalr bound is given, it is applied to all length scales.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
grad(X, mean=None, lenscale=None)

Get the gradients of this basis w.r.t.the mean and length scales.

Parameters:
  • x (ndarray) – (n, d) array of observations where n is the number of samples, and d is the dimensionality of x.
  • mean (ndarray, optional) – array of shape (d,) frequency means (one for each dimension of X). If not input, this uses the value of the initial mean.
  • lenscale (ndarray, optional) – array of shape (d,) length scales (one for each dimension of X). If not input, this uses the value of the initial length scale.
Returns:

  • ndarray – shape (n, 4*nbases) where nbases is number of random rbf bases, again to the nearest larger two power. This is \(\partial \phi(\mathbf{x}) / \partial \boldsymbol\mu\)
  • ndarray – shape (n, 4*nbases) where nbases is number of random rbf bases, again to the nearest larger two power. This is \(\partial \phi(\mathbf{x}) / \partial \mathbf{l}\)
transform(X, mean=None, lenscale=None)

Apply the spectral mixture component basis to X.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • mean (ndarray, optional) – array of shape (d,) frequency means (one for each dimension of X). If not input, this uses the value of the initial mean.
  • lenscale (ndarray, optional) – array of shape (d,) length scales (one for each dimension of X). If not input, this uses the value of the initial length scale.
Returns:

of shape (N, 4*nbases) where nbases is number of random bases to use, given in the constructor (to nearest larger two power).
Return type:

ndarray
class revrand.basis_functions.FastFoodRBF(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Fast Food radial basis function.

This is an approximation of the random radial basis function for a large number of bases.

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \exp\left( -\frac{\| \mathbf{x} - \mathbf{x}' \|^2}{2 l^2} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – a scalar for how many (unique) random bases to create approximately, this actually will be to the nearest larger two power.
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
grad(X, lenscale=None)

Get the gradients of this basis w.r.t.the length scale.

Parameters:
  • x (ndarray) – (n, d) array of observations where n is the number of samples, and d is the dimensionality of x.
  • lenscale (scalar or ndarray) – scalar or array of shape (d,) length scales (one for each dimension of x).If not input, this uses the value of the initial length scale.
Returns:

shape (n, 2*nbases) where nbases is number of random rbf bases, again to the nearest larger two power. This is \(\partial \phi(\mathbf{x}) / \partial l\)
Return type:

ndarray
transform(X, lenscale=None)

Apply the Fast Food RBF basis to X.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • lenscale (scalar or ndarray, optional) – scalar or array of shape (d,) length scales (one for each dimension of X).If not input, this uses the value of the initial length scale.
Returns:

of shape (N, 2*nbases) where nbases is number of random bases to use, given in the constructor (to nearest larger two power).
Return type:

ndarray
class revrand.basis_functions.LinearBasis(onescol=True, regularizer=None)

Linear basis class, basically this just prepends a columns of ones onto X.

\[\phi(\mathbf{X}) = [\mathbf{1}, \mathbf{X}]\]
Parameters:
  • onescol (bool, optional) – If true, prepend a column of ones onto X.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
transform(X)

Return this basis applied to X.

Parameters:X (ndarray) – of shape (N, d) of observations where N is the number of samples, and d is the dimensionality of X.
Returns:of shape (N, d+1), or (N, d) depending on onescol.
Return type:ndarray
class revrand.basis_functions.OrthogonalRBF(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Orthogonal Random RBF Basis – Approximates an RBF kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) RBF covariance function,

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \exp\left( -\frac{\| \mathbf{x} - \mathbf{x}' \|^2}{2 l^2} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed

Note

This should need fewer random bases to approximate an RBF kernel than the RandomRBF basis.

See: Yu, X Felix et. al. “Orthogonal Random Features”, in Advances in Neural Information Processing Systems”, Barcelona 2016.

class revrand.basis_functions.PolynomialBasis(order, include_bias=True, regularizer=None)

Polynomial basis class.

This essentially creates the concatenation,

\[\phi(\mathbf{X}) = [\mathbf{1}, \mathbf{X}^1, \ldots, \mathbf{X}^p]\]

where \(p\) is the order of the polynomial.

Parameters:
  • order (int) – the order of the polynomial to create.
  • include_bias (bool, optional) – If True (default), include the bias column (column of ones which acts as the intercept term in a linear model)
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
transform(X)

Return this basis applied to X.

Parameters:X (ndarray) – of shape (N, d) of observations where N is the number of samples, and d is the dimensionality of X.
Returns:of shape (N, d*order+1), the extra 1 is from a prepended ones column.
Return type:ndarray
class revrand.basis_functions.RadialBasis(centres, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None)

Radial basis class.

\[\phi(\mathbf{X}) = \exp \left( -\frac{\|\mathbf{X} - \mathbf{C}\|^2} {2 l^2} \right)\]

Where \(\mathbf{C}\) are radial basis centres, and \(l\) is a length scale.

Parameters:
  • centres (ndarray) – array of shape (Dxd) where D is the number of centres for the radial bases, and d is the dimensionality of X.
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).

Note

This will have relevance vector machine-like behaviour with uncertainty.

grad(X, lenscale=None)

Get the gradients of this basis w.r.t.the length scale.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • lenscale (scalar or ndarray, optional) – scalar or array of shape (d,) length scales (one for each dimension of X). If not input, this uses the value of the initial length scale.
Returns:

of shape (N, D) where D is number of RBF centres. This is \(\partial \Phi(\mathbf{X}) / \partial l\)
Return type:

ndarray
transform(X, lenscale=None)

Apply the RBF to X.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • lenscale (scalar or ndarray, optional) – scalar or array of shape (d,) length scales (one for each dimension of X). If not input, this uses the value of the initial length scale.
Returns:

of shape (N, D) where D is number of RBF centres.
Return type:

ndarray
class revrand.basis_functions.RandomCauchy(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Random Cauchy Basis – Approximates a Cauchy kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) Cauchy covariance function.

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \frac{1}{1 + (\| \mathbf{x} - \mathbf{x}' \| / l)^2}\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
class revrand.basis_functions.RandomLaplace(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Random Laplace Basis – Approximates a Laplace kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) Laplace covariance function.

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \exp\left( -\frac{\| \mathbf{x} - \mathbf{x}' \|}{l} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
class revrand.basis_functions.RandomMatern32(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Random Matern 3/2 Basis – Approximates a Matern 3/2 kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) Matern covariance function.

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \left(1 + \frac{\sqrt{3} \| \mathbf{x} - \mathbf{x}' \|}{l} \right) \exp \left(- \frac{\sqrt{3} \| \mathbf{x} - \mathbf{x}' \|}{l} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
class revrand.basis_functions.RandomMatern52(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Random Matern 5/2 Basis – Approximates a Matern 5/2 kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) Matern covariance function.

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \left(1 + \frac{\sqrt{5} \| \mathbf{x} - \mathbf{x}' \|}{l} + \frac{5 \| \mathbf{x} - \mathbf{x}' \|^2}{3l^2} \right) \exp \left(- \frac{\sqrt{5} \| \mathbf{x} - \mathbf{x}' \|}{l} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
class revrand.basis_functions.RandomRBF(nbases, Xdim, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None, random_state=None)

Random RBF Basis – Approximates an RBF kernel function.

This will make a linear regression model approximate a GP with an (optionally ARD) RBF covariance function,

\[\phi(\mathbf{x})^\top \phi(\mathbf{x}') \approx \exp\left( -\frac{\| \mathbf{x} - \mathbf{x}' \|^2}{2 l^2} \right)\]

with a length scale, \(l\) (a vector in \(\mathbb{R}^d\) for ARD).

Parameters:
  • nbases (int) – how many unique random bases to create (twice this number will be actually created, i.e. real and imaginary components for each base)
  • Xdim (int) – the dimension (d) of the observations (or the dimension of the slices if using apply_ind).
  • lenscale (Parameter, optional) – A scalar or vector of shape (1,) or (d,) Parameter to bound and initialise the length scales for optimization. If this is shape (d,), ARD length scales will be expected, otherwise an isotropic lenscale is learned.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
  • random_state (None, int or RandomState, optional) – random seed
class revrand.basis_functions.SigmoidalBasis(centres, lenscale=Parameter(value=1.0, bounds=Positive(upper=None), shape=()), regularizer=None)

Sigmoidal Basis.

\[\phi(\mathbf{X}) = \sigma \left( \frac{\|\mathbf{X} - \mathbf{C}\|}{l} \right)\]

where \(\mathbf{C}\) are sigmoidal basis centres, \(l\) is a length scale and \(\sigma\) is the logistic sigmoid function defined by

\[\sigma(a) = \frac{1}{1+e^{-a}}.\]
Parameters:
  • centres (ndarray) – array of shape (Dxd) where D is the number of centres for the bases, and d is the dimensionality of X.
  • lenscale (Parameter, optional) – A scalar parameter to bound and initialise the length scales for optimization.
  • regularizer (None, Parameter, optional) – The (initial) value of the regularizer/prior variance to apply to the regression weights of this basis function. The Parameter object must have a scalar value. If it is not set, it will take on a default value of Parameter(gamma(1.), Positive()).
grad(X, lenscale=None)

Get the gradients of this basis w.r.t. the length scale.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • lenscale (float, optional) – the length scale (scalar) of the RBFs to apply to X. If not input, this uses the value of the initial length scale.
Returns:

of shape (N, D) where D is number of centres. This is \(\partial \Phi(\mathbf{X}) / \partial l\)
Return type:

ndarray
transform(X, lenscale=None)

Apply the sigmoid basis function to X.

Parameters:
  • X (ndarray) – (N, d) array of observations where N is the number of samples, and d is the dimensionality of X.
  • lenscale (float) – the length scale (scalar) of the RBFs to apply to X. If not input, this uses the value of the initial length scale.
Returns:

of shape (N, D) where D is number of centres.
Return type:

ndarray
revrand.basis_functions.apply_grad(fun, grad)

Apply a function that takes a gradient matrix to a sequence of 2 or 3 dimensional gradients.

This is partucularly useful when the gradient of a basis concatenation object is quite complex, eg.

>>> X = np.random.randn(100, 3)
>>> y = np.random.randn(100)
>>> N, d = X.shape
>>> base = RandomRBF(Xdim=d, nbases=5) + RandomRBF(Xdim=d, nbases=5,
... lenscale=Parameter(np.ones(d), Positive()))
>>> Phi = base.transform(X, 1., np.ones(d))
>>> dffun = lambda dPhi: y.dot(Phi).dot(dPhi.T).dot(y)
>>> df = apply_grad(dffun, base.grad(X, 1., np.ones(d)))
>>> np.isscalar(df[0])
True
>>> df[1].shape
(3,)
Parameters:
  • fun (callable) – the function too apply to the (2d) gradient.
  • grad (ndarray or generator) – the gradient of the basis function (output of base.grad).
Returns:

the result of applying fun(grad) for a structured grad.
Return type:

scalar, ndarray or sequence
revrand.basis_functions.count_args(func)

Count the number of arguments in a function/method.

Parameters:func (callable) – a function or class method
Returns:the number of arguments, excluding self
Return type:int
revrand.basis_functions.slice_init(func)

Decorator for adding partial application functionality to a basis object.

This will add an “apply_ind” argument to a basis object initialiser that can be used to apply the basis function to only the dimensions specified in apply_ind. E.g.,

>>> X = np.ones((100, 20))
>>> base = LinearBasis(onescol=False, apply_ind=slice(0, 10))
>>> base.transform(X).shape
(100, 10)
revrand.basis_functions.slice_transform(func, self, X, *vargs, **kwargs)

Decorator for implementing partial application.

This must decorate the transform and grad methods of basis objects if the slice_init decorator was used.