bayesianbandits.EmpiricalBayesGammaRegressor#

class bayesianbandits.EmpiricalBayesGammaRegressor(alpha: float, beta: float, *, n_eb_iter: int = 10, eb_tol: float = 0.0001, learning_rate: float = 1.0, random_state: int | None | Generator = None)#

Bases: GammaRegressor

Gamma-Poisson regressor with empirical Bayes prior tuning.

Extends GammaRegressor with automatic optimization of the Gamma prior parameters (alpha, beta) via the Negative Binomial marginal likelihood, using Minka’s EM algorithm [1].

During fit, the prior is iteratively updated to maximize the marginal likelihood across all groups. During partial_fit, a single EM step is performed using the current group posteriors.

The EM treats per-group rates as latent Gamma variables. The E-step computes posterior moments; the M-step solves a Gamma MLE for the shared prior, using the generalized Newton method for the shape parameter [1].

When learning_rate < 1, exponential forgetting is applied. Stabilized forgetting re-injects the EB-tuned prior after each decay step, ensuring the prior contribution converges to the tuned value rather than zero.

Parameters:

  • alpha (float) – Initial Gamma shape parameter.

  • beta (float) – Initial Gamma rate parameter.

  • n_eb_iter (int, default 10) – Maximum number of EB iterations during fit.

  • eb_tol (float, default 1e-4) – Convergence tolerance on change in log marginal likelihood.

  • learning_rate (float, default 1.0) – Decay rate for sequential updates.

  • random_state (int, np.random.Generator, or None, default None) – Controls RNG for sample.

log_evidence_#

Log marginal likelihood at convergence.

Type:

float

n_eb_iterations_#

Number of EB iterations in last fit.

Type:

int

eb_converged_#

Whether the EB loop converged within eb_tol.

Type:

bool

See also

GammaRegressor

Base estimator without empirical Bayes tuning.

EmpiricalBayesDirichletClassifier

EB tuning for Dirichlet classification.

EmpiricalBayesNormalRegressor

EB tuning for Normal regression.

Notes

Negative Binomial marginal likelihood

The Gamma-Poisson marginal for group \(g\) with effective count \(c_g\) and exposure \(n_g\) is:

\[\log p(c_g \mid \alpha, \beta) = \log\Gamma(c_g + \alpha) - \log\Gamma(\alpha) - \log\Gamma(c_g + 1) + \alpha \log\frac{\beta}{\beta + n_g} + c_g \log\frac{n_g}{\beta + n_g}\]

EM update (Minka 2002, section 2.1)

E-step: posterior moments of the latent rate \(\lambda_g\):

\[\begin{split}\mathbb{E}[\lambda_g] &= \frac{\alpha + c_g}{\beta + n_g} \\ \mathbb{E}[\log \lambda_g] &= \psi(\alpha + c_g) - \log(\beta + n_g)\end{split}\]

M-step: Gamma MLE on expected sufficient statistics. The rate update is closed-form given the shape:

\[\beta = \frac{\alpha}{\bar{\mathbb{E}}[\lambda]}\]

The shape uses the generalized Newton method from [1]:

\[\frac{1}{\alpha^{\text{new}}} = \frac{1}{\alpha} + \frac{\log\bar{\mathbb{E}}[\lambda] - \bar{\mathbb{E}}[\log\lambda] - \log\alpha + \psi(\alpha)} {\alpha^2 (1/\alpha - \psi'(\alpha))}\]

Stabilized forgetting

Every time learning_rate \(\gamma < 1\) causes decay, the prior is re-injected:

\[\mathbf{p}_g \leftarrow \gamma^n \mathbf{p}_g + (1 - \gamma^n) \mathbf{p}^{\text{prior}}\]

where \(\mathbf{p}_g = (\alpha_g, \beta_g)\) are the posterior parameters. This ensures effective counts and exposures are correctly decayed and the prior contribution never vanishes.

References

Examples

Basic rate estimation with EB-tuned prior:

>>> import numpy as np
>>> from bayesianbandits import EmpiricalBayesGammaRegressor
>>> rng = np.random.default_rng(42)
>>> X = np.repeat(np.arange(1, 6), 30).reshape(-1, 1)
>>> y = rng.poisson(3.0, size=150)
>>> model = EmpiricalBayesGammaRegressor(
...     alpha=1.0, beta=1.0, random_state=0
... )
>>> model.fit(X, y)
EmpiricalBayesGammaRegressor(alpha=..., beta=..., random_state=0)
__init__(alpha: float, beta: float, *, n_eb_iter: int = 10, eb_tol: float = 0.0001, learning_rate: float = 1.0, random_state: int | None | Generator = None) → None#
decay(X: NDArray[Any], *, decay_rate: float | None = None) → None#

Decay with stabilized prior re-injection.

Applies exponential forgetting and re-injects the EB-tuned prior so that the prior contribution converges to prior_ rather than zero.

fit(X: NDArray[Any], y: NDArray[Any], sample_weight: NDArray[Any] | None = None) → Self#

Fit with empirical Bayes hyperparameter tuning.

Iteratively optimizes the Gamma prior parameters by maximizing the Negative Binomial marginal likelihood using Minka’s EM algorithm, then performs a final posterior update with the converged prior.

partial_fit(X: NDArray[Any], y: NDArray[Any], sample_weight: NDArray[Any] | None = None) → Self#

Incrementally update and retune hyperparameters.

Performs a posterior update (via the base class), then runs one EM step to adjust the prior. All group posteriors are corrected to reflect the new prior.