bayesianbandits.LearnerPipeline#

class bayesianbandits.LearnerPipeline(steps: List[Tuple[str, Any]], learner: Learner[Any])#

Bases: Generic[X_contra]

Pipeline that implements the Learner protocol with generic input type.

Enables sklearn transformers to work with Bayesian learners by implementing the standard Learner interface (partial_fit, sample, predict, decay). This pipeline is designed to be used as a learner within Arms, particularly for LipschitzContextualAgent where enriched features (context + arm info) need sklearn preprocessing before reaching the final Bayesian learner.

Type Parameters#

X_contratype: The contravariant input type that this pipeline accepts. Common types include np.ndarray, list, pd.DataFrame, or Any.

param steps:: List of (name, transformer) tuples for preprocessing steps. All transformers must be either stateless or already fitted on historical data.
type steps:: List[Tuple[str, Any]]
param learner:: Final learner that implements the Learner protocol (partial_fit, sample, predict, decay methods).
type learner:: Any

Examples

Post-Featurization Preprocessing with LipschitzContextualAgent:

>>> import numpy as np
>>> from sklearn.preprocessing import StandardScaler
>>> from sklearn.decomposition import PCA
>>> from bayesianbandits import (
...     LipschitzContextualAgent, ThompsonSampling, NormalRegressor,
...     ArmColumnFeaturizer, Arm
... )
>>> from bayesianbandits.pipelines import LearnerPipeline
>>>
>>> # After ArmFeaturizer enriches context with arm features,
>>> # we want to standardize and reduce dimensionality
>>> # Pre-fit transformers on historical data
>>> scaler = StandardScaler()
>>> pca = PCA(n_components=2)
>>> # Fit on dummy historical enriched features (context + arm_features)
>>> dummy_features = np.random.randn(100, 3)  # 2 context + 1 arm feature
>>> _ = scaler.fit(dummy_features)
>>> _ = pca.fit(scaler.transform(dummy_features))
>>>
>>> learner_pipeline = LearnerPipeline[np.ndarray](
...     steps=[
...         ('standardize', scaler),  # Pre-fitted scaler
...         ('reduce_dims', pca),     # Pre-fitted PCA
...     ],
...     learner=NormalRegressor(alpha=1.0, beta=1.0)  # Final learner
... )
>>>
>>> # Use numeric arm featurizer to avoid string conversion issues
>>> from bayesianbandits.featurizers import FunctionArmFeaturizer
>>> def numeric_featurizer(X, action_tokens):
...     n_contexts, n_features = X.shape
...     n_arms = len(action_tokens)
...     result = np.zeros((n_contexts, n_features + 1, n_arms))
...     for i, token in enumerate(action_tokens):
...         result[:, :-1, i] = X  # Original features
...         result[:, -1, i] = i   # Numeric arm ID
...     return result
>>>
>>> # Use in LipschitzContextualAgent - all arms share this learner
>>> arms = [Arm(f'product_{i}', learner=learner_pipeline) for i in range(5)]
>>> agent = LipschitzContextualAgent(
...     arms=arms,
...     policy=ThompsonSampling(),
...     arm_featurizer=FunctionArmFeaturizer(numeric_featurizer),
...     learner=learner_pipeline  # Shared learner with preprocessing
... )
>>>
>>> # Data flow:
>>> # Raw context -> ArmFeaturizer -> [context + arm_features]
>>> #            -> StandardScaler -> PCA (2D) -> NormalRegressor
>>> user_context = np.array([[25, 50000]])  # [age, income]
>>> recommendations = agent.pull(user_context)
>>> len(recommendations)  # Should return number of contexts
1

High-Dimensional Features with Dimensionality Reduction:

>>> from sklearn.decomposition import PCA
>>>
>>> # After arm featurization, we have high-dimensional features
>>> # Reduce dimensionality before learning for efficiency
>>> pca = PCA(n_components=10)
>>> _ = pca.fit(np.random.randn(100, 20))  # Pre-fit on dummy historical data
>>> high_dim_learner = LearnerPipeline[np.ndarray](
...     steps=[('reduce_dims', pca)],  # Pre-fitted PCA
...     learner=NormalRegressor(alpha=0.1, beta=1.0)
... )
>>>
>>> # Useful when ArmFeaturizer creates high-dimensional encodings
>>> # e.g., interaction features between context and many arms

Different Input Types with Type Parameters:

>>> from sklearn.feature_extraction import DictVectorizer
>>> from typing import List, Dict
>>>
>>> # Pipeline for list of dicts input - pre-fit vectorizer
>>> vectorizer = DictVectorizer()
>>> _ = vectorizer.fit([{'a': 1, 'b': 2}, {'a': 3, 'c': 4}])  # Fit on dummy data
>>> dict_pipeline = LearnerPipeline[List[Dict[str, float]]](
...     steps=[('vectorize', vectorizer)],  # Pre-fitted vectorizer
...     learner=NormalRegressor(alpha=1.0, beta=1.0)
... )
>>>
>>> # Pipeline for numpy arrays - pre-fit scaler
>>> scaler = StandardScaler()
>>> _ = scaler.fit(np.random.randn(50, 5))  # Fit on dummy historical data
>>> array_pipeline = LearnerPipeline[np.ndarray](
...     steps=[('scale', scaler)],  # Pre-fitted scaler
...     learner=NormalRegressor(alpha=1.0, beta=1.0)
... )

Stateless Transformations:

>>> from sklearn.preprocessing import FunctionTransformer
>>>
>>> def log_transform(X):
...     # Apply log transformation to certain features
...     X_log = X.copy()
...     X_log[:, 0] = np.log1p(X_log[:, 0])  # Log-transform first feature
...     return X_log
>>>
>>> # Stateless transformations don't need fitting
>>> stateless_learner = LearnerPipeline[np.ndarray](
...     steps=[('log_transform', FunctionTransformer(log_transform))],
...     learner=NormalRegressor(alpha=1.0, beta=1.0)
... )  # No fitting needed for stateless transformers

Notes

The LearnerPipeline is specifically designed for the workflow where:

Raw context is processed by ArmFeaturizer to create enriched features
Enriched features need sklearn preprocessing (standardization, PCA, etc.)
Preprocessed features are used by the final Bayesian learner

This enables sophisticated feature engineering on the post-featurization feature space, which is not possible with agent-level preprocessing alone.

Important: All sklearn transformers must be either stateless (like FunctionTransformer) or already fitted on historical data. No fitting occurs during online operation to maintain performance and predictability.

The pipeline correctly implements all methods of the Learner protocol: - partial_fit: Transforms input and updates final learner - sample: Transforms input and samples from final learner - predict: Transforms input and predicts with final learner - decay: Transforms input and decays final learner - random_state: Delegates to final learner

bayesianbandits.LearnerPipeline#

Type Parameters#

This Page