Examples

Worked examples as Jupyter notebooks.

• Comparing Bayesian Approaches to Disjoint Linear Bandits
  • Approach 1: NormalInverseGammaRegressor (Exact Bayesian Inference)
  • Approach 2: BayesianGLM (Accurate Likelihood Modeling)
  • Comparing Thompson Sampling Approaches
  • Comprehensive Comparison
  • Analysis: The Power of Misspecified Models
• Hybrid vs Disjoint Contextual Bandits with LipschitzContextualAgent
  • Key Concepts
  • Block Sparse Hybrid Bandits = Disjoint Bandits
  • Content Personalization Example: Partial Feature Sharing
  • Define true reward model
  • Approach 1: Disjoint Bandits
  • Approach 2: Hybrid Bandits with LipschitzContextualAgent
  • Running the Comparison Experiment
  • Analyzing the Results
  • Examining Learned Parameters
  • Computational Efficiency and Production Scale Considerations
• Empirical Bayes for Automatic Hyperparameter Tuning
  • Setup: Linear Reward Oracle with Regime Shift
  • Defining the Arms and Agents
  • Running the Simulation
  • Results: Cumulative Regret
  • Hyperparameter Evolution
  • Discussion
• Adversarial Bandits: Playing Rock-Paper-Scissors Against an Exploitative Opponent
  • The Exploitative Opponent
  • Playing Against the Adversary
  • The Exploitation Problem: Game-Theoretic Foundations
  • EXP3A: Importance-Weighted Updates
  • Key Insight
  • Final Performance Summary
  • Against Suboptimal Opponents
  • Summary
• Delayed and non-stationary multi-armed bandit problems
  • How can we implement a geographic pricing strategy?
  • Simulating the problem
• Offline learning with a loan pricing example
  • Loan pricing example
  • Conclusion: run the experiment multiple times
  • Bonus: Optimize the decay rates with optuna