Guide: Prior Calibration using Historical Data

This guide explains the prior_calibration/priors_calibration.py tool, designed to help inform prior selection for Bayesian Marketing Mix Modeling (MMM) channel coefficients using historical model outputs.

Methodology: The tool aggregates historical posterior means for beta_channel from one or more results folders (each containing csv/model_summary.csv), computes a per-channel mu, and emits a YAML block for the model config. Because AMMM’s default prior for beta_channel is HalfNormal (non‑negative effect) with a single sigma parameter, the tool supports:

  • --dist HalfNormal (default): converts each positive mu to a HalfNormal sigma via sigma ≈ mu * sqrt(pi/2) and falls back to --default-sigma if conversion is not possible (e.g., mu ≤ 0).

  • --dist Normal: outputs both mu and a uniform sigma list.

Table of Contents

Introduction

In Bayesian MMM, specifying appropriate priors is crucial. This script assists by suggesting the center (mu) for Normal priors for media channel coefficients (beta_channel) based on the average coefficients observed in historical models. It explicitly avoids deriving the prior’s width (sigma) from limited historical data, instead recommending a standard, weakly informative value to reflect uncertainty.

Prerequisites

  • Python 3.x

  • Libraries:

    • pandas

    • numpy

Script Overview

  1. Configuration: Sets the weakly informative sigma and default mu.

  2. Imports libraries.

  3. Prepares historical coefficient data (currently hardcoded for 2 models).

  4. Merges data to align coefficients by channel.

  5. Defines the list of media channels for the new model (order matters for YAML).

  6. Calculates the suggested mu for each channel based on the historical mean. Assigns default mu if no historical data exists for a channel. Uses the configured weakly informative sigma for all.

  7. Formats the mu and sigma lists for YAML output.

  8. Prints suggestions and YAML-compatible output for a Normal prior.

Detailed Explanation

1. Configuration

Sets the WEAKLY_INFORMATIVE_SIGMA (default: 2.0) applied to all channels and DEFAULT_MU (default: 0.0) for channels missing historical data.

2. Import Libraries

Imports pandas, numpy, and warnings.

3. Data Preparation

Loads coefficient data from previous models (currently hardcoded dictionaries). Replace this with your actual data.

4. Merge Coefficient Data

Merges the historical DataFrames using an ‘outer’ join to include channels present in either model.

5. Define Media Channels

Defines paid_media_channels as a list of tuples (display_name, channel_variable_name). The order must match the intended order in the final YAML configuration.

6. Calculate Prior Suggestions (mu)

Iterates through the defined paid_media_channels.

  • If a channel exists in the merged historical data, it calculates the mean of the available coefficients (mean_mod_avg_model1, mean_mod_avg_model2, handling NaNs). This mean becomes the suggested mu.

  • If a channel is not found in historical data, the DEFAULT_MU is used.

  • The WEAKLY_INFORMATIVE_SIGMA is assigned to all channels.

  • Prints progress and warnings if N < 3.

7. Prepare Output for YAML

Creates lists (mu_list, sigma_list, prior_comments) ordered according to paid_media_channels.

8. Output Results

Prints the suggested mu for each channel and then prints the full suggested beta_channel prior section in YAML format, using Normal distribution with the generated mu list and the fixed sigma list.

Limitations

  • Reliance on Limited Data (N=2): The primary limitation is the use of only two historical data points per channel to calculate the mean (mu). While this provides a central estimate, it’s based on very little information. The true coefficient could vary significantly.

  • Weakly Informative Sigma: The use of a fixed sigma is a pragmatic choice due to the unreliable variance from N=2 data. It ensures the prior isn’t overly narrow but doesn’t capture channel-specific variance learned from historical data (because that variance estimate is unreliable). Consider adjusting this sigma based on domain knowledge or running sensitivity analyses with different sigma values.

  • Assumes Comparability: The script assumes the coefficients from the historical models are reasonably comparable to the context of the new model (e.g., similar market conditions, target variable, model structure).

Usage Instructions

Use the CLI to read historical results and produce a YAML prior block.

CLI

python prior_calibration/priors_calibration.py \
  --results-dirs path/to/runA/results path/to/runB/results \
  --config-yaml path/to/model_config.yml \
  --dist HalfNormal \
  --default-sigma 2.0 \
  --out suggested_priors.yml \
  --verbose

Flags

  • --results-dirs: one or more results folders that contain csv/model_summary.csv.

  • --config-yaml: model config to read channel order from media[].spend_col (preferred; ensures alignment with the model).

  • --channels-file or --channels: ordered list of channel variable names for the final YAML. If omitted, the tool infers order from observed channels (best‑effort; prefer explicit order).

  • --dist: HalfNormal (default) or Normal.

  • --default-sigma: fallback sigma if conversion not possible (e.g., mu ≤ 0 for HalfNormal).

  • --round: decimal places (default 4).

  • --out: optionally write YAML to a file (in addition to printing).

  • --verbose: print aggregated mu per channel.

Notes

  • The tool reads csv/model_summary.csv and extracts rows where parameter starts with beta_channel[... ].

  • For HalfNormal, per‑channel sigma is computed as sigma_i ≈ max(ε, mu_i * sqrt(pi/2)) with ε≈1e‑6, or defaults to --default-sigma if not meaningful.

  • Ensure historical runs are comparable to your target model (same scaling, adstock, saturation settings), or interpret results cautiously.

Worked example

  1. Prepare a channels file (channels.txt) with variable names in the model’s channel order:

linear_radio
digital_ooh
video
linear_tv
bvod
social
digital_audio
influencers
display
partnership

  1. Run the tool on two prior runs and write YAML (reading channel order from a config YAML):

python prior_calibration/priors_calibration.py \
  --results-dirs runs/2025-08-01/results runs/2025-09-12/results \
  --config-yaml configs/model.yml \
  --dist HalfNormal \
  --default-sigma 2.0 \
  --out suggested_priors.yml \
  --verbose

  1. Inspect suggested_priors.yml and paste the beta_channel section into your model config. If you prefer a Normal prior, rerun with --dist Normal to get mu + sigma lists.

Customisation

  • Sigma Value: Change WEAKLY_INFORMATIVE_SIGMA in the script for a different prior width.

  • Default Mu: Modify DEFAULT_MU in the script for missing channels.

  • Historical Data Source: Adapt the script to load data from CSV files or databases instead of hardcoded dictionaries.

  • Prior Distribution: Prefer HalfNormal for strictly non‑negative channel effects (AMMM default). Use Normal if you intend to center around positive/negative means. The CLI supports both and converts mu → sigma when using HalfNormal.