notes

  (

index

)

Towards A Principled Bayesian Workflow by Michael Betancourt in 2020

keywords

See also

Schad, D. J., Betancourt, M., & Vasishth, S. (2019). Toward a principled Bayesian workflow in cognitive science. Retrieved from http://arxiv.org/abs/1904.12765

pymc3 implementation of principled bayesian workflow

computational environment

The related code can be run via Run python session on the remote container.

Contents

1. Questioning authority

  • How do we construct models consistent with our domain expertise that are adequate to capture the structure of the phenomena of interest, and support accurate and relevant inference from observations we can feasibly perform with respect to the specific questions we need to answer in order to make well-informed decisions?

  • Questions

    • Is our model consistent with our domain expertise?

      • __domain expertise consistency__

    • Will our computational tools be sufficient to accurately fit our posteriors?

      • __computational faithfulness__

    • Will our inferences provide enough information to answer our questions?

      • __inferential adequacy__

    • Is our model rich enough to capture the relevant structure of the true data generating process?

      • __model adequacy__

  • 1.1 Domain expertise consistency

    • 1.1.1 Quantifying consequences
    • 1.1.2 Prior pushforward checks
    • 1.1.3 Prior predictive checks

  • 1.2 Computational faithfulness

  • 1.3 Inferential calibration

  • 1.4 Model adequacy

    • 1.4.1
    • 1.4.2
    • 1.4.3 Posterior retrodiction checks
    • 1.4.4 Limitations of posterior retrodiction checks

2. Building a mystery

3. Components of a principled Bayesian workflow

  • Phases of the workflow

    • Pre-model, Pre-data

        1. Conceptual analysis
        1. Define observation space
        1. Construct summary statistics

    • Post-model, Pre-data

        1. Develop model
        1. Construct summary functions
        1. Simulate Bayesian ensemble
        1. Prior checks
        1. Configure algorithm
        1. Fit simulated ensemble
        1. Algorithmic calibration
        1. Inferential calibration

    • Post-model, Post-data

        1. Fit observed data
        1. Diagnose posterior fit
        1. Posterior retrodictive checks

  • Example workflow executions

    • Incompatible model assumptions

4. Close enough for an effective demonstration

  • 4.1 First iteration

    • 4.1.1 Conceptual analysis
    • 4.1.2 Define observational space
    • 4.1.3 Construct summary statistics
    • 4.1.4 Model development
    • 4.1.5 Construct summary functions
    • 4.1.6 Simulate Bayesian ensemble
    • 4.1.7 Prior checks
    • 4.1.8 Configure algorithm
    • 4.1.9 Fit simulated ensemble
    • 4.1.10 Algorithm calibration
    • 4.1.11 Inferential calibration
    • 4.1.12 Fit the observation
    • 4.1.13 Diagnose posterior fit
    • 4.1.14 Posterior retrodictive checks

  • 4.2 Second iteration

Links to this note