Discussion of conformal prediction for hierarchical uncertainty

7th September 2023 @ IIF Reconciliation Workshop

Mitchell O’Hara-Wild, Monash University

Summary

Methodology

• Estimate non-conformity of h-step forecasts with hold-out sets
• Obtain quantiles from the non-conformity over the horizon

Innovations

• Great to see more work on quantifying uncertainty
• Especially the application of a general methodology
• A useful approach for short-term h-step variance

Improvement seems domain & model specific

Thoughts

Potential developments

• Compatibility with black box methods is noted but not shown
• How well does this technique apply to LGBM and TimeGPT?

Some discussion questions

• How does this compare to other boostrap sample paths for reconciliation?
• How costly (in time complexity) is this method?
• How does the performance compare with other h-step forecast variance estimates?