Welcome to calpit’s documentation!

Overview

calpit is a Python package for diagnosing and recalibrating conditional density estimates. The package is built on top of Pytorch (with other ML backends to be added soon) and provides a simple and flexible interface matching the scikit-learn API.

Basic Usage

The following is a basic recipe for using the calpit package:

from calpit import CalPit #import the CalPit class

calpit_model = CalPit(model=model) #Any Pytorch model CalPit class

trained_model = calpit_model.fit(x_calib,y_calib, cde_cali,y_grid) #Fit the model with a calibration dataset

pp_result = calpit_model.predict(x_test, cov_grid) #Predict the local PIT distribution for a test dataset

new_cde = calpit_model.transform(x_test, cde_test, y_grid) #Recalibrate the conditional density estimate for a test dataset

Installation

To install the current release of the package, you can run the following command:

>> pip install calpit

To install the latest version of the code from Github, you can run the following command:

>> pip install git+https://github.com/lee-group-cmu/Cal-PIT

If you would like to install the package for development purposes, you can clone the repository and install the package in editable mode:

>> git clone https://github.com/lee-group-cmu/Cal-PIT.git
>> cd Cal-PIT
>> pip install -e .

Note

  • The package is intended for use with Python 3.10 or later.

  • Pytorch is a required dependency for calpit. Please follow the instructions on the Pytorch website to install the appropriate version for your system.

References

The calpit package is based on the work described in following papers:

  • Dey et al. [2021] and Dey et al. [2022] which introduces the recalibration framework for conditional density estimates.

  • Zhao et al. [2021], which introduces diagnostics for conditional density estimation methods.

[1]

Biprateep Dey, Jeffrey A. Newman, Brett H. Andrews, Rafael Izbicki, Ann B. Lee, David Zhao, Markus Michael Rau, and Alex I. Malz. Re-calibrating Photometric Redshift Probability Distributions Using Feature-space Regression. arXiv e-prints, pages arXiv:2110.15209, October 2021. arXiv:2110.15209, doi:10.48550/arXiv.2110.15209.

[2]

Biprateep Dey, David Zhao, Jeffrey A. Newman, Brett H. Andrews, Rafael Izbicki, and Ann B. Lee. Conditionally Calibrated Predictive Distributions by Probability-Probability Map: Application to Galaxy Redshift Estimation and Probabilistic Forecasting. arXiv e-prints, pages arXiv:2205.14568, May 2022. arXiv:2205.14568, doi:10.48550/arXiv.2205.14568.

[3]

David Zhao, Niccolò Dalmasso, Rafael Izbicki, and Ann B. Lee. Diagnostics for Conditional Density Models and Bayesian Inference Algorithms. arXiv e-prints, pages arXiv:2102.10473, February 2021. arXiv:2102.10473, doi:10.48550/arXiv.2102.10473.