calpit.metrics

Module Contents

Functions

cde_loss(→ tuple)

Calculates conditional density estimation loss on holdout data.

kolmogorov_smirnov_statistic(→ numpy.ndarray)

Calculate the Kolmogorov-Smirnov statistic between two cumulative distribution functions (CDFs).

cramer_von_mises_statistic(→ numpy.ndarray)

Calculates the Cramer-von Mises statistic between two cumulative distribution functions (CDFs).

anderson_darling_statistic(→ numpy.ndarray)

Calculates the Anderson-Darling statistic between two cumulative distribution functions (CDFs).

probability_integral_transform(→ numpy.ndarray)

Calculates the Probability Integral Transform (PIT) based on Conditional Density Estimates (CDE).
cde_loss(cde_estimates: numpy.ndarray, y_grid: numpy.ndarray, y_test: numpy.ndarray) → tuple[source]

Calculates conditional density estimation loss on holdout data.

Parameters:

  • cde_estimates (numpy.array) – An array where each row is a density estimate on y_grid.

  • z_grid (numpy.array) – An array of the grid points at which cde_estimates is evaluated.

  • z_test (numpy.array) – An array of the true y values corresponding to the rows of cde_estimates.

Returns:

A tuple containing the loss and the standard error of the loss.

Return type:

tuple

Raises:

ValueError – If the dimensions of the input tensors are not compatible.

kolmogorov_smirnov_statistic(cdf_test: numpy.ndarray, cdf_ref: numpy.ndarray) → numpy.ndarray[source]

Calculate the Kolmogorov-Smirnov statistic between two cumulative distribution functions (CDFs).

Parameters: cdf_test (np.ndarray): CDF of the test distribution. cdf_ref (np.ndarray): CDF of the reference distribution on the same grid.

Returns: np.ndarray: The Kolmogorov-Smirnov statistic.

cramer_von_mises_statistic(cdf_test: numpy.ndarray, cdf_ref: numpy.ndarray) → numpy.ndarray[source]

Calculates the Cramer-von Mises statistic between two cumulative distribution functions (CDFs).

Parameters:

  • cdf_test (np.ndarray) – CDF of the test distribution.

  • cdf_ref (np.ndarray) – CDF of the reference distribution on the same grid.

Returns:

The Cramer-von Mises statistic.

Return type:

np.ndarray

anderson_darling_statistic(cdf_test: numpy.ndarray, cdf_ref: numpy.ndarray, n_tot: int = 1) → numpy.ndarray[source]

Calculates the Anderson-Darling statistic between two cumulative distribution functions (CDFs).

Parameters:

  • cdf_test (np.ndarray) – CDF of the test distribution (1D array).

  • cdf_ref (np.ndarray) – CDF of the reference distribution on the same grid (1D array).

  • n_tot (int) – Scaling factor equal to the number of PDFs used to construct ECDF.

Returns:

The Anderson-Darling statistic.

Return type:

np.ndarray

probability_integral_transform(cde: numpy.ndarray, y_grid: numpy.ndarray, y_test: numpy.ndarray) → numpy.ndarray[source]

Calculates the Probability Integral Transform (PIT) based on Conditional Density Estimates (CDE).

Parameters:

  • cde (np.ndarray) – A numpy array of conditional density estimates. Each row corresponds to an observation, each column corresponds to a grid point.

  • y_grid (np.ndarray) – A numpy array of the grid points at which cde is evaluated.

  • y_test (np.ndarray) – A numpy array of the true y values corresponding to the rows of cde.

Returns:

A numpy array of PIT values.

Return type:

np.ndarray

Raises:

ValueError – If the number of samples in cde is not the same as in y_test, or if the number of grid points in cde is not the same as in y_grid.