ToyCalculator

class pyhf.infer.calculators.ToyCalculator(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', ntoys=2000, track_progress=True)

Bases: object

The Toy-based Calculator.

__init__(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', ntoys=2000, track_progress=True)

Toy-based Calculator.

Parameters

• data (tensor) – The observed data.

• pdf (Model) – The statistical model adhering to the schema model.json.

• init_pars (tensor of float) – The starting values of the model parameters for minimization.

• par_bounds (tensor) – The extrema of values the model parameters are allowed to reach in the fit. The shape should be (n, 2) for n model parameters.

• fixed_params (tensor of bool) – The flag to set a parameter constant to its starting value during minimization.

• test_stat (str) – The test statistic to use as a numerical summary of the data: 'qtilde', 'q', or 'q0'. 'qtilde' (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), as defined under the Wald approximation in Equation (62) of [1007.1727] (qmu_tilde()), 'q' performs the calculation using the test statistic \(q_{\mu}\) (qmu()), and 'q0' perfoms the calculation using the discovery test statistic \(q_{0}\) (q0()).

• ntoys (int) – Number of toys to use (how many times to sample the underlying distributions).

• track_progress (bool) – Whether to display the tqdm progress bar or not (outputs to stderr).

Returns

The calculator for toy-based quantities.

Return type

ToyCalculator

Methods

distributions(poi_test, track_progress=None)

Probability Distributions of the test statistic value under the signal + background and background-only hypothesis.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.hepdata_like(
...     signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> sig_plus_bkg_dist.pvalue(mu_test), bkg_dist.pvalue(mu_test)
(array(0.14), array(0.76))

Parameters

• poi_test (float or tensor) – The value for the parameter of interest.

• track_progress (bool) – Whether to display the tqdm progress bar or not (outputs to stderr)

Returns

The distributions under the hypotheses.

Return type

Tuple (EmpiricalDistribution)

expected_pvalues(sig_plus_bkg_distribution, bkg_only_distribution)

Calculate the \(\mathrm{CL}_{s}\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\left(\mu=0\right)\) at \((-2,-1,0,1,2)\sigma\).

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.hepdata_like(
...     signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> CLsb_exp_band, CLb_exp_band, CLs_exp_band = toy_calculator.expected_pvalues(sig_plus_bkg_dist, bkg_dist)
>>> CLs_exp_band
[array(0.), array(0.), array(0.06186224), array(0.28450033), array(1.)]

Parameters

• sig_plus_bkg_distribution (EmpiricalDistribution) – The distribution for the signal + background hypothesis.

• bkg_only_distribution (EmpiricalDistribution) – The distribution for the background-only hypothesis.

Returns

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type

Tuple (tensor)

pvalues(teststat, sig_plus_bkg_distribution, bkg_only_distribution)

Calculate the \(p\)-values for the observed test statistic under the signal + background and background-only model hypotheses.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.hepdata_like(
...     signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> q_tilde = toy_calculator.teststatistic(mu_test)
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> CLsb, CLb, CLs = toy_calculator.pvalues(q_tilde, sig_plus_bkg_dist, bkg_dist)
>>> CLsb, CLb, CLs
(array(0.01), array(0.41), array(0.02439024))

Parameters

• teststat (tensor) – The test statistic.

• sig_plus_bkg_distribution (EmpiricalDistribution) – The distribution for the signal + background hypothesis.

• bkg_only_distribution (EmpiricalDistribution) – The distribution for the background-only hypothesis.

Returns

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type

Tuple (tensor)

teststatistic(poi_test)

Compute the test statistic for the observed data under the studied model.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.hepdata_like(
...     signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> toy_calculator.teststatistic(mu_test)
array(3.93824492)

Parameters

poi_test (float or tensor) – The value for the parameter of interest.

Returns

The value of the test statistic.

Return type

Tensor