pyhf.infer.hypotest¶
-
pyhf.infer.
hypotest
(poi_test, data, pdf, init_pars=None, par_bounds=None, fixed_params=None, qtilde=False, **kwargs)[source]¶ Compute \(p\)-values and test statistics for a single value of the parameter of interest.
Example
>>> import pyhf >>> pyhf.set_backend("numpy") >>> model = pyhf.simplemodels.hepdata_like( ... signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0] ... ) >>> observations = [51, 48] >>> data = pyhf.tensorlib.astensor(observations + model.config.auxdata) >>> test_poi = 1.0 >>> CLs_obs, CLs_exp_band = pyhf.infer.hypotest( ... test_poi, data, model, qtilde=True, return_expected_set=True ... ) >>> CLs_obs array(0.05251497) >>> CLs_exp_band [array(0.00260626), array(0.01382005), array(0.06445321), array(0.23525644), array(0.57303621)]
- Parameters
poi_test (Number or Tensor) – The value of the parameter of interest (POI)
data (Number or Tensor) – The data considered
pdf (Model) – The statistical model adhering to the schema
model.json
init_pars (tensor) – The initial parameter values to be used for minimization
par_bounds (tensor) – The parameter value bounds to be used for minimization
fixed_params (tensor) – Whether to fix the parameter to the init_pars value during minimization
qtilde (Bool) – When
True
perform the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), as defined under the Wald approximation in Equation (62) of [1007.1727].
- Keyword Arguments
return_tail_probs (bool) – Bool for returning \(\mathrm{CL}_{s+b}\) and \(\mathrm{CL}_{b}\)
return_expected (bool) – Bool for returning \(\mathrm{CL}_{\mathrm{exp}}\)
return_expected_set (bool) – Bool for returning the \((-2,-1,0,1,2)\sigma\) \(\mathrm{CL}_{\mathrm{exp}}\) — the “Brazil band”
- Returns
\(\mathrm{CL}_{s}\): The modified \(p\)-value compared to the given threshold \(\alpha\), typically taken to be \(0.05\), defined in [1007.1727] as
\[\mathrm{CL}_{s} = \frac{\mathrm{CL}_{s+b}}{\mathrm{CL}_{b}} = \frac{p_{s+b}}{1-p_{b}}\]to protect against excluding signal models in which there is little sensitivity. In the case that \(\mathrm{CL}_{s} \leq \alpha\) the given signal model is excluded.
\(\left[\mathrm{CL}_{s+b}, \mathrm{CL}_{b}\right]\): The signal + background model hypothesis \(p\)-value
\[\mathrm{CL}_{s+b} = p_{s+b} = p\left(q \geq q_{\mathrm{obs}}\middle|s+b\right) = \int\limits_{q_{\mathrm{obs}}}^{\infty} f\left(q\,\middle|s+b\right)\,dq = 1 - F\left(q_{\mathrm{obs}}(\mu)\,\middle|\mu'\right)\]and 1 minus the background only model hypothesis \(p\)-value
\[\mathrm{CL}_{b} = 1- p_{b} = p\left(q \geq q_{\mathrm{obs}}\middle|b\right) = 1 - \int\limits_{-\infty}^{q_{\mathrm{obs}}} f\left(q\,\middle|b\right)\,dq = 1 - F\left(q_{\mathrm{obs}}(\mu)\,\middle|0\right)\]for signal strength \(\mu\) and model hypothesis signal strength \(\mu'\), where the cumulative density functions \(F\left(q(\mu)\,\middle|\mu'\right)\) are given by Equations (57) and (65) of [1007.1727] for upper-limit-like test statistic \(q \in \{q_{\mu}, \tilde{q}_{\mu}\}\). Only returned when
return_tail_probs
isTrue
.Note
The definitions of the \(\mathrm{CL}_{s+b}\) and \(\mathrm{CL}_{b}\) used are based on profile likelihood ratio test statistics. This procedure is common in the LHC-era, but differs from procedures used in the LEP and Tevatron eras, as briefly discussed in \(\S\) 3.8 of [1007.1727].
\(\mathrm{CL}_{s,\mathrm{exp}}\): The expected \(\mathrm{CL}_{s}\) value corresponding to the test statistic under the background only hypothesis \(\left(\mu=0\right)\). Only returned when
return_expected
isTrue
.\(\mathrm{CL}_{s,\mathrm{exp}}\) band: The set of expected \(\mathrm{CL}_{s}\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\left(\mu=0\right)\) at \((-2,-1,0,1,2)\sigma\). That is, the \(p\)-values that satisfy Equation (89) of [1007.1727]
\[\mathrm{band}_{N\sigma} = \mu' + \sigma\,\Phi^{-1}\left(1-\alpha\right) \pm N\sigma\]for \(\mu'=0\) and \(N \in \left\{-2, -1, 0, 1, 2\right\}\). These values define the boundaries of an uncertainty band sometimes referred to as the “Brazil band”. Only returned when
return_expected_set
isTrue
.- Return type
Tuple of Floats and lists of Floats