scipy/acceptable_failure_rate.patch

diff --git a/scipy/conftest.py b/scipy/conftest.py
index e5fac23..1a971e8 100644
--- a/scipy/conftest.py
+++ b/scipy/conftest.py
@@ -37,3 +38,15 @@ def check_fpu_mode(request):
         warnings.warn("FPU mode changed from {0:#x} to {1:#x} during "
                       "the test".format(old_mode, new_mode),
                       category=FPUModeChangeWarning, stacklevel=0)
+
+
+ACCEPTABLE_FAILURE_RATE = int(os.environ.get('ACCEPTABLE_FAILURE_RATE', 0))
+
+
+@pytest.hookimpl()
+def pytest_sessionfinish(session, exitstatus):
+    if exitstatus != 1:
+        return
+    failure_rate = (100.0 * session.testsfailed) / session.testscollected
+    if failure_rate <= ACCEPTABLE_FAILURE_RATE:
+        session.exitstatus = 0
Don't ignore the tests results but rather have a tolerance rate Historically, we ignored the test results as we don't have the manpower to investigate the failures on various architectures. Since most of the test pass, I've added a 1% failure tolerance on everything except x86_64, so we do no longer need to ignore the results. This makes it easier to spat that it's no longer a couple test that fails on some random architecture but rather a more severe problem. In the process I had to disable test_decomp on ppc64le, because it segfaults. 2018-06-23 21:00:13 +00:00			`diff --git a/scipy/conftest.py b/scipy/conftest.py`
Update to 1.2.0 2019-02-06 23:30:32 +00:00			`index e5fac23..1a971e8 100644`
Don't ignore the tests results but rather have a tolerance rate Historically, we ignored the test results as we don't have the manpower to investigate the failures on various architectures. Since most of the test pass, I've added a 1% failure tolerance on everything except x86_64, so we do no longer need to ignore the results. This makes it easier to spat that it's no longer a couple test that fails on some random architecture but rather a more severe problem. In the process I had to disable test_decomp on ppc64le, because it segfaults. 2018-06-23 21:00:13 +00:00			`--- a/scipy/conftest.py`
			`+++ b/scipy/conftest.py`
Update to 1.2.0 2019-02-06 23:30:32 +00:00			`@@ -37,3 +38,15 @@ def check_fpu_mode(request):`
Don't ignore the tests results but rather have a tolerance rate Historically, we ignored the test results as we don't have the manpower to investigate the failures on various architectures. Since most of the test pass, I've added a 1% failure tolerance on everything except x86_64, so we do no longer need to ignore the results. This makes it easier to spat that it's no longer a couple test that fails on some random architecture but rather a more severe problem. In the process I had to disable test_decomp on ppc64le, because it segfaults. 2018-06-23 21:00:13 +00:00			`warnings.warn("FPU mode changed from {0:#x} to {1:#x} during "`
			`"the test".format(old_mode, new_mode),`
			`category=FPUModeChangeWarning, stacklevel=0)`
			`+`
			`+`
			`+ACCEPTABLE_FAILURE_RATE = int(os.environ.get('ACCEPTABLE_FAILURE_RATE', 0))`
			`+`
			`+`
			`+@pytest.hookimpl()`
			`+def pytest_sessionfinish(session, exitstatus):`
acceptable_failure_rate.patch: Don't use pytest private API The private API used here was removed in pytest 5. There is a public API now, pytest.ExitCode.TESTS_FAILED, however that is not yet available in pytest 4. We hardcode the value 1 instead, it is unlikely to be changed. Once pytest 4 is gone, we can switch to the new API. See https://github.com/pytest-dev/pytest/issues/5125 2020-01-02 15:19:30 +00:00			`+ if exitstatus != 1:`
Don't ignore the tests results but rather have a tolerance rate Historically, we ignored the test results as we don't have the manpower to investigate the failures on various architectures. Since most of the test pass, I've added a 1% failure tolerance on everything except x86_64, so we do no longer need to ignore the results. This makes it easier to spat that it's no longer a couple test that fails on some random architecture but rather a more severe problem. In the process I had to disable test_decomp on ppc64le, because it segfaults. 2018-06-23 21:00:13 +00:00			`+ return`
			`+ failure_rate = (100.0 * session.testsfailed) / session.testscollected`
			`+ if failure_rate <= ACCEPTABLE_FAILURE_RATE:`
			`+ session.exitstatus = 0`