#11 [crashes] Update to 1.3.1, drop Py2 packages (not supported)
Closed 2 months ago by lbalhar. Opened 3 months ago by churchyard.
rpms/ churchyard/scipy 1.3.1  into  master

file modified
+18 -60

@@ -6,8 +6,8 @@ 

  

  Summary:    Scientific Tools for Python

  Name:       scipy

- Version:    1.2.1

- Release:    8%{?dist}

+ Version:    1.3.1

+ Release:    1%{?dist}

  

  # BSD -- whole package except:

  # Boost -- scipy/special/cephes/scipy_iv.c

@@ -25,8 +25,6 @@ 

  # structure fields accessed in __odrpack.c are now deprecated.

  Patch1:     numpy-deprecation-warnings.patch

  

- BuildRequires: python2-numpy, python2-devel,python2-numpy-f2py

- BuildRequires: python2-pytest

  BuildRequires: fftw-devel, blas-devel, lapack-devel, suitesparse-devel

  %ifarch %{openblas_arches}

  BuildRequires: openblas-devel

@@ -45,11 +43,6 @@ 

  BuildRequires:  python3-pytest-timeout

  

  %if %{with doc}

- %if 0%{?fedora} < 31

- BuildRequires:  python2-sphinx

- BuildRequires:  python2-matplotlib

- BuildRequires:  python2-numpydoc

- %endif

  BuildRequires:  python3-sphinx

  BuildRequires:  python3-matplotlib

  BuildRequires:  python3-numpydoc

@@ -68,23 +61,14 @@ 

  

  %description %_description

  

- %package -n python2-scipy

+ %package -n python3-scipy

  Summary:    Scientific Tools for Python

- Requires:   numpy, f2py

- %{?python_provide:%python_provide python2-scipy}

- # General provides of plain 'scipy' in F24

- Provides:       scipy = %{version}-%{release}

- Provides:       scipy%{?_isa} = %{version}-%{release}

- Obsoletes:      scipy <= 0.16.0

- %description -n python2-scipy  %_description

+ License:    BSD and LGPLv2+

+ Requires:   python3-numpy, python3-f2py

+ %{?python_provide:%python_provide python3-scipy}

+ %description -n python3-scipy %_description

  

  %if %{with doc}

- %package -n python2-scipy-doc

- Summary:    Scientific Tools for Python - documentation

- Requires:   python2-scipy = %{version}-%{release}

- %description -n python2-scipy-doc

- HTML documentation for Scipy

- 

  %package -n python3-scipy-doc

  Summary:    Scientific Tools for Python - documentation

  Requires:   python3-scipy = %{version}-%{release}

@@ -92,12 +76,6 @@ 

  HTML documentation for Scipy

  %endif

  

- %package -n python3-scipy

- Summary:    Scientific Tools for Python

- License:    BSD and LGPLv2+

- Requires:   python3-numpy, python3-f2py

- %{?python_provide:%python_provide python3-scipy}

- %description -n python3-scipy %_description

  

  %prep

  %autosetup -p1 -n %{name}-%{version}%{?rcver}

@@ -129,7 +107,7 @@ 

  rm $(grep -rl '/\* Generated by Cython') PKG-INFO

  

  %build

- for PY in %{python3_version} %{python2_version}; do

+ for PY in %{python3_version}; do

Maybe remove the loop to just keep the loop body?

    env CFLAGS="$RPM_OPT_FLAGS -lm" \

        FFLAGS="$RPM_OPT_FLAGS -fPIC" \

    %ifarch %{openblas_arches}

@@ -143,24 +121,18 @@ 

      build

  

    %if %{with doc}

-   # No python2-sphinx on Fedora 31+

-   if [ $PY == %{python3_version} -o 0%{?fedora} -lt 31 ]

-   then

-     pushd doc

-     export PYTHONPATH=$(echo ../build/lib.linux-*-$PY/)

-     make html SPHINXBUILD=sphinx-build-$PY

-     rm -rf build/html/.buildinfo

-     mv build build-$PY

-     popd

-   fi

+   pushd doc

+   export PYTHONPATH=$(echo ../build/lib.linux-*-$PY/)

+   make html SPHINXBUILD=sphinx-build-$PY

+   rm -rf build/html/.buildinfo

+   mv build build-$PY

+   popd

    %endif

  done

  

  %install

  %py3_install

- %py2_install

  # Some files got ambiguous python shebangs, we fix them after everything else is done

- pathfix.py -pni "%{__python2} %{py2_shbang_opts}" %{buildroot}%{python2_sitearch}

  pathfix.py -pni "%{__python3} %{py3_shbang_opts}" %{buildroot}%{python3_sitearch}

  

  %check

@@ -197,24 +169,6 @@ 

  rm -rf gram{A,B}

  popd

  

- pushd %{buildroot}/%{python2_sitearch}

- %{__python2} -m pytest -k "$k" scipy

- # Remove test remnants

- rm -f gram{A,B}

- popd

- 

- 

- %files -n python2-scipy

- %doc LICENSE.txt

- %{python2_sitearch}/scipy/

- %{python2_sitearch}/*.egg-info

- 

- %if %{with doc} && 0%{?fedora} < 31

- %files -n python2-scipy-doc

- %license LICENSE.txt

- %doc doc/build-%{python2_version}/html

- %endif

- 

  %files -n python3-scipy

  %doc LICENSE.txt

  %{python3_sitearch}/scipy/

@@ -227,6 +181,10 @@ 

  %endif

  

  %changelog

+ * Fri Oct 18 2019 Miro Hrončok <mhroncok@redhat.com> - 1.3.1-1

+ - Update to 1.3.1 (#1674101)

+ - Drop Python 2 packages (not supported by SciPy >= 1.3)

+ 

  * Thu Oct 03 2019 Miro Hrončok <mhroncok@redhat.com> - 1.2.1-8

  - Rebuilt for Python 3.8.0rc1 (#1748018)

  

Maybe remove the loop to just keep the loop body?

The change itself LGTM (I just made a minor suggestion on a loop, you're free to change it or not, it's up to you). I'm fine with dropping Python 2 support, since it is scipy who decided to drop Python 2 support ;-)

But I cannot say if it's ok to upgrade scipy in Fedora today. Is there a risk of breaking other scientific packages which rely on the scipy versions? I guess that it's fine, since we have to upgrade it soon anyway, to reduce the technical debt. And it's rawhide role to ensure that all packages are working all together ;-)

Maybe remove the loop to just keep the loop body?

I've copied this from the previous PR by @orion - it think the git diff is nicer this way and it not that important.

The only package that needed python2-scipy is gnuradio and that is already broken AFAIK. ccing @jskarvad to let him know.

Anyway, x86_64 failed, because 12 tests failed:

_________________________ spatial/tests/test_kdtree.py _________________________
[gw4] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw4' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_compiled::test_in_ball'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw3] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw3' crashed while running 'spatial/tests/test_kdtree.py::test_onetree_query_compiled'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw7] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw7' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_compiled::test_found_all'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw0] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw0' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_l1_compiled::test_in_ball'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw6] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw6' crashed while running 'spatial/tests/test_kdtree.py::test_query_pairs_single_node_compiled'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw8] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw8' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_approx_compiled::test_in_ball'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw9] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw9' crashed while running 'spatial/tests/test_kdtree.py::test_len0_arrays'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw12] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw12' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_l1_compiled::test_found_all'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw11] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw11' crashed while running 'spatial/tests/test_kdtree.py::test_random_ball_vectorized_compiled'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw1] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw1' crashed while running 'spatial/tests/test_kdtree.py::Test_random_ball_approx_compiled::test_found_all'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw14] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw14' crashed while running 'spatial/tests/test_kdtree.py::test_query_ball_point_vector_r'
_________________________ spatial/tests/test_kdtree.py _________________________
[gw16] linux -- Python 3.8.0 /usr/bin/python3
worker 'gw16' crashed while running 'spatial/tests/test_kdtree.py::test_query_ball_point_length'

Not nice.

Btw, the same tests are crashing on other arches as well but only x86_64 has ACCEPTABLE_FAILURE_RATE=0.

Looking at this.

The simplest reproducer from test_kdtree.py:

from test_kdtree import *

# from test_onetree_query_compiled

np.random.seed(0)
n = 100
k = 4
points = np.random.randn(n,k)
T = cKDTree(points)

# from check_onetree_query
d = 0.1


# r = T.query_ball_tree(T, d)
# s = set()
# for i, l in enumerate(r):
#     for j in l:
#         if i < j:
#             s.add((i,j))

T.query_pairs(d)

Error from GDB:

[Detaching after fork from child process 93]
/lib64/../share/gcc-9/python/libstdcxx/v6/xmethods.py:731: SyntaxWarning: list indices must be integers or slices, not str; perhaps you missed a comma?
  refcounts = ['_M_refcount']['_M_pi']
/usr/include/c++/9/bits/stl_vector.h:1120: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::front() [with _Tp = ordered_pair; _Alloc = std::allocator<ordered_pair>; std::vector<_Tp, _Alloc>::reference = ordered_pair&]: Assertion '__builtin_expect(!this->empty(), true)' failed.

Thread 1 "python3-debug" received signal SIGABRT, Aborted.
0x00007ffff7e39625 in raise () from /lib64/libc.so.6

Upstream issue: https://github.com/scipy/scipy/issues/11021

@vstinner it looks similar to what you reported during Python 3.8 beta phase but it's not the same issue.

It seems that the problem is already fixed upstream so I'll backport a patch, do a scratch build and eventually open a new PR.

Pull-Request has been closed by lbalhar

2 months ago