Re: numpy 1.26.4 (python 3.12 and cython 3 support)

On 27 August 2024 17:03:10 BST, Stuart Henderson <stu@spacehopper.org> wrote:
>On 2024/08/21 21:32, Daniel Dickman wrote:
>>
>>
>> On Sat, 17 Aug 2024, Theo Buehler wrote:
>>
>> > On Tue, Aug 13, 2024 at 06:20:27PM -0400, Daniel Dickman wrote:
>> > > Here's a possible update of numpy to 1.26.4.
>> > >
>> > > The build switches to meson in this release. And this release is the first
>> > > one to support Python 3.12 and also the first to officialy support Cython
>> > > 3.
>> > >
>> > > This should probably be put into bulks on multiple amd64 and other archs
>> > > before being committed.
>> >
>> > After working around a debug package issue in numpy itself, this
>> > completed a bulk without fallout. Further testing on other
>> > architectures should probably wait until this and issues with
>> > regress is sorted.
>> >
>>
>> Thanks to feedback from tb@, new diff below with 2 changes:
>>
>> - add "DWZ=:" to fix the build of debug packages, this is an issue on both
>> Python 3.11 and 3.12 likely due to 128 bit math.
>>
>> - disable tests. The regress tests work fine under Python 3.12 but fail
>> under Python 3.11 due to what looks to me like incomplete refactoring
>> of code after numpy.distutils was removed from numpy.
>>
>> Believe tb@ ran a bulk on amd64 with no issues, so !amd64 bulks would be
>> very useful at this point.
>
>Bulk builds on the archs which are most likely to have problems is
>really unlikely to happen - few people have the type of machines
>suitable for running them in any reasonable timeframe.
>
>Results from building and "make test" on other archs (especially
>sparc64) are easier and would be useful.
>
>The extra patches below fix things so tests can run on 3.11 (remove
>NO_TEST from Makefile too). "make test" on amd64 Py 3.11 results below
>the diff (summary: 31 failed, 35649 passed, 2372 skipped, 1305
>deselected, 33 xfailed, 1 xpassed, 21 warnings in 207.79s"; I'll run
>it on aarch64 when my machine's free and reply if there are many
>differences. Maybe i386 too when the current bulk finishes (a day or
>two).
>
>
>Index: patches/patch-numpy__pytesttester_py
>===================================================================
>RCS file: patches/patch-numpy__pytesttester_py
>diff -N patches/patch-numpy__pytesttester_py
>--- /dev/null 1 Jan 1970 00:00:00 -0000
>+++ patches/patch-numpy__pytesttester_py 27 Aug 2024 15:50:18 -0000
>@@ -0,0 +1,14 @@
>+1.26.4 no longer has numpy.distutils but tests still reference it
>+
>+Index: numpy/_pytesttester.py
>+--- numpy/_pytesttester.py.orig
>++++ numpy/_pytesttester.py
>+@@ -135,7 +135,7 @@ class PytestTester:
>+ # offset verbosity. The "-q" cancels a "-v".
>+ pytest_args += ["-q"]
>+
>+- if sys.version_info < (3, 12):
>++ if sys.version_info < (3, 11):
>+ with warnings.catch_warnings():
>+ warnings.simplefilter("always")
>+ # Filter out distutils cpu warnings (could be localized to
>Index: patches/patch-numpy_tests_test_public_api_py
>===================================================================
>RCS file: patches/patch-numpy_tests_test_public_api_py
>diff -N patches/patch-numpy_tests_test_public_api_py
>--- /dev/null 1 Jan 1970 00:00:00 -0000
>+++ patches/patch-numpy_tests_test_public_api_py 27 Aug 2024 15:50:18 -0000
>@@ -0,0 +1,41 @@
>+1.26.4 no longer has numpy.distutils but tests still reference it
>+
>+Index: numpy/tests/test_public_api.py
>+--- numpy/tests/test_public_api.py.orig
>++++ numpy/tests/test_public_api.py
>+@@ -159,7 +159,7 @@ PUBLIC_MODULES = ['numpy.' + s for s in [
>+ "typing.mypy_plugin",
>+ "version" # Should be removed for NumPy 2.0
>+ ]]
>+-if sys.version_info < (3, 12):
>++if sys.version_info < (3, 11):
>+ PUBLIC_MODULES += [
>+ 'numpy.' + s for s in [
>+ "distutils",
>+@@ -240,7 +240,7 @@ PRIVATE_BUT_PRESENT_MODULES = ['numpy.' + s for s in [
>+ "random.bit_generator",
>+ "testing.print_coercion_tables",
>+ ]]
>+-if sys.version_info < (3, 12):
>++if sys.version_info < (3, 11):
>+ PRIVATE_BUT_PRESENT_MODULES += [
>+ 'numpy.' + s for s in [
>+ "distutils.armccompiler",
>+@@ -335,7 +335,7 @@ SKIP_LIST = [
>+ "numpy.core.generate_numpy_api",
>+ "numpy.core.umath_tests",
>+ ]
>+-if sys.version_info < (3, 12):
>++if sys.version_info < (3, 11):
>+ SKIP_LIST += ["numpy.distutils.msvc9compiler"]
>+
>+
>+@@ -380,7 +380,7 @@ SKIP_LIST_2 = [
>+ 'numpy.matlib.ctypeslib',
>+ 'numpy.matlib.ma',
>+ ]
>+-if sys.version_info < (3, 12):
>++if sys.version_info < (3, 11):
>+ SKIP_LIST_2 += [
>+ 'numpy.distutils.log.sys',
>+ 'numpy.distutils.log.logging',
>Index: patches/patch-numpy_typing_tests_test_isfile_py
>===================================================================
>RCS file: patches/patch-numpy_typing_tests_test_isfile_py
>diff -N patches/patch-numpy_typing_tests_test_isfile_py
>--- /dev/null 1 Jan 1970 00:00:00 -0000
>+++ patches/patch-numpy_typing_tests_test_isfile_py 27 Aug 2024 15:50:18 -0000
>@@ -0,0 +1,14 @@
>+1.26.4 no longer has numpy.distutils but tests still reference it
>+
>+Index: numpy/typing/tests/test_isfile.py
>+--- numpy/typing/tests/test_isfile.py.orig
>++++ numpy/typing/tests/test_isfile.py
>+@@ -21,7 +21,7 @@ FILES = [
>+ ROOT / "random" / "__init__.pyi",
>+ ROOT / "testing" / "__init__.pyi",
>+ ]
>+-if sys.version_info < (3, 12):
>++if sys.version_info < (3, 11):
>+ FILES += [ROOT / "distutils" / "__init__.pyi"]
>+
>+
>
>$ make test
>===> Regression tests for py3-numpy-1.26.4
>cd /pobj/py-numpy-1.26.4-python3 && CC=cc PYTHONUSERBASE= PORTSDIR="/usr/ports" LIBTOOL="/usr/bin/libtool" CCACHE_DIR=/usr/obj/ports/.ccache CCACHE_SLOPPINESS=pch_defines,time_macros PATH='/pobj/py-numpy-1.26.4-python3/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin' PREFIX='/usr/local' LOCALBASE='/usr/local' X11BASE='/usr/X11R6' CFLAGS='-O2 -pipe -g' TRUEPREFIX='/usr/local' DESTDIR='' HOME='/py-numpy-1.26.4_writes_to_HOME' PICFLAG="-fpic" BINGRP=bin BINOWN=root BINMODE=755 NONBINMODE=644 DIRMODE=755 INSTALL_COPY=-c INSTALL_STRIP= MANGRP=bin MANOWN=root MANMODE=644 BSD_INSTALL_PROGRAM="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 755" BSD_INSTALL_SCRIPT="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 755" BSD_INSTALL_DATA="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 644" BSD_INSTALL_MAN="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 644" BSD_INSTALL_PROGRAM_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_SCRIPT_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_DATA_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_MAN_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" /usr/local/bin/python3.11 -c 'import numpy ; numpy.test(verbose=2)'
>NumPy version 1.26.4
>NumPy relaxed strides checking option: True
>NumPy CPU features: SSE SSE2 SSE3 SSSE3* SSE41* POPCNT* SSE42* AVX* F16C* FMA3* AVX2* AVX512F? AVX512CD? AVX512_KNL? AVX512_KNM? AVX512_SKX? AVX512_CLX? AVX512_CNL? AVX512_ICL?
>============================= test session starts ==============================
>platform openbsd7 -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
>benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
>rootdir: /usr/obj/ports/py-numpy-1.26.4-python3
>plugins: subtests-0.12.1, benchmark-4.0.0, flaky-3.8.1, filter-subpackage-0.2.0, timeout-1.4.2, typeguard-4.3.0, hypothesis-6.47.1, anyio-3.7.1, cov-5.0.0, mock-3.14.0, doctestplus-1.2.1
>collected 39391 items / 1305 deselected / 38086 selected
>
>array_api/tests/test_array_object.py ........................ [ 0%]
>array_api/tests/test_creation_functions.py .............. [ 0%]
>array_api/tests/test_data_type_functions.py ..... [ 0%]
>array_api/tests/test_elementwise_functions.py .. [ 0%]
>array_api/tests/test_indexing_functions.py .... [ 0%]
>array_api/tests/test_manipulation_functions.py ... [ 0%]
>array_api/tests/test_set_functions.py .. [ 0%]
>array_api/tests/test_sorting_functions.py .... [ 0%]
>array_api/tests/test_validation.py ... [ 0%]
>compat/tests/test_compat.py . [ 0%]
>core/tests/test__exceptions.py ........... [ 0%]
>core/tests/test_abc.py ..... [ 0%]
>core/tests/test_api.py ................................................. [ 0%]
>........... [ 0%]
>core/tests/test_argparse.py ..... [ 0%]
>core/tests/test_array_coercion.py ...................................... [ 0%]
>.......................x................................................ [ 0%]
>........................................................................ [ 0%]
>.......................... [ 0%]
>core/tests/test_arraymethod.py ........................................ [ 1%]
>core/tests/test_arrayprint.py ..x....................................... [ 1%]
>..................... [ 1%]
>core/tests/test_casting_floatingpoint_errors.py ........................ [ 1%]
>........................................................................ [ 1%]
>........................................................................ [ 1%]
>............... [ 1%]
>core/tests/test_casting_unittests.py ................................... [ 1%]
>........................................................................ [ 1%]
>........................................................................ [ 2%]
>........................................................................ [ 2%]
>................................................................ [ 2%]
>core/tests/test_conversion_utils.py ........................... [ 2%]
>core/tests/test_cpu_dispatcher.py . [ 2%]
>core/tests/test_cpu_features.py .......ssss [ 2%]
>core/tests/test_custom_dtypes.py ................................... [ 2%]
>core/tests/test_cython.py ....... [ 2%]
>core/tests/test_datetime.py ............................................ [ 2%]
>........................................................................ [ 3%]
>........................................................................ [ 3%]
>...........................................x..........x..x..........x... [ 3%]
>. [ 3%]
>core/tests/test_defchararray.py ........................................ [ 3%]
>................................................. [ 3%]
>core/tests/test_deprecations.py ........................................ [ 3%]
>........................................................................ [ 3%]
>........... [ 3%]
>core/tests/test_dlpack.py .............................................. [ 4%]
>............. [ 4%]
>core/tests/test_dtype.py ............................................... [ 4%]
>................................................ssss.................... [ 4%]
>........................................................................ [ 4%]
>........................................................................ [ 4%]
>........................................................................ [ 4%]
>........................................................................ [ 5%]
>........................................................................ [ 5%]
>........................................................................ [ 5%]
>........................................................................ [ 5%]
>........................................................................ [ 5%]
>........................................................................ [ 6%]
>........................................................................ [ 6%]
>........................................................................ [ 6%]
>........................................................................ [ 6%]
>................................................. [ 6%]
>core/tests/test_einsum.py .............................................. [ 6%]
>..................... [ 6%]
>core/tests/test_errstate.py .... [ 7%]
>core/tests/test_extint128.py ............ [ 7%]
>core/tests/test_function_base.py ..................................... [ 7%]
>core/tests/test_getlimits.py ................ [ 7%]
>core/tests/test_half.py ..................................... [ 7%]
>core/tests/test_hashtable.py ......... [ 7%]
>core/tests/test_indexerrors.py ........ [ 7%]
>core/tests/test_indexing.py ............................................ [ 7%]
>....................................... [ 7%]
>core/tests/test_item_selection.py ...................................... [ 7%]
>........................................................................ [ 7%]
>........................................................................ [ 8%]
>........................................................................ [ 8%]
>............ [ 8%]
>core/tests/test_limited_api.py . [ 8%]
>core/tests/test_longdouble.py .Fsssss.s......ssss.sss...........s. [ 8%]
>core/tests/test_machar.py . [ 8%]
>core/tests/test_mem_overlap.py .................. [ 8%]
>core/tests/test_mem_policy.py .....s.... [ 8%]
>core/tests/test_memmap.py ................... [ 8%]
>core/tests/test_multiarray.py .......................................... [ 8%]
>........................................................................ [ 8%]
>.......................xx............................................... [ 8%]
>........................................................................ [ 9%]
>........................................................................ [ 9%]
>...........................................s............................ [ 9%]
>........................................................................ [ 9%]
>........................................................................ [ 9%]
>........................................................................ [ 10%]
>........................................................................ [ 10%]
>........................................................................ [ 10%]
>........................................................................ [ 10%]
>........................................................................ [ 10%]
>........................................................................ [ 11%]
>........................................................................ [ 11%]
>........................................................................ [ 11%]
>........................................................................ [ 11%]
>........................................................................ [ 11%]
>........................................................................ [ 11%]
>........................................................................ [ 12%]
>........................................................................ [ 12%]
>........................................................................ [ 12%]
>........................................................................ [ 12%]
>........................................................................ [ 12%]
>........................................................................ [ 13%]
>........................................................................ [ 13%]
>........................................................................ [ 13%]
>........................................................................ [ 13%]
>........................................................................ [ 13%]
>........................................................................ [ 14%]
>........................................................................ [ 14%]
>........................................................................ [ 14%]
>........................................................................ [ 14%]
>........................................................................ [ 14%]
>........................................................................ [ 14%]
>........................................................................ [ 15%]
>........................................................................ [ 15%]
>........................................................................ [ 15%]
>........................................................................ [ 15%]
>........................................................................ [ 15%]
>........................................................................ [ 16%]
>........................................................................ [ 16%]
>........................................................................ [ 16%]
>........................................................................ [ 16%]
>........................................................................ [ 16%]
>........................................................................ [ 17%]
>........................................................................ [ 17%]
>........................................................................ [ 17%]
>........................................................................ [ 17%]
>........................................................................ [ 17%]
>........................................................................ [ 18%]
>........................................................................ [ 18%]
>........................................................................ [ 18%]
>........................................................................ [ 18%]
>........................................................................ [ 18%]
>........................................................................ [ 18%]
>........................................................................ [ 19%]
>........................................................................ [ 19%]
>........................................................................ [ 19%]
>........................................................................ [ 19%]
>........................................................................ [ 19%]
>........................................................................ [ 20%]
>........................................................................ [ 20%]
>........................................................................ [ 20%]
>........................................................................ [ 20%]
>........................................................................ [ 20%]
>........................................................................ [ 21%]
>........................................................................ [ 21%]
>........................................................................ [ 21%]
>........................................................................ [ 21%]
>........................................................................ [ 21%]
>........................................................................ [ 21%]
>........................................................................ [ 22%]
>........................................................................ [ 22%]
>........................................................................ [ 22%]
>........................................................................ [ 22%]
>........................................................................ [ 22%]
>........................................................................ [ 23%]
>........................................................................ [ 23%]
>........................................................................ [ 23%]
>........................................................................ [ 23%]
>........................................................................ [ 23%]
>........................................................................ [ 24%]
>........................................................................ [ 24%]
>........................................................................ [ 24%]
>........................................................................ [ 24%]
>........................................................................ [ 24%]
>........................................................................ [ 25%]
>........................................................................ [ 25%]
>........................................................................ [ 25%]
>........................................................................ [ 25%]
>........................................................................ [ 25%]
>........................................................................ [ 25%]
>........................................................................ [ 26%]
>........................................................................ [ 26%]
>........................................................................ [ 26%]
>........................................................................ [ 26%]
>........................................................................ [ 26%]
>........................................................................ [ 27%]
>........................................................................ [ 27%]
>........................................................................ [ 27%]
>........................................................................ [ 27%]
>........................................................................ [ 27%]
>........................................................................ [ 28%]
>........................................................................ [ 28%]
>........................................................................ [ 28%]
>........................................................................ [ 28%]
>........................................................................ [ 28%]
>........................................................................ [ 28%]
>........................................................................ [ 29%]
>........................................................................ [ 29%]
>........................................................................ [ 29%]
>........................................................................ [ 29%]
>........................................................................ [ 29%]
>........................................................................ [ 30%]
>........................................................................ [ 30%]
>........................................................................ [ 30%]
>........................................................................ [ 30%]
>........................................................................ [ 30%]
>........................................................................ [ 31%]
>........................................................................ [ 31%]
>........................................................................ [ 31%]
>........................................................................ [ 31%]
>........................................................................ [ 31%]
>........................................................................ [ 32%]
>........................................................................ [ 32%]
>........................................................................ [ 32%]
>........................................................................ [ 32%]
>................................ [ 32%]
>core/tests/test_nditer.py .............................................. [ 32%]
>........................................................................ [ 32%]
>........................................................................ [ 33%]
>........................................................................ [ 33%]
>........................................................................ [ 33%]
>........................................................................ [ 33%]
>........................................................................ [ 33%]
>........................................................................ [ 34%]
>........................................................................ [ 34%]
>........................................................................ [ 34%]
>........................................................................ [ 34%]
>.................................... [ 34%]
>core/tests/test_nep50_promotions.py ................s............... [ 34%]
>core/tests/test_numeric.py ..............................x...x.......... [ 34%]
>..................s..s.................................................. [ 35%]
>........................................................................ [ 35%]
>...........................................................X............ [ 35%]
>........................................................................ [ 35%]
>.................... [ 35%]
>core/tests/test_numerictypes.py ........................................ [ 35%]
>........................................................................ [ 36%]
>...... [ 36%]
>core/tests/test_numpy_2_0_compat.py ... [ 36%]
>core/tests/test_overrides.py ........................................... [ 36%]
>........................................................................ [ 36%]
>....................... [ 36%]
>core/tests/test_print.py .....................F [ 36%]
>core/tests/test_protocols.py .. [ 36%]
>core/tests/test_records.py ......................................... [ 36%]
>core/tests/test_regression.py .......................................... [ 36%]
>........................................................................ [ 36%]
>........................................................................ [ 37%]
>........................................................................ [ 37%]
>.........s.............................................................. [ 37%]
>........................................................................ [ 37%]
>............. [ 37%]
>core/tests/test_scalar_ctors.py ........................................ [ 37%]
>........................................................................ [ 38%]
>.................................................. [ 38%]
>core/tests/test_scalar_methods.py ............................s......... [ 38%]
>........................................................................ [ 38%]
>. [ 38%]
>core/tests/test_scalarbuffer.py ........................................ [ 38%]
>...................................... [ 38%]
>core/tests/test_scalarinherit.py ..... [ 38%]
>core/tests/test_scalarmath.py .........................s................ [ 38%]
>........................................................................ [ 38%]
>........................................................................ [ 39%]
>......................ssssss............................................ [ 39%]
>........................................................................ [ 39%]
>...... [ 39%]
>core/tests/test_scalarprint.py ....Fs [ 39%]
>core/tests/test_shape_base.py .......................................... [ 39%]
>........................................................................ [ 39%]
>........................................................................ [ 40%]
>........................ [ 40%]
>core/tests/test_simd.py ................................................ [ 40%]
>........................................................................ [ 40%]
>........................................................................ [ 40%]
>........................................................................ [ 40%]
>........................................................................ [ 40%]
>........................................................................ [ 41%]
>........................................................................ [ 41%]
>........................................................................ [ 41%]
>........................................................................ [ 41%]
>........................................................................ [ 41%]
>........................................................................ [ 42%]
>........................................................................ [ 42%]
>........................................................................ [ 42%]
>........................................................................ [ 42%]
>.........
Can you build and test scipy using this numpy port?

Dima