NumPy 1.21.0 Release Notes¶
The NumPy 1.21.0 release highlights are
continued SIMD work covering more functions and platforms,
initial work on the new dtype infrastructure and casting,
universal2 wheels for Python 3.8 and Python 3.9 on Mac,
improved documentation,
improved annotations,
new
PCG64DXSMbitgenerator for random numbers.
In addition there are the usual large number of bug fixes and other improvements.
The Python versions supported for this release are 3.7-3.9. Official support for Python 3.10 will be added when it is released.
Warning
There are unresolved problems compiling NumPy 1.20.0 with gcc-11.1.
Optimization level -O3 results in many incorrect warnings when running the tests.
On some hardware NumPY will hang in an infinite loop.
New functions¶
Add PCG64DXSM BitGenerator¶
Uses of the PCG64 BitGenerator in a massively-parallel context have been
shown to have statistical weaknesses that were not apparent at the first
release in numpy 1.17. Most users will never observe this weakness and are
safe to continue to use PCG64. We have introduced a new PCG64DXSM
BitGenerator that will eventually become the new default BitGenerator
implementation used by default_rng in future releases. PCG64DXSM solves
the statistical weakness while preserving the performance and the features of
PCG64.
See Upgrading PCG64 with PCG64DXSM for more details.
(gh-18906)
Expired deprecations¶
The
shapeargumentunravel_indexcannot be passed asdimskeyword argument anymore. (Was deprecated in NumPy 1.16.)(gh-17900)
The function
PyUFunc_GenericFunctionhas been disabled. It was deprecated in NumPy 1.19. Users should call the ufunc directly using the Python API.(gh-18697)
The function
PyUFunc_SetUsesArraysAsDatahas been disabled. It was deprecated in NumPy 1.19.(gh-18697)
The class
PolyBasehas been removed (deprecated in numpy 1.9.0). Please use the abstractABCPolyBaseclass instead.(gh-18963)
The unused
PolyErrorandPolyDomainErrorexceptions are removed.(gh-18963)
Deprecations¶
The .dtype attribute must return a dtype¶
A DeprecationWarning is now given if the .dtype attribute
of an object passed into np.dtype or as a dtype=obj argument
is not a dtype. NumPy will stop attempting to recursively coerce the
result of .dtype.
(gh-13578)
Inexact matches for numpy.convolve and numpy.correlate are deprecated¶
convolve and correlate now emit a warning when there are case
insensitive and/or inexact matches found for mode argument in the functions.
Pass full "same", "valid", "full" strings instead of
"s", "v", "f" for the mode argument.
(gh-17492)
np.typeDict has been formally deprecated¶
np.typeDict is a deprecated alias for np.sctypeDict and
has been so for over 14 years (6689502).
A deprecation warning will now be issued whenever getting np.typeDict.
(gh-17586)
Exceptions will be raised during array-like creation¶
When an object raised an exception during access of the special
attributes __array__ or __array_interface__, this exception
was usually ignored.
A warning is now given when the exception is anything but AttributeError.
To silence the warning, the type raising the exception has to be adapted
to raise an AttributeError.
(gh-19001)
Four ndarray.ctypes methods have been deprecated¶
Four methods of the ndarray.ctypes object have been deprecated,
as they are (undocumentated) implementation artifacts of their respective
properties.
The methods in question are:
_ctypes.get_data(use_ctypes.datainstead)_ctypes.get_shape(use_ctypes.shapeinstead)_ctypes.get_strides(use_ctypes.stridesinstead)_ctypes.get_as_parameter(use_ctypes._as_parameter_instead)
(gh-19031)
Expired deprecations¶
The
shapeargumentnumpy.unravel_indexcannot be passed asdimskeyword argument anymore. (Was deprecated in NumPy 1.16.)(gh-17900)
The function
PyUFunc_GenericFunctionhas been disabled. It was deprecated in NumPy 1.19. Users should call the ufunc directly using the Python API.(gh-18697)
The function
PyUFunc_SetUsesArraysAsDatahas been disabled. It was deprecated in NumPy 1.19.(gh-18697)
Remove deprecated PolyBase and unused PolyError and PolyDomainError¶
The class PolyBase has been removed (deprecated in numpy 1.9.0). Please use
the abstract ABCPolyBase class instead.
Furthermore, the unused PolyError and PolyDomainError exceptions are
removed from the numpy.polynomial.
(gh-18963)
Compatibility notes¶
Error type changes in universal functions¶
The universal functions may now raise different errors on invalid input in some
cases. The main changes should be that a RuntimeError was replaced with a
more fitting TypeError. When multiple errors were present in the same
call, NumPy may now raise a different one.
(gh-15271)
__array_ufunc__ argument validation¶
NumPy will now partially validate arguments before calling __array_ufunc__.
Previously, it was possible to pass on invalid arguments (such as a
non-existing keyword argument) when dispatch was known to occur.
(gh-15271)
__array_ufunc__ and additional positional arguments¶
Previously, all positionally passed arguments were checked for
__array_ufunc__ support. In the case of reduce, accumulate, and
reduceat all arguments may be passed by position. This means that when
they were passed by position, they could previously have been asked to handle
the ufunc call via __array_ufunc__. Since this depended on the way the
arguments were passed (by position or by keyword), NumPy will now only dispatch
on the input and output array. For example, NumPy will never dispatch on the
where array in a reduction such as np.add.reduce.
(gh-15271)
Validate input values in Generator.uniform¶
Checked that high - low >= 0 in np.random.Generator.uniform. Raises
ValueError if low > high. Previously out-of-order inputs were accepted
and silently swapped, so that if low > high, the value generated was
high + (low - high) * random().
(gh-17921)
/usr/include removed from default include paths¶
The default include paths when building a package with numpy.distutils no
longer include /usr/include. This path is normally added by the compiler,
and hardcoding it can be problematic. In case this causes a problem, please
open an issue. A workaround is documented in PR 18658.
(gh-18658)
Changes to comparisons with dtype=...¶
When the dtype= (or signature) arguments to comparison
ufuncs (equal, less, etc.) is used, this will denote
the desired output dtype in the future.
This means that:
np.equal(2, 3, dtype=object)
will give a FutureWarning that it will return an object
array in the future, which currently happens for:
np.equal(None, None, dtype=object)
due to the fact that np.array(None) is already an object
array. (This also happens for some other dtypes.)
Since comparisons normally only return boolean arrays, providing
any other dtype will always raise an error in the future and
give a DeprecationWarning now.
(gh-18718)
Changes to dtype and signature arguments in ufuncs¶
The universal function arguments dtype and signature
which are also valid for reduction such as np.add.reduce
(which is the implementation for np.sum) will now issue
a warning when the dtype provided is not a “basic” dtype.
NumPy almost always ignored metadata, byteorder or time units on these inputs. NumPy will now always ignore it and raise an error if byteorder or time unit changed. The following are the most important examples of changes which will give the error. In some cases previously the information stored was not ignored, in all of these an error is now raised:
# Previously ignored the byte-order (affect if non-native)
np.add(3, 5, dtype=">i32")
# The biggest impact is for timedelta or datetimes:
arr = np.arange(10, dtype="m8[s]")
# The examples always ignored the time unit "ns":
np.add(arr, arr, dtype="m8[ns]")
np.maximum.reduce(arr, dtype="m8[ns]")
# The following previously did use "ns" (as opposed to `arr.dtype`)
np.add(3, 5, dtype="m8[ns]") # Now return generic time units
np.maximum(arr, arr, dtype="m8[ns]") # Now returns "s" (from `arr`)
The same applies for functions like np.sum which use these internally.
This change is necessary to achieve consistent handling within NumPy.
If you run into these, in most cases pass for example dtype=np.timedelta64
which clearly denotes a general timedelta64 without any unit or byte-order
defined. If you need to specify the output dtype precisely, you may do so
by either casting the inputs or providing an output array using out=.
NumPy may choose to allow providing an exact output dtype here in the
future, which would be preceded by a FutureWarning.
(gh-18718)
Ufunc signature=... and dtype= generalization and casting¶
The behaviour for np.ufunc(1.0, 1.0, signature=...) or
np.ufunc(1.0, 1.0, dtype=...) can now yield different loops in 1.21
compared to 1.20 because of changes in promotion.
When signature was previously used, the casting check on inputs
was relaxed, which could lead to downcasting inputs unsafely especially
if combined with casting="unsafe".
Casting is now guaranteed to be safe. If a signature is only
partially provided, for example using signature=("float64", None, None),
this could lead to no loop being found (an error).
In that case, it is necessary to provide the complete signature
to enforce casting the inputs.
If dtype="float64" is used or only outputs are set (e.g.
signature=(None, None, "float64") the is unchanged.
We expect that very few users are affected by this change.
Further, the meaning of dtype="float64" has been slightly modified and
now strictly enforces only the correct output (and not input) DTypes.
This means it is now always equivalent to:
signature=(None, None, "float64")
(If the ufunc has two inputs and one output). Since this could lead to no loop being found in some cases, NumPy will normally also search for the loop:
signature=("float64", "float64", "float64")
if the first search failed.
In the future, this behaviour may be customized to achieve the expected
results for more complex ufuncs. (For some universal functions such as
np.ldexp inputs can have different DTypes.)
(gh-18880)
Distutils forces strict floating point model on clang¶
NumPy distutils will now always add the -ffp-exception-behavior=strict
compiler flag when compiling with clang. Clang defaults to a non-strict
version, which allows the compiler to generate code that does not set
floating point warnings/errors correctly.
(gh-19049)
C API changes¶
Use of ufunc->type_resolver and “type tuple”¶
NumPy now normalizes the “type tuple” argument to the type resolver functions
before calling it. Note that in the use of this type resolver is legacy
behaviour and NumPy will not do so when possible. Calling
ufunc->type_resolver or PyUFunc_DefaultTypeResolver is strongly
discouraged and will now enforce a normalized type tuple if done. Note that
this does not affect providing a type resolver, which is expected to keep
working in most circumstances. If you have an unexpected use-case for calling
the type resolver, please inform the NumPy developers so that a solution can be
found.
(gh-18718)
New Features¶
Added a mypy plugin for handling platform-specific numpy.number precisions¶
A mypy plugin is now available for automatically assigning the (platform-dependent)
precisions of certain number subclasses, including the likes of
int_, intp and longlong. See the documentation on
scalar types for a comprehensive overview
of the affected classes.
Note that while usage of the plugin is completely optional, without it the
precision of above-mentioned classes will be inferred as Any.
To enable the plugin, one must add it to their mypy configuration file:
[mypy]
plugins = numpy.typing.mypy_plugin
(gh-17843)
Let the mypy plugin manage extended-precision numpy.number subclasses¶
The mypy plugin, introduced in numpy/numpy#17843, has been expanded:
the plugin now removes annotations for platform-specific extended-precision
types that are not available to the platform in question.
For example, it will remove float128 when not available.
Without the plugin all extended-precision types will, as far as mypy is concerned, be available on all platforms.
To enable the plugin, one must add it to their mypy configuration file:
[mypy]
plugins = numpy.typing.mypy_plugin
(gh-18322)
New min_digits argument for printing float values¶
A new min_digits argument has been added to the dragon4 float printing
functions format_float_positional and format_float_scientific
. This kwd guarantees that at least the given number of digits will be printed
when printing in unique=True mode, even if the extra digits are unnecessary to
uniquely specify the value. It is the counterpart to the precision argument
which sets the maximum number of digits to be printed. When unique=False in
fixed precision mode, it has no effect and the precision argument fixes the
number of digits.
(gh-18629)
f2py now recognizes Fortran abstract interface blocks¶
f2py can now parse abstract interface blocks.
(gh-18695)
BLAS and LAPACK configuration via environment variables¶
Autodetection of installed BLAS and LAPACK libraries can be bypassed by using
the NPY_BLAS_LIBS and NPY_LAPACK_LIBS environment variables. Instead,
the link flags in these environment variables will be used directly, and the
language is assumed to be F77. This is especially useful in automated builds
where the BLAS and LAPACK that are installed are known exactly. A use case is
replacing the actual implementation at runtime via stub library links.
If NPY_CBLAS_LIBS is set (optional in addition to NPY_BLAS_LIBS), this
will be used as well, by defining HAVE_CBLAS and appending the environment
variable content to the link flags.
(gh-18737)
A runtime-subcriptable alias has been added for ndarray¶
numpy.typing.NDArray has been added, a runtime-subscriptable alias for
np.ndarray[Any, np.dtype[~Scalar]]. The new type alias can be used
for annotating arrays with a given dtype and unspecified shape. 1
1 NumPy does not support the annotating of array shapes as of 1.21, this is expected to change in the future though (see PEP 646).
Examples¶
>>> import numpy as np
>>> import numpy.typing as npt
>>> print(npt.NDArray)
numpy.ndarray[typing.Any, numpy.dtype[~ScalarType]]
>>> print(npt.NDArray[np.float64])
numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]
>>> NDArrayInt = npt.NDArray[np.int_]
>>> a: NDArrayInt = np.arange(10)
>>> def func(a: npt.ArrayLike) -> npt.NDArray[Any]:
... return np.array(a)
(gh-18935)
Improvements¶
Arbitrary period option for numpy.unwrap¶
The size of the interval over which phases are unwrapped is no longer restricted to 2 * pi.
This is especially useful for unwrapping degrees, but can also be used for other intervals.
>>> phase_deg = np.mod(np.linspace(0,720,19), 360) - 180
>>> phase_deg
array([-180., -140., -100., -60., -20., 20., 60., 100., 140.,
-180., -140., -100., -60., -20., 20., 60., 100., 140.,
-180.])
>>> unwrap(phase_deg, period=360)
array([-180., -140., -100., -60., -20., 20., 60., 100., 140.,
180., 220., 260., 300., 340., 380., 420., 460., 500.,
540.])
(gh-16987)
np.unique now returns single NaN¶
When np.unique operated on an array with multiple NaN entries,
its return included a NaN for each entry that was NaN in the original array.
This is now improved such that the returned array contains just one NaN as the
last element.
Also for complex arrays all NaN values are considered equivalent
(no matter whether the NaN is in the real or imaginary part). As the
representant for the returned array the smallest one in the
lexicographical order is chosen - see np.sort for how the lexicographical
order is defined for complex arrays.
(gh-18070)
Generator.rayleigh and Generator.geometric performance improved¶
The performance of Rayleigh and geometric random variate generation
in Generator has improved. These are both transformation of exponential
random variables and the slow log-based inverse cdf transformation has
been replaced with the Ziggurat-based exponential variate generator.
This change breaks the stream of variates generated when variates from either of these distributions are produced.
(gh-18666)
Performance improvements¶
Improved performance in integer division of NumPy arrays¶
Integer division of NumPy arrays now uses
libdivide when the divisor is a constant. With the
usage of libdivide and other minor optimizations, there is a large speedup.
The // operator and np.floor_divide makes use of the new changes.
(gh-17727)
Improve performance of np.save and np.load for small arrays¶
np.save is now a lot faster for small arrays.
np.load is also faster for small arrays,
but only when serializing with a version >= (3, 0).
Both are done by removing checks that are only relevant for Python 2, while still maintaining compatibility with arrays which might have been created by Python 2.
(gh-18657)
Changes¶
numpy.piecewise output class now matches the input class¶
When ndarray subclasses are used on input to piecewise,
they are passed on to the functions. The output will now be of the
same subclass as well.
(gh-18110)
Enable Accelerate Framework¶
With the release of macOS 11.3, several different issues that numpy was encountering when using Accelerate Framework’s implementation of BLAS and LAPACK should be resolved. This change enables the Accelerate Framework as an option on macOS. If additional issues are found, please file a bug report against Accelerate using the developer feedback assistant tool (https://developer.apple.com/bug-reporting/). We intend to address issues promptly and plan to continue supporting and updating our BLAS and LAPACK libraries.
(gh-18874)