CPU/SIMD optimizations

NumPy comes with a flexible working mechanism that allows it to harness the SIMD features that CPUs own, in order to provide faster and more stable performance on all popular platforms. Currently, NumPy supports the X86, IBM/Power, ARM7 and ARM8 architectures.

The optimization process in NumPy is carried out in three layers:

• Code is written using the universal intrinsics which is a set of types, macros and functions that are mapped to each supported instruction-sets by using guards that will enable use of the them only when the compiler recognizes them. This allow us to generate multiple kernels for the same functionality, in which each generated kernel represents a set of instructions that related one or multiple certain CPU features. The first kernel represents the minimum (baseline) CPU features, and the other kernels represent the additional (dispatched) CPU features.

• At compile time, CPU build options are used to define the minimum and additional features to support, based on user choice and compiler support. The appropriate intrinsics are overlaid with the platform / architecture intrinsics, and multiple kernels are compiled.

• At runtime import, the CPU is probed for the set of supported CPU features. A mechanism is used to grab the pointer to the most appropriate kernel, and this will be the one called for the function.

Note

NumPy community had a deep discussion before implementing this work, please check NEP 38 — Using SIMD optimization instructions for performance for more clarification.

• CPU build options
  • Description
  • Quick start
    • I am building NumPy for my local use
    • I do not want to support the old processors of the x86 architecture
    • I’m facing the same case above but with ppc64 architecture
    • Having issues with AVX512 features?
  • Supported features
    • On x86
    • On IBM/POWER big-endian
    • On IBM/POWER little-endian
    • On ARMv7/A32
    • On ARMv8/A64
    • On IBM/ZSYSTEM(S390X)
  • Special options
  • Behaviors
  • Platform differences
    • On x86::Intel Compiler
    • On x86::Microsoft Visual C/C++
  • Build report
  • Runtime dispatch
  • Tracking dispatched functions
• How does the CPU dispatcher work?
  • 1- Configuration
  • 2- Discovering the environment
  • 3- Validating the requested optimizations
  • 4- Generating the main configuration header
  • 5- Dispatch-able sources and configuration statements