Version history: NumPy

v1

scan: pass2026-08-11 — Ingested from source

Initial version.

v2

currentscan: pass2026-11-02 — Currency pass 2026-11-01: targets NumPy 2.x (numpy>=2.0 pin, new section on 2.x breaking changes); covariance example now uses the centered data it computes; CuPy/GPU demoted from core principle to an optional array-API note; typing.Tuple -> tuple; python 3.12; dropped 'in 2025' and team boilerplate; quoted globs so the frontmatter parses.
  ---- description: This guide provides definitive, actionable best practices for writing high-performance, maintainable, and correct NumPy code, emphasizing vectorization, explicit dtypes, and modern GPU acceleration.
- globs: **/*.py+ description: "Actionable best practices for fast, correct NumPy 2.x code: vectorization and broadcasting, explicit dtypes, views vs copies, pre-allocation, typed signatures, and numpy.testing."
+ globs: "**/*.py"
+ alwaysApply: false  ---
  # numpy Best Practices
  - NumPy is the bedrock of numerical computing in Python. To fully leverage its power for AI/ML and data science in 2025, we must write code that is not just functional but also fast, readable, and robust. This guide outlines our team's definitive best practices for NumPy.+ NumPy is the bedrock of numerical computing in Python. To fully leverage its power for AI/ML and data science, write code that is not just functional but also fast, readable, and robust. This guide outlines best practices for NumPy 2.x.  
  ## Core Principles
  
  1.  **Vectorize Everything**: Eliminate Python loops over arrays. NumPy operations are implemented in C and are orders of magnitude faster.
  2.  **Be Explicit with `dtype`**: Always specify array data types to prevent unexpected casting and optimize memory/performance.- 3.  **Embrace GPU Acceleration**: Utilize CuPy-compatible calls for critical, large-scale computations.+ 3.  **Know When You Copy**: Slices are views; use `.copy()` deliberately and pre-allocate instead of growing arrays.  
  ## Code Organization and Structure
  
  Organize NumPy-heavy logic into dedicated, well-named functions and modules.
  
  ### 1. Modular Design
  
  Isolate numerical operations into functions that accept and return NumPy arrays. This improves testability and reusability.
  
  ❌ BAD: Monolithic script
  ```python
  import numpy as np
  
  def process_data_script(data_path):
      data = np.loadtxt(data_path)
      # ... many lines of processing ...
      mean_val = np.mean(data)
      std_val = np.std(data)
      normalized_data = (data - mean_val) / std_val
      # ... more processing ...
      return normalized_data
  ```
  
  ✅ GOOD: Modular functions
  ```python
  import numpy as np
  
  def load_numerical_data(file_path: str) -> np.ndarray:
      """Loads numerical data from a file."""
      return np.loadtxt(file_path)
  
  def normalize_array(arr: np.ndarray) -> np.ndarray:
      """Normalizes a NumPy array to have zero mean and unit variance."""
      if arr.size == 0:
          return arr
      mean_val = np.mean(arr)
      std_val = np.std(arr)
      # Avoid division by zero for constant arrays
      if std_val == 0:
          return np.zeros_like(arr)
      return (arr - mean_val) / std_val
  
  def process_pipeline(data_path: str) -> np.ndarray:
      """Orchestrates data loading and processing."""
      data = load_numerical_data(data_path)
      processed_data = normalize_array(data)
      return processed_data
  ```
  
  ### 2. Clear Naming and Docstrings
  
  Adhere to PEP 8 for variable and function names (`snake_case`). Use comprehensive docstrings (PEP 257) for all public functions, classes, and modules, especially for complex numerical logic.
  
  ❌ BAD: Ambiguous names, no docs
  ```python
  def calc(a, b):
      # What does 'a' mean? What does 'b' mean?
      # What does this function do?
      return np.dot(a, b.T)
  ```
  
  ✅ GOOD: Descriptive names, clear docstrings
  ```python
  import numpy as np
  
  def calculate_covariance_matrix(data_matrix: np.ndarray) -> np.ndarray:
      """
      Calculates the covariance matrix for a given data matrix.
  
      Args:
          data_matrix: A 2D NumPy array where rows are observations
                       and columns are features.
  
      Returns:
          A 2D NumPy array representing the covariance matrix.
      """
      if data_matrix.ndim != 2:
          raise ValueError("Input data_matrix must be 2-dimensional.")
      
      # Center the data by subtracting the mean of each feature
      centered_data = data_matrix - np.mean(data_matrix, axis=0)-     
-     # Calculate covariance matrix: (X^T * X) / (n - 1)
-     covariance_matrix = np.cov(data_matrix, rowvar=False)+ 
+     # Calculate covariance matrix: (X^T * X) / (n - 1); equivalent to np.cov(data_matrix, rowvar=False)
+     n_observations = data_matrix.shape[0]
+     covariance_matrix = centered_data.T @ centered_data / (n_observations - 1)      return covariance_matrix
  ```
  
  ## Common Patterns and Anti-patterns
  
  ### 1. Vectorization over Python Loops
  
  **Always** prefer NumPy's vectorized operations and broadcasting over explicit Python `for` loops when working with arrays.
  
  ❌ BAD: Element-wise loop
  ```python
  import numpy as np
  
  data = np.random.rand(1_000_000)
  result = np.empty_like(data)
  for i in range(len(data)):
      result[i] = data[i] * 2 + 5 # Slow for large arrays
  ```
  
  ✅ GOOD: Vectorized operation
  ```python
  import numpy as np
  
  data = np.random.rand(1_000_000)
  result = data * 2 + 5 # Fast and concise
  ```
  
  ### 2. Broadcasting
  
  Leverage broadcasting for operations between arrays of different shapes. It's efficient and avoids unnecessary memory allocation.
  
  ❌ BAD: Manual tiling or explicit loops for scalar/vector operations
  ```python
  import numpy as np
  
  matrix = np.array([[1, 2, 3], [4, 5, 6]])
  scalar = 10
  result = np.empty_like(matrix)
  for i in range(matrix.shape[0]):
      for j in range(matrix.shape[1]):
          result[i, j] = matrix[i, j] + scalar # Avoid
  ```
  
  ✅ GOOD: Broadcasting
  ```python
  import numpy as np
  
  matrix = np.array([[1, 2, 3], [4, 5, 6]])
  scalar = 10
  result = matrix + scalar # NumPy handles this efficiently
  ```
  
  ## Performance Considerations
  
  ### 1. Explicit `dtype` Specification
  
  **Always** specify `dtype` when creating arrays or performing operations that might change types. This prevents unexpected memory usage and performance penalties from implicit type conversions.
  
  ❌ BAD: Implicit `dtype`
  ```python
  import numpy as np
  
  # Python integers become int64 by default, even if int32 is sufficient
  arr = np.array([1, 2, 3]) 
  # float operations might implicitly upcast, consuming more memory
  result = arr / 2.0 
  ```
  
  ✅ GOOD: Explicit `dtype`
  ```python
  import numpy as np
  
  # Use the smallest appropriate dtype
  arr_int = np.array([1, 2, 3], dtype=np.int32)
  arr_float = np.array([1.0, 2.0, 3.0], dtype=np.float32)
  
  # Ensure consistent dtypes in operations
  result = arr_int.astype(np.float32) / 2.0
  ```
  - ### 2. Leverage GPU Acceleration (CuPy-compatible)+ ### 2. Optional: GPU Acceleration via the Array API  - For computationally intensive tasks on large arrays, especially in AI/ML, integrate GPU-accelerated libraries like CuPy. Write functions that can seamlessly switch between NumPy and CuPy.+ This is an opt-in for genuinely large workloads, not a default. NumPy 2.x and CuPy both implement the Python array API standard, so a function can be written once against an array namespace (`xp`) and run on either library. Only add this when profiling shows the CPU path is the bottleneck.  
  ```python
  import numpy as np
  try:
      import cupy as cp
      _has_cupy = True
  except ImportError:
      _has_cupy = False
  
  def matrix_multiply(a: np.ndarray, b: np.ndarray, use_gpu: bool = False) -> np.ndarray:
      """
      Performs matrix multiplication, optionally using GPU if available.
      """
      xp = cp if use_gpu and _has_cupy else np
      
      a_xp = xp.asarray(a)
      b_xp = xp.asarray(b)
      
      result = xp.dot(a_xp, b_xp)
      
      return result.get() if use_gpu and _has_cupy else result
  
  # Example usage
  matrix_a = np.random.rand(1000, 500)
  matrix_b = np.random.rand(500, 2000)
  
  # CPU computation
  cpu_result = matrix_multiply(matrix_a, matrix_b, use_gpu=False)
  
  # GPU computation (if CuPy installed)
  if _has_cupy:
      gpu_result = matrix_multiply(matrix_a, matrix_b, use_gpu=True)
      assert np.allclose(cpu_result, gpu_result)
  ```
  
  ### 3. Pre-allocate Arrays
  
  When filling an array iteratively (e.g., in a loop where vectorization isn't fully possible), pre-allocate the array with `np.empty` or `np.zeros` to avoid costly reallocations.
  
  ❌ BAD: Appending to a list then converting
  ```python
  import numpy as np
  
  results = []
  for i in range(1000):
      results.append(np.random.rand(10))
  final_array = np.array(results) # Inefficient for many appends
  ```
  
  ✅ GOOD: Pre-allocation
  ```python
  import numpy as np
  
  num_iterations = 1000
  array_shape = (10,)
  final_array = np.empty((num_iterations,) + array_shape, dtype=np.float32)
  for i in range(num_iterations):
      final_array[i] = np.random.rand(*array_shape) # Efficient
  ```
  
  ## Common Pitfalls and Gotchas
  
  ### 1. Shape Mismatches
  
  NumPy operations are strict about array shapes. Understand broadcasting rules and use `reshape`, `transpose`, `newaxis`, or `squeeze` explicitly.
  
  ❌ BAD: Implicit shape assumptions
  ```python
  import numpy as np
  
  vec = np.array([1, 2, 3]) # (3,)
  matrix = np.array([[10, 20, 30], [40, 50, 60]]) # (2, 3)
  # This will raise a ValueError if not careful, e.g., matrix + vec.T
  # Or produce unexpected results due to broadcasting rules
  ```
  
  ✅ GOOD: Explicit shape handling
  ```python
  import numpy as np
  
  vec_row = np.array([1, 2, 3]) # (3,)
  vec_col = np.array([1, 2, 3])[:, np.newaxis] # (3, 1)
  matrix = np.array([[10, 20, 30], [40, 50, 60]]) # (2, 3)
  
  # Correctly add a row vector to each row of a matrix
  result = matrix + vec_row # (2,3) + (3,) -> (2,3)
  
  # Correctly add a column vector to each column of a matrix
  result = matrix + vec_col # (2,3) + (3,1) -> (2,3)
  ```
  
  ### 2. View vs. Copy
  
  Understand when NumPy returns a view (modifying it changes the original array) versus a copy. Operations like slicing often return views. Use `.copy()` explicitly when you need an independent array.
  
  ❌ BAD: Unintended modification
  ```python
  import numpy as np
  
  original_data = np.arange(10)
  subset = original_data[2:5] # This is a view!
  subset[:] = 99 # Modifies original_data
  print(original_data) # Output: [ 0  1 99 99 99  5  6  7  8  9] - DANGER!
  ```
  
  ✅ GOOD: Explicitly create a copy when needed
  ```python
  import numpy as np
  
  original_data = np.arange(10)
  subset_copy = original_data[2:5].copy() # This is a copy!
  subset_copy[:] = 99 # Only modifies subset_copy
  print(original_data) # Output: [0 1 2 3 4 5 6 7 8 9] - SAFE!
  ```
  + ### 3. NumPy 2.x Changes
+ 
+ Code written for NumPy 1.x may break on 2.x. Watch for:
+ 
+ *   Removed aliases: `np.float_`, `np.complex_`, `np.product`, `np.round_`, `np.alltrue` (use `np.float64`, `np.complex128`, `np.prod`, `np.round`, `np.all`).
+ *   `np.array(x, copy=False)` now raises if a copy is needed; use `np.asarray(x)` for "copy only if necessary".
+ *   Scalar reprs changed (`np.float64(3.0)` instead of `3.0`); never parse `repr` output.
+ *   Prefer the `np.random.default_rng()` Generator over the legacy global `np.random.rand`/`seed` functions in new code.
+   ## Type Hints
  
  **Always** use type hints for function signatures involving NumPy arrays. This improves code readability, enables static analysis, and catches errors early. Use `np.ndarray` and specify `dtype` where type precision is critical.
  
  ```python- from typing import Tuple  import numpy as np
  
  def process_sensor_data(-     readings: np.ndarray, +     readings: np.ndarray,      threshold: float = 0.5- ) -> Tuple[np.ndarray, np.ndarray]:+ ) -> tuple[np.ndarray, np.ndarray]:      """
      Processes sensor readings, separating values above/below a threshold.
  
      Args:
          readings: A 1D NumPy array of sensor readings (expected float32).
          threshold: The threshold value.
  
      Returns:
          A tuple containing two 1D NumPy arrays: (above_threshold, below_threshold).
      """
      if readings.dtype != np.float32:
          raise TypeError("Input 'readings' must be of dtype np.float32")
  
      above = readings[readings >= threshold]
      below = readings[readings < threshold]
      return above, below
  
  # Example usage
  data = np.random.rand(100).astype(np.float32)
  high_values, low_values = process_sensor_data(data, threshold=0.7)
  ```
  
  ## Virtual Environments
  
  **Always** use virtual environments (`venv` or `conda`) for every project. This isolates dependencies and prevents version conflicts.
  
  ❌ BAD: Installing packages globally
  ```bash
  pip install numpy scipy pandas # Pollutes global Python environment
  ```
  
  ✅ GOOD: Using a virtual environment
  ```bash
  # For venv
  python -m venv .venv
  source .venv/bin/activate
  pip install numpy scipy pandas
  
  # For conda- conda create -n my_project_env python=3.10+ conda create -n my_project_env python=3.12  conda activate my_project_env
  conda install numpy scipy pandas
  ```
  
  ## Packaging
  
  For any reusable NumPy code, package it as a standard Python library. Use `pyproject.toml` for modern packaging.
  
  ```toml
  # pyproject.toml
  [project]
  name = "my_numpy_utils"
  version = "0.1.0"
  dependencies = [-     "numpy>=1.26.0",
-     "scipy>=1.12.0",+     "numpy>=2.0",
+     "scipy>=1.13",  ]
  
  [build-system]
  requires = ["setuptools>=61.0"]
  build-backend = "setuptools.build_meta"
  ```
  
  ## Testing Approaches
  
  **Always** write unit tests for your NumPy code. Numerical stability and correctness are paramount.
  
  ### 1. `numpy.testing` for Numerical Comparisons
  
  Use `numpy.testing` functions (e.g., `assert_allclose`, `assert_array_equal`) for comparing arrays, as standard equality checks can fail due to floating-point inaccuracies.
  
  ❌ BAD: Direct equality for floats
  ```python
  import numpy as np
  
  a = np.array([0.1 + 0.2])
  b = np.array([0.3])
  assert a == b # Fails due to floating point precision
  ```
  
  ✅ GOOD: `numpy.testing.assert_allclose`
  ```python
  import numpy as np
  from numpy.testing import assert_allclose
  
  a = np.array([0.1 + 0.2])
  b = np.array([0.3])
  assert_allclose(a, b, rtol=1e-5, atol=1e-8) # Passes
  ```
  
  ### 2. `pytest` Integration
  
  Integrate `numpy.testing` with `pytest` for a robust testing framework.
  
  ```python
  # tests/test_my_module.py
  import numpy as np
  from numpy.testing import assert_allclose, assert_array_equal
  from my_package.my_module import normalize_array, calculate_covariance_matrix
  
  def test_normalize_array_basic():
      arr = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float32)
      expected = np.array([-1.34164079, -0.4472136 ,  0.4472136 ,  1.34164079], dtype=np.float32)
      result = normalize_array(arr)
      assert_allclose(result, expected, rtol=1e-6)
  
  def test_normalize_array_constant():
      arr = np.array([5.0, 5.0, 5.0], dtype=np.float32)
      expected = np.array([0.0, 0.0, 0.0], dtype=np.float32)
      result = normalize_array(arr)
      assert_allclose(result, expected)
  
  def test_calculate_covariance_matrix():
      data = np.array([[1, 2], [2, 3], [3, 4]], dtype=np.float64)
      expected_cov = np.array([[1.0, 1.0], [1.0, 1.0]], dtype=np.float64)
      result_cov = calculate_covariance_matrix(data)
      assert_allclose(result_cov, expected_cov, rtol=1e-9)
  
  def test_calculate_covariance_matrix_raises_error_for_1d():
      data = np.array([1, 2, 3], dtype=np.float64)
      with np.testing.assert_raises(ValueError):
          calculate_covariance_matrix(data)- ```+ ```