Advanced NumPy: Broadcasting and Vectorizing

We have earlier been introduced to Numpy, a Python library for numerical computing. In today's self study day, we are going to dive into two advanced Numpy concepts, namely broadcasting and vectorizing. Note that both concepts will be integral to your creative brief.

Today's learning objectives

  • Understand NumPy broadcasting.
  • Understand NumPy vectorising.

Broadcasting

The term broadcasting describes how NumPy treats arrays with different shapes during arithmetic operations. For example, in order to add two arrays, the two arrays would have to have the same shape. In general, the smaller array is "broadcast" across the larger array so that they have compatible shapes.

Observe what happens when you run the following piece of coded

import numpy as np

a = np.array([1.0, 2.0, 3.0])
b = np.array([2.0, 2.0, 2.0])
c = a * b
print(c)

Now, to see broadcasting in action, run the following code snippet:

import numpy as np

a = np.array([1.0, 2.0, 3.0])
b = np.array([2.0])
c = a * b
print(c)

To paraphrase the official Numpy documentation - ‘the result is equivalent to the previous example where b was an array. We can think of the scalar b being stretched during the arithmetic operation into an array with the same shape as a. The new elements in b, as shown in Figure 1, are simply copies of the original scalar. The stretching analogy is only conceptual. NumPy is smart enough to use the original scalar value without actually making copies so that broadcasting operations are as memory and computationally efficient as possible.'

Elements of a flowchart


Please watch the following video to understanding Broadcasting as explained by one of the founding fathers of modern AI, Dr. Andrew Ng.

Vectorising

Loops in Python are often the cause behind code that takes forever to run. This is because when you execute a loop in Python, it goes line-by-line through the code at each iteration, and performs several checks every single time for example like determining the type of variable (remember that Python is dynamically typed), resolving it's scope, checking for any exceptions etc.

So loops can indeed slow down your code. How about if we can restrict our lists to have only one data type, avoding the check that needs to take place every iteration? That should definitely speed up the process! Vectorising in NumPy does something similar. NumPy allows arrays to only have a single data type and ensures that your code executes much faster compared to traditional python loops.

Assignment

  • Write a Python function that multiplies two lists of unequal sizes using loops.
  • Rewrite the function using NumPy using broadcasting and vectorising.
  • Time the two functions and comment on the improvements you observe in speed.

Use Python's time module to time (or profile) your code.