NumPy / SciPy Recipes for Image Processing

NumPy / SciPy Recipes for Image Processing: Creating Fractal Images Christian Bauckhage B-IT, University of Bonn, Germany Fraunhofer IAIS, Sankt Augustin, Germany

Abstract—In this note, we show how to use of NumPy meshgrids and boolean arrays for efficient image processing. As an application example, we compute fractal images that visualize Julia- or Mandelbrot sets.

I. I NTRODUCTION Digital image processing is all about manipulating 2D pixel arrays and usually requires code to loop over the rows and columns of an image1 . Since nested loops in Python are rather inefficient, the prejudice persists that it is too slow for pixel by pixel operations. But this ain’t so! NumPy and SciPy specifically support large multi-dimensional arrays and provide numerous functions for common image processing tasks. However, many of these functions still seem relatively under appreciated and their proper use requires thinking in terms of vectorized operations. In this note, we therefore explore how to use mesh-grids and boolean masks to vectorize elaborate computations on 2D arrays. To have a fun practical example, we consider the problem of computing fractal images like this:

Our discussion assumes that readers are passably familiar with NumPy [1] and Matplotlib [2]. However, since we do not expect our readers to be experts on fractals, we will discuss some theory first. II. T HEORY To better appreciate the problem of computing images such as the one in Fig. 1, let us briefly review basics of complex numbers, Julia sets, chaos, and fractals. Readers familiar with these concepts might want to skip this section. Recall that a complex number z ∈ C can be expressed as z = x + iy where x, y ∈ R and the imaginary unit i is defined by the relation i2 = −1. The modulus of a complex number is given by p (1) |z| = x2 + y 2 and is therefore real valued. The square of a complex number, on the other hand, is yet another complex number, because z 2 = (x + iy) · (x + iy) = x2 + y 2 + i2xy.

(2)

Often, complex numbers are thought of as points in the complex plane and we can thus picture them, their moduli, and their squares as shown in Fig. 2. Now, consider a rational function f : C → C such as, for example, the simple polynomial f (z) = z 2 + c

(3)

where c ∈ C is a constant. Roughly speaking, the Julia set of such a function is a characterization of its behavior under iteration. To be more precise, let us consider any point z0 , compute the t-th iterate zt = f t (z0 ), and ask for what will happen to the modulus |zt | if t becomes large. Two major cases can occur: either |zt | will grow towards infinity or it will remain bounded. In other words, every rational function f divides the complex plane into two disjoint sets. Points for which f t diverges form the escape set of f and points for which f t does not diverge are in the prisoner set of f . The Julia set of f is then defined to be the boundary of the prisoner set, or, formally n o J(f ) = ∂ z ∈ C lim f t (z) < ∞ . (4) t→∞

Fig. 1: Visualization of a Julia set. 1 Readers new to image processing and its basic concepts may want to check out our introductory lecture series available on YouTube: https://www. youtube.com/playlist?list=PL8NTI-xZ0OWnuPNA8QD1BmFyBYbeKIaLM

Note that infinite iterations of a rational function can also be expressed in terms of recursions. For example, for the function f (z) in (3), we could consider the dynamic system zt+1 = zt2 + c.

(5)

z2

i

i

y

i

z

1

z

x

(a) z ∈ C, z = x + iy

z

|z| 1

1

(b) modulus of z

(c) square of z

Fig. 2: Geometric interpretation of complex numbers.

i

1 r=2

Fig. 3: . Chaotic behavior of the dynamic system zt+1 = zt2 +c where c = −0.065 + i 0.66; whereas the blue curve shows the trajectory of the system for z0 = 0.49 + i 0.30, the red curve corresponds to the trajectory for z0 = 0.50 + i 0.29

Recursions like this provide an indirect approach towards trying to determine the Julia set of a function. Given an initial point z0 , we can iterate (5) and, in each iteration, test if |zt | exceeds the threshold radius r(c) = max(|c|, 2), because once a trajectory z0 → z1 → z2 → . . . leaves the disk of radius r(c) it is certain that it will escape toward infinity and therefore that z0 is in the escape set [3]. A striking observation is that dynamic systems like the one in (5) behave chaotic for almost all choices of c. This is to say that small differences in the initial conditions typically lead to widely diverging outcomes in the long term. In other words, if such a system is started from two nearby points z0 and z0 + , both runs will likely yield different trajectories. This is illustrated in Fig. 3 which shows two trajectories we obtained from (5) with c = −0.065 + i 0.66. The blue trajectory starts at z0 = 0.49 + i 0.30 and quickly leaves the disc of radius r = 2; the red trajectory starts at the very close point z0 = 0.50 + i 0.29 but behaves considerably different. It is important to note that this kind of chaos is not induced by the numerical imprecision of digital computers but rather is a fundamental property of complex dynamic systems. Therefore, Julia sets can hardly ever be determined exactly; depending on the choice of f it is nigh impossible to predict if a point z0 is contained in J(f ) or not.

Fig. 4: Julia sets are fractals that are self-similar across scales, i.e. they consist of structures that reappear when magnified.

Nevertheless, there is a form of regularity because Julia sets generally have a fractal structure. Without being too technical, this is to say that they display self-similarity on all scales. Loosely speaking, a natural or mathematical object is called self-similar if it looks (more or less) like its constituent parts. In other words, in order for an object to be self-similar, it need not necessarily have the exact same structure across scales, but it will exhibit the same type of structure in all of its parts. This loose kind of self-similarity of Julia sets becomes apparent from looking at the image in Fig. 1. Each pixel in this image corresponds to a point in the complex plane for which we iterated the dynamic system in (5) with c = −0.065 + i 0.66 and the color of each pixel indicates how many iterations it took for the corresponding point to escape beyond r(c)2 . Apparently, the object consists of a beautiful galaxy of spirals which reoccur in various sizes. In fact, the examples in Fig. 4 demonstrate that if we zoom into this Julia set, i.e. visualize its structure on smaller and smaller scales, these spirals continue to appear again and again. 2 Strictly

speaking the image thus shows an escape- rather than a Julia set.

Listing 1: examples of complex valued functions def f1(z, c=-.065+.66j): return z**2 + c

c

zmax i

zmax i

r

def f2(z, c=-.06+.67j): return z**2 + c

1 zmin

1 zmin

Listing 2: na¨ıve Julia set (escape time) computation def JuliaNaive(f, zmin, zmax, m, n, tmax=256): xs = np.linspace(zmin.real, zmax.real, n) ys = np.linspace(zmin.imag, zmax.imag, m)

(a) rectangular area A ⊂ C

(b) m × n sample points in A

Fig. 5: Continuous and discrete subsets of the complex plane.

J = np.ones((m, n)) * tmax for r, y in enumerate(ys): for c, x in enumerate(xs): z = x + 1j * y for t in xrange(tmax): z = f(z) if np.abs(z) > 2: J[r,c] = t break return J

number of iterations to be performed, our implementation in JuliaNaive proceeds as follows: •

III. P RACTICE Having familiarized ourselves with the theory behind (a certain kind of) fractals, we now look at how to compute (corresponding) fractal images using NumPy. To be specific, we address the following problem: • given a rational complex valued function f (z) such as in (3), compute an m × n images, i.e. an image of m rows and n columns, that illustrates the behavior of |f t (z)| over a rectangular area of the complex plane which is defined by its lower left corner zmin = xmin + iymin and its upper right corner zmax = xmax + iymax . In order to see how to do this efficiently, we will first present a na¨ıve solution and then show how to improve on it. Also, in order for each of our code snippets to work, we require that NumPy and Matplotlib modules are imported as follows: # import numpy import numpy as np # import matplotlib import matplotlib.pyplot as plt # import matplotlib color manager import matplotlib.cm as cm

Listing 1 presents simple Python implementations of two complex valued functions where function f1 corresponds to the function we have been considering in our examples so far. In what follows, we continue to focus on this function, but encourage the reader to experiment with others as well. A. Na¨ıve Solution Listing 2 provides a straightforward implementation of the escape time algorithm which we discussed in the previous section. Given the function f to be iterated, the rectangular area A = [zmin , zmax ] ⊂ C to be considered, the extension m × n of the digital image to be computed, and the maximum

• •

•

•

•

first, we need to address the fact that any rectangular subset of the complex plane contains a continuum of points but a digital image or 2D pixel array only consists of a discrete finite set of pixels; we thus have to sample the area under consideration and identify the resulting sample point with pixels in the image; here, we approach this problem using linspace to create two 1D arrays containing n and m regularly spaced numbers which indicate x- and y-coordinates of complex numbers in the area under consideration (see Fig. 5) next, we create an m × n array or image J and initialize all its values to tmax we then iterate over all rows of J and we note that our use of enumerate(ys) provides a convenient way of identifying each row index r with an imaginary coordinate y ∈ C for each row, we iterate over all columns of J and use enumerate(xs) to identify column indices c with real coordinates x ∈ C one of the many conveniences of Python is that it readily provides a data type for complex numbers; given x and y, we therefore compute z = x + iy and begin iterating the function f passed as an argument using abs, we test if the current iterate exceeds a radius of 2; should this be the case, we set the pixel in row r and columns c of J to the the current value t of the iteration counter and terminate the innermost loop

Upon termination of this procedure, the content of array J can thus be thought of as a characterization of the Julia set of f . That is to say that its element Jrc in row r and column c will contain an integer t which indicates how quickly (if at all) the iterates of the corresponding complex number z escaped beyond the threshold radius. Having discussed this implementation, we will not even bother measuring its run time. Despite of our use of NumPy arrays, the code snippet is more or less vanilla Python and contains three nested for loops which most programmers will recognize as a recipe for run time disasters.

Fig. 6: Different visualization of a Julia set resulting from various Matplotlib color maps.

Listing 3: efficient Julia set (escape time) computation def Julia(f, zmin, zmax, m, n, tmax=256): xs = np.linspace(zmin.real, zmax.real, n) ys = np.linspace(zmin.imag, zmax.imag, m) X, Y = np.meshgrid(xs, ys) Z = X + 1j * Y J = np.ones(Z.shape) * tmax for t in xrange(tmax): mask = np.abs(Z)