In this article by Armando Fandango author of the book Python Data Analysis - Second Edition, discuss how the NumPy provides a multidimensional array object called ndarray. NumPy arrays are typed arrays of fixed size. Python lists are heterogeneous and thus elements of a list may contain any object type, while NumPy arrays are homogenous and can contain object of only one type. An ndarray consists of two parts, which are as follows:
The actual data that is stored in a contiguous block of memory
The metadata describing the actual data
Since the actual data is stored in a contiguous block of memory hence loading of the large data set as ndarray is affected by availability of large enough contiguous block of memory. Most of the array methods and functions in NumPy leave the actual data unaffected and only modify the metadata.
Actually, we made a one-dimensional array that held a set of numbers. The ndarray can have more than a single dimension.
(For more resources related to this topic, see here.)
Advantages of NumPy arrays
The NumPy array is, in general, homogeneous (there is a particular record array type that is heterogeneous)—the items in the array have to be of the same type. The advantage is that if we know that the items in an array are of the same type, it is easy to ascertain the storage size needed for the array. NumPy arrays can execute vectorized operations, processing a complete array, in contrast to Python lists, where you usually have to loop through the list and execute the operation on each element. NumPy arrays are indexed from 0, just like lists in Python. NumPy utilizes an optimized C API to make the array operations particularly quick.
We will make an array with the arange() subroutine again. You will see snippets from Jupyter Notebook sessions where NumPy is already imported with instruction import numpy as np. Here's how to get the data type of an array:
In: a = np.arange(5)
In: a.dtype
Out: dtype('int64')
The data type of the array a is int64 (at least on my computer), but you may get int32 as the output if you are using 32-bit Python. In both the cases, we are dealing with integers (64 bit or 32 bit). Besides the data type of an array, it is crucial to know its shape. A vector is commonly used in mathematics but most of the time we need higher-dimensional objects. Let's find out the shape of the vector we produced a few minutes ago:
As you can see, the vector has five components with values ranging from 0 to 4.
The shape property of the array is a tuple; in this instance, a tuple of 1 element, which holds the length in each dimension.
Creating a multidimensional array
Now that we know how to create a vector, we are set to create a multidimensional NumPy array. After we produce the matrix, we will again need to show its, as demonstrated in the following code snippets:
Create a multidimensional array as follows:
In: m = np.array([np.arange(2), np.arange(2)])
In: m
Out:
array([[0, 1],
[0, 1]])
We can show the array shape as follows:
In: m.shape
Out: (2, 2)
We made a 2 x 2 array with the arange() subroutine. The array() function creates
an array from an object that you pass to it. The object has to be an array, for example,
a Python list. In the previous example, we passed a list of arrays. The object is the only required parameter of the array() function. NumPy functions tend to have
a heap of optional arguments with predefined default options.
Selecting NumPy array elements
From time to time, we will wish to select a specific constituent of an array. We will take a look at how to do this, but to kick off, let's make a 2 x 2 matrix again:
In: a = np.array([[1,2],[3,4]])
In: a
Out:
array([[1, 2],
[3, 4]])
The matrix was made this time by giving the array() function a list of lists. We will now choose each item of the matrix one at a time, as shown in the following code snippet. Recall that the index numbers begin from 0:
As you can see, choosing elements of an array is fairly simple. For the array a, we just employ the notation a[m,n], where m and n are the indices of the item in the array. Have a look at the following figure for your reference:
NumPy numerical types
Python has an integer type, a float type, and complex type; nonetheless, this is not sufficient for scientific calculations. In practice, we still demand more data types with varying precisions and, consequently, different storage sizes of the type. For this reason, NumPy has many more data types. The bulk of the NumPy mathematical types ends with a number. This number designates the count of bits related to the type. The following table (adapted from the NumPy user guide) presents an overview of NumPy numerical types:
It is important to be aware that you are not allowed to change a complex number into an integer. Attempting to do that sparks off a TypeError:
In: np.int(42.0 + 1.j)
Traceback (most recent call last):
<ipython-input-24-5c1cd108488d> in <module>()
----> 1 np.int(42.0 + 1.j)
TypeError: can't convert complex to int
The same goes for conversion of a complex number into a floating-point number.
By the way, the j component is the imaginary coefficient of a complex number.
Even so, you can convert a floating-point number to a complex number, for example, complex(1.0). The real and imaginary pieces of a complex number can be pulled out with the real() and imag() functions, respectively.
Data type objects
Data type objects are instances of the numpy.dtype class. Once again, arrays have
a data type. To be exact, each element in a NumPy array has the same data type.
The data type object can tell you the size of the data in bytes. The size in bytes is given by the itemsize property of the dtype class :
In: a.dtype.itemsize
Out: 8
Character codes
Character codes are included for backward compatibility with Numeric. Numeric is the predecessor of NumPy. Its use is not recommended, but the code is supplied here because it pops up in various locations. You should use the dtype object instead. The following table lists several different data types and character codes related to them:
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
Type
Character code
integer
i
Unsigned integer
u
Single precision float
f
Double precision float
d
bool
b
complex
D
string
S
unicode
U
Void
V
Take a look at the following code to produce an array of single precision floats:
We have a variety of means to create data types. Take the case of floating-point data (have a look at dtypeconstructors.py in this book's code bundle):
We can use the general Python float, as shown in the following lines of code:
In: np.dtype(float)
Out: dtype('float64')
We can specify a single precision float with a character code:
In: np.dtype('f')
Out: dtype('float32')
We can use a double precision float with a character code:
In: np.dtype('d')
Out: dtype('float64')
We can pass the dtype constructor a two-character code. The first character stands for the type; the second character is a number specifying the number of bytes in the type (the numbers 2, 4, and 8 correspond to floats of 16, 32, and 64 bits, respectively):
In: np.dtype('f8')
Out: dtype('float64')
A (truncated) list of all the full data type codes can be found by applying sctypeDict.keys():
The dtype class has a number of useful properties. For instance, we can get information about the character code of a data type through the properties of dtype:
In: t = np.dtype('Float64')
In: t.char
Out: 'd'
The type attribute corresponds to the type of object of the array elements:
In: t.type
Out: numpy.float64
The str attribute of dtype gives a string representation of a data type. It begins with a character representing endianness, if appropriate, then a character code, succeeded by a number corresponding to the number of bytes that each array item needs. Endianness, here, entails the way bytes are ordered inside a 32- or 64-bit word. In the big-endian order, the most significant byte is stored first, indicated by >. In the little-endian order, the least significant byte is stored first, indicated by <, as exemplified in the following lines of code:
In: t.str
Out: '<f8'
One-dimensional slicing and indexing
Slicing of one-dimensional NumPy arrays works just like the slicing of standard Python lists. Let's define an array containing the numbers 0, 1, 2, and so on up to and including 8. We can select a part of the array from indexes 3 to 7, which extracts the elements of the arrays 3 through 6:
We have already learned about the reshape() function. Another repeating chore is the flattening of arrays. Flattening in this setting entails transforming a multidimensional array into a one-dimensional array. Let us create an array b that we shall use for practicing the further examples:
Flatten: The appropriately named function, flatten(), does the same
as ravel(). However, flatten() always allocates new memory, whereas ravel gives back a view of the array. This means that we can directly manipulate the array as follows:
As you can understand, the preceding code alters the array immediately. Now, we have a 6 x 4 array.
Transpose: In linear algebra, it is common to transpose matrices. Transposing is a way to transform data. For a two-dimensional table, transposing means that rows become columns and columns become rows. We can do this too
by using the following code:
Arrays can be stacked horizontally, depth wise, or vertically. We can use, for this goal, the vstack(), dstack(), hstack(), column_stack(), row_stack(), and concatenate() functions. To start with, let's set up some arrays:
In: a = np.arange(9).reshape(3,3)
In: a
Out:
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
In: b = 2 * a
In: b
Out:
array([[ 0, 2, 4],
[ 6, 8, 10],
[12, 14, 16]])
As mentioned previously, we can stack arrays using the following techniques:
Horizontal stacking: Beginning with horizontal stacking, we will shape a tuple of ndarrays and hand it to the hstack() function to stack the arrays.
This is shown as follows:
The following diagram depicts horizontal stacking:
Vertical stacking: With vertical stacking, a tuple is formed again. This time it is given to the vstack() function to stack the arrays. This can be seen as follows:
The concatenate() function gives the same outcome with the axis parameter fixed to 0. This is the default value for the axis parameter,
as portrayed in the following code:
Refer to the following figure for vertical stacking:
Depth stacking: To boot, there is the depth-wise stacking employing dstack() and a tuple, of course. This entails stacking a list of arrays
along the third axis (depth). For example, we could stack 2D arrays
of image data on top of each other as follows:
Yes, you guessed it right! We compared two arrays with the == operator.
Row stacking: NumPy, naturally, also has a function that does row-wise stacking. It is named row_stack() and for 1D arrays, it just stacks the arrays in rows into a 2D array:
Arrays can be split vertically, horizontally, or depth wise. The functions involved
are hsplit(), vsplit(), dsplit(), and split(). We can split arrays either into arrays of the same shape or indicate the location after which the split should happen. Let's look at each of the functions in detail:
Horizontal splitting: The following code splits a 3 x 3 array on its horizontal axis into three parts of the same size and shape (see splitting.py in this book's code bundle):
Let's learn more about the NumPy array attributes with the help of an example. Let us create an array b that we shall use for practicing the further examples:
The flat property gives back a numpy.flatiter object. This is the only means to get a flatiter object; we do not have access to a flatiter constructor. The flat iterator enables us to loop through an array as if it were a flat array, as shown in the following code snippet:
In: b = np.arange(4).reshape(2,2)
In: b
Out:
array([[0, 1],
[2, 3]])
In: f = b.flat
In: f
Out: <numpy.flatiter object at 0x103013e00>
In: for item in f: print(item)
Out:
0
1
2
3
It is possible to straightaway obtain an element with the flatiter object:
In: b.flat[2]
Out: 2
Also, you can obtain multiple elements as follows:
In: b.flat[[1,3]]
Out: array([1, 3])
The flat property can be set. Setting the value of the flat property leads to overwriting the values of the entire array:
The astype() function transforms the array to an array of the specified data type:
In: b
Out: array([ 1.+1.j, 3.+2.j])
In: b.astype(int)
/usr/local/lib/python3.5/site-packages/ipykernel/__main__.py:1: ComplexWarning: Casting complex values to real discards the imaginary part
…
Out: array([1, 3])
In: b.astype('complex')
Out: array([ 1.+1.j, 3.+2.j])
We are dropping off the imaginary part when casting from the complex type to int. The astype() function takes the name of a data type as a string too.
The preceding code won't display a warning this time because we used the right data type.
Summary
In this article, we found out a heap about the NumPy basics: data types and arrays. Arrays have various properties that describe them. You learned that one of these properties is the data type, which, in NumPy, is represented by a full-fledged object.
NumPy arrays can be sliced and indexed in an effective way, compared to standard Python lists. NumPy arrays have the extra ability to work with multiple dimensions.
The shape of an array can be modified in multiple ways, such as stacking, resizing, reshaping, and splitting.
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
Singapore
Canary Islands
Hungary
Ukraine
Luxembourg
Estonia
Lithuania
South Korea
Turkey
Switzerland
Colombia
Taiwan
Chile
Norway
Ecuador
Indonesia
New Zealand
Cyprus
Denmark
Finland
Poland
Malta
Czechia
Austria
Sweden
Italy
Egypt
Belgium
Portugal
Slovenia
Ireland
Romania
Greece
Argentina
Netherlands
Bulgaria
Latvia
South Africa
Malaysia
Japan
Slovakia
Philippines
Mexico
Thailand
