Tensors

Introduction

Tensors are containers of data.
- They usually store numbers.
- But strings are also common.
A tensor can be multi-dimensional:
- The number of dimensions is called the rank.
- Each dimension is called an axis.
Rank-0 tensors:
- These store a single number.
```
import numpy as np

tensor = np.array(5)
print(tensor.ndim) # prints '0'
```
- A rank-0 tensor is also called a scalar or a 0D tensor.
Rank-1 tensors:
- These store an array of numbers.
```
import numpy as np

tensor = np.array([1, 2, 3, 4, 5])
print(tensor.ndim) # prints '1'
```
- A rank-1 tensor is also called a vector or a 1D tensor.
- A rank-1 tensor has 1 axis.
- Note that dimension is also sometimes used to describe the number of items in a vector
  - In this case the vector has a dimension of 5
  - But it is not a 5D tensor.
Rank-2 tensors:
- These store an array of arrays.
```
import numpy as np

tensor = np.array([[1, 2],
                   [3, 4],
                   [5, 6]])
print(tensor.ndim) # prints '2'
```
- A rank-2 tensor is also called a matrix or a 2D tensor.
- It has two axes, which are often called the rows and columns.
- It is often visualized as a rectangular grid of numbers.

Rank-3 tensors:

These store an array of matrices.

import numpy as np

tensor = np.array([[[1, 2],
                    [3, 4]],
                   [[5, 6],
                    [7, 8]]])
print(tensor.ndim) # prints '3'

A rank-3 tensor is also called a 3D tensor.
It can be visualized as a cube of numbers.

Higher level tensors:
- It is possible to define tensors for any number of ranks.
- Machine Learning algorithms usually use tensors with up to 5 ranks.

Tensor Shape

The shape of a tensor describes:
- The number of axes (i.e. the rank).
- The length of each axis.

For example: a matrix with 3 rows and two columns

This is a rank-2 tensor with 3 rows and 2 columns

import numpy as np

tensor = np.array([[1, 2],
                   [3, 4],
                   [5, 6]])
print(tensor.shape) # prints '(3, 2)'

A rank-0 tensor has no shape

import numpy as np

tensor = np.array(5)
print(tensor.shape) # prints '()'

Tensor Data Types

Tensors store data of a specific type.

For numbers, this is usually float32, float64, or uint8

import numpy as np

tensor = np.array([5.0, 6.0])
print(tensor.dtype) # prints 'float64'

For strings, unicode arrays are used:

import numpy as np

tensor = np.array(['cat', 'mouse'])
print(tensor.dtype) # prints '<U5'

A real world example

MNIST is a dataset of 60,000 handwritten numbers.

from tensorflow.keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

We can preview the data and display the first image:

image = x_train[0]

import matplotlib.pyplot as plt
plt.imshow(image, cmap=plt.cm.binary)
plt.show()

We can check the rank, shape, and data type of the data set:

print(x_train.ndim)
print(x_train.shape)
print(x_train.dtype)

3
(60000, 28, 28)
uint8

It contains 60,000 matrices, each is a 28x28 array of uint8 numbers

Slicing a tensor

Sometimes we want to only work with part of a tensor.

To do this, we can slice it:

from tensorflow.keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

slice = x_train[0:100]
print(slice.shape) # prints (100, 28, 28)

This slices it along it's first axis.

We can also slice along multiple axes:

slice = x_train[0:100, 5:23, 5:23]
print(slice.shape) # prints (100, 18, 18)

To specify the start or end of an axis, we can omit the index completely:

slice = x_train[:100]
print(slice.shape) # prints (100, 28, 28)

slice = x_train[100:]
print(slice.shape) # prints (59900, 28, 28)

slice = x_train[:]
print(slice.shape) # prints (60000, 28, 28)

To specify the index relative to the end, use a negative number:

slice = x_train[:, 5:-5, 5:-5]
print(slice.shape) # prints (60000, 18, 18)