Lecture 01: Python primer

Course: Deep Learning for Solving and Estimating Dynamic Models in Economics and Finance
Script reference: Front matter and Appendix~E — NumPy arrays, broadcasting, linear algebra
Notebook role: primer (pre-course self-study; skip if you write Python every day)
Author: Simon Scheidegger

Python Basics 9: Numpy Arrays and Linear Algebra¶

The ability to do basic things with numerical arrays (vectors and matrices) is extremely important in data science applications. We do numerical algebra in Python using the Numpy. Numpy is a huge Python library and it has a lot of details. We are going to cover the very basics here. When there is something specific that you want to do, the best way to figure out how to do it is to Google it.

First, let’s import numpy. We typically, do it by creating a shortcut called np:

In this way you can access whatever is in numpy by going throuh np which is less characters. Let’s create a 1D numpy array from a list:

This behaves just like a vector in Matlab. You can multiply with a number:

You can add another vector (of the same size):

You can raise all elements to a power:

Or you can apply a standard function to all elements:

Notice that the functions you can use are in np. Here is how you can get how many elements you have in the array:

Also, to see if this is a 1D array (higher dimensional arrays, 2D, 3D, etc. are also possible), you can use this:

You access the elements of the array just like the elements of a list:

You can find the minimum or the maximum of the array like this:

So, notice that a is an object of a class. The class is called np.ndarray:

and min() and max() are functions of the np.ndarray class. Another function of the class is the dot product:

There is also a submodule called numpy.linalg which has some linear algebra functions you can apply to arrays. For 1D arrays (aka vectors) the vector norm is a useful one:

Questions¶

Consider the two vectors:

Use numpy functionality to:

• Find a unit vector in the direction of r1:

• Find the angle between r1 and r2 in radians:

• The projection of r2 on r1:

• Use numpy.cross to find the cross product between r2 and r1. Then, verify numerically (using numpy functionality) that $\vec{r}_1\cdot\left(\vec{r}_1\times\vec{r}_2\right) = 0$.

Matrices¶

Now, let’s go to two dimensional arrays. Of course, you can think of the vectors as two dimensional arrays. In particular, you can think of them as a matrix with a single column. To do this in numpy you need to reshape the vectors:

The code a[:, None] means “pretend that the array has one more index that is always zero.”

Now, since acm is a 2D array:

Look also at the shape:

So, the shape is (5, 1). This means that there are 5 rows and one colum. You can access elements in this 2D array with indices like this:

And so on. Of course, the second index can only be zero because you only have one column.

Let’s now make a 2D matrix to play with:

Notice that we used an list of rows and each row was a list of numbers. Here are the dimensions of A:

Let’s access some elements of A:

You can access the 3rd row like this:

You can get a submatrix, say the first 3x3 submatrix like this:

And so on.

You can multiply matrices with numbers:

You can raise them to a power:

You can pass them through functions:

You can add them together:

You can multiply them together (assuming that the dimensions match):

You can multiply a matrix with a vector:

Remember the column matrix acm:

You can also multiply the matrix with it since the dimensions match:

You can also multiply acm (5x1) with its transpose (1x5). The transpose is:

And here is the result of the multiplication:

Here is another example of the transpose operator:

There are some special numpy functions for creating arrays. Here is how to make an array of zeros:

An array of ones:

A unit matrix:

Now, we typically want to do some more linear algebra with matrices. For example, we may want to get the determinant of a matrix:

Or we may want to solve a linear system of the form:

where $b$ is a given vector and $x$ is the uknown vector we are looking for. Let’s create and solve such a linear system.

Let’s see if the solution works as expected by calculating $Ax$:

You can also find the eigenvalues of a matrix:

The eigenvalues are:

The eigenvectors are put in a matrix:

So, each column of that matrix is an eigenvector. Let’s test that

for the $i$-th eigenvalue:

By the way, v[:, i] gives you the $i$-th column of the matrix v.

Questions¶

• Consider the quadratic function:

  $$f(x_1, x_2) = x_1^2 -x_1x_2 + x_2^2 + 3x_1 + 4x_2.$$

  (3)

  We are interested in finding its minimum. A necessary condition for the minimum is that the gradient of $f(x_1, x_2)$ is zero. The gradient is:

  $$\frac{\partial f(x_1,x_2)}{\partial x_1} = 2x_1 -x_2 + 3 = 0,$$

  (4)

  and

  $$\frac{\partial f(x_1, x_2)}{\partial x_2} = -x_1 + 2x_2 + 4 = 0.$$

  (5)

  Rearranging, we get the two equations:

  $$\begin{array}{ccc} 2x_1 - x_2 &=& -3,\\ -x_1 + 2x_2 &=& -4.\\ \end{array}$$

  (6)

  These can also be written in matrix-vector form: $$

= \begin{bmatrix} -3\ -4 \end{bmatrix}. $$Use the numpy functionality to solve this linear system and thus find the minimum of$f(x_1,x_2)$.

• For the point you found above to be a true minimum of the $f(x_1,x_2)$ (and not a maximum for example) is that the matrix of second derivatives (the so-called Hessian matrix) must be positive definite, i.e., that the matrix has only positive eigenvalues. The hessian matrix in this problem is: $$ H = \begin{bmatrix} \frac{\partial^2 f(x_1,x_2)}{\partial x_1^2}& \frac{\partial^2 f(x_1,x_2)}{\partial x_1\partial x_2}\ \frac{\partial^2 f(x_1,x_2)}{\partial x_1\partial x_2} & \frac{\partial^2 f(x_1,x_2)}{\partial x_1^2} \end{bmatrix} =

$$Use numpy functionality to verify that the matrix$H$ is indeed positive definite.

• If $H$ is a positive definite matrix, then the following property is true for any vector $x\not = 0$:

  $$x^THx \ge 0.$$

  (10)

  In words, if you sandwich $H$ around the same vector produce a scalar, that scalar is a positive number. Test this hypothesis numerically using numpy functioanlity. Hint: Use the dot function and twich to calculate $x^THx$.

Multi-dimensional numpy arrays¶

You can have 3D numpy arrays:

Or even higher dimensional:

Of course, most of the time we will be working with 1D (Vectors) and 2D (Matrices).