1 Introduction and objectives

This session will focus on linear maps. Specifically, we aim to accomplish the following objectives:

Relate the geometry of linear maps to the concepts of rank, null space, determinant and invertibility.
Create, subset, join, operate and visualize matrices in R.

2 Linear transformations

Simply put, we can think of a transformation as a function or program: something that takes an input vector and “spits” an output vector. Such a transformation is deemed “linear” if it fulfills two additional requirements:

Applying the transformation of a sum of vectors gives the same result as applying the sum of the respective transformations:

\[ L(u + v) = L(u) + L(v) \]

Scaling a vector by a scalar \(\lambda\) then applying the transformation gives the same result as applying the transformation of the vector first then scaling the result by \(\lambda\):

\[ L(\lambda u) = \lambda L(u) \]

In particular, applying a linear transformation to the zero vector gives always the zero vector as result. \[ L(\vec{0})=\vec{0} \]

As we have previously studied, linear maps can be represented with matrices. To obtain the result of applying the transformation to a vector, we just need to multiply the matrix associated to the linear map with the vector.

\[ L(u)= Au \]

Let us see an example:

We will consider the following matrix to define a linear map:

\[ A = \begin{bmatrix}2&2 \\0&1 \\\end{bmatrix} \]

And the following vector: \[ u = \begin{bmatrix}1 \\3 \\\end{bmatrix} \]

We obtain the result of applying the transformation to a vector by multiplying the matrix associated to the linear map with the vector.

\[ L(u)= Au = \begin{bmatrix}2&2 \\0&1 \\\end{bmatrix}\begin{bmatrix}1 \\3 \\\end{bmatrix}=\begin{bmatrix}8 \\3 \\\end{bmatrix} \]

In fact, each of the column vectors of the matrix are the output vector when we apply the transformation to each of the canonical basis vectors.

\[ \begin{bmatrix}2&2 \\0&1 \\\end{bmatrix}\begin{bmatrix}1 \\0 \\\end{bmatrix}=\begin{bmatrix}2 \\0 \\\end{bmatrix} \hspace{15pt} and \hspace{15pt} \begin{bmatrix}2&2 \\0&1 \\\end{bmatrix}\begin{bmatrix}0 \\1 \\\end{bmatrix}=\begin{bmatrix}2 \\1 \\\end{bmatrix} \]

We can think of linear transformations as linear combinations of the column vectors of the matrix.

To illustrate this, consider the following vector \(u = \begin{bmatrix}u1 \\u2 \\\end{bmatrix} \in R^2\), which can be expressed as a linear combination of the vectors of the canonical basis:

\[ u = u_1 \begin{bmatrix}1 \\0 \\\end{bmatrix} + u_2 \begin{bmatrix}0 \\1 \\\end{bmatrix} \]

Let us take the linear transformation \(L\) of the vector \(u\) and apply the two above mentioned properties of linear transformations:

\[ L(u) = L \left( u_1 \begin{bmatrix}1 \\0 \\\end{bmatrix} + u_2 \begin{bmatrix}0 \\1 \\\end{bmatrix}\right) = L \left(u_1 \begin{bmatrix}1 \\0 \\\end{bmatrix}\right) + L \left(u_2 \begin{bmatrix}0 \\1 \\\end{bmatrix}\right) = u_1 L\left(\begin{bmatrix}1 \\0 \\\end{bmatrix}\right) + u_2 L\left(\begin{bmatrix}0 \\1 \\\end{bmatrix}\right) \]

In our example: \[ u = \begin{bmatrix}1 \\3 \\\end{bmatrix} = 1 \begin{bmatrix}1 \\0 \\\end{bmatrix} + 3 \begin{bmatrix}0 \\1 \\\end{bmatrix} \] \[ L(u) = 1 L\left(\begin{bmatrix}1 \\0 \\\end{bmatrix}\right) + 3 L\left(\begin{bmatrix}0 \\1 \\\end{bmatrix}\right) = 1 \begin{bmatrix}2 \\0 \\\end{bmatrix} + 3 \begin{bmatrix}2 \\1 \\\end{bmatrix} = \begin{bmatrix}8 \\3 \\\end{bmatrix} \]

We can conclude that:

Linear transformations can be expressed as a matrix-vector product.

\[ L(u) = Au \]

Linear transformations are completely determined by the transformations of the vectors of the canonical basis.

\[ L(u) = u_1 L(e_1) + u_2 L(e_2) \]

Exercise

Calculate the transformation of the vector \(v = \begin{bmatrix}5 \\6 \\\end{bmatrix} \in R^2\) with the linear map defined with the matrix \(B\). \[ B = \begin{bmatrix}2&5 \\-1&3 \\\end{bmatrix} \]
Determine if the following vector functions are linear transformations:

\(F_1(u_1,u_2)=(u_1+u_2,2u_1)\)
\(F_2(u_1,u_2)=(0,1)\)
\(F_3(u_1,u_2,u_3)=(0,0)\)
\(F_4(u_1,u_2)=(u_1^2, u_1-u_2)\)
\(F_5(u_1,u_2)=(u_1 \cos(\pi/4)-u_2 \sin(\pi/4),u_1 \sin(\pi/4)+u_2 \cos(\pi/4))\)
\(F_6(u_1,u_2)=(u_1+2,2u_1,0)\)

2.1 Important definitions

Given an \(nxn\) matrix \(A\), we can define different concepts associated to it:

Column space: vector space spanned by the column vectors of \(A\), denoted \(C(A)\).
Rank: number of linearly independent columns of \(A\), which coincides with the dimension of \(C(A)\) and the dimension of \(R(A)\).
Determinant: scalar value that can be calculated recursively based on the elements of the matrix (see chapter 3 of the course notes).
It has the following property: the determinant is the factor by which a unit of length (n=1), area (n=2) or volume (n=3) is multiplied after transforming the space with the linear transformation encoded by \(A\).
Inverse matrix: matrix that, when multiplied with the original matrix, results in the identity matrix. If \(A^{-1}\) is the inverse matrix of A, then \(AA^{-1} = A^{-1}A = I\).
Null space: denoted as \(N(A)\) is the vector space of all vectors that are transformed to the zero-vector by means of \(A\). In other words:

\[ N(A) = \left\{ w \in \mathbb{R}^n \hspace{5pt} |\hspace{5pt} Aw = \vec{0} \right\} \]

2.2 Important relations

Fundamental Theorem of Linear Algebra

For any \(n\) x \(m\) matrix \(A\), \[ dim(C(A)) + dim(N(A)) = m \]

Other relations

An \(n\) x \(m\) matrix \(A\) is full rank if and only if the null space is the zero vector space. Remember that a matrix is full rank also means that all column vectors are linearly independent.

In a more condensed notation, if \(\hspace{2pt} a_1, \ldots, a_m\) are the column vectors of \(A\), then: \[ a_1, \ldots, a_m \hspace{0.2cm} \textrm{are linearly independent} \Longleftrightarrow N(A) = \{\vec{0}\} \]

You can find the proof of this theorem in the Khan Academy videos in the prerequisites sections.

Additionally, if the matrix \(A\) is a square matrix of size \(n\), the following are equivalent (see chapter 3 of the course notes):

\(A\) is full rank, that means that all the column vectors are linearly independent.
\(N(A) = \{\vec{0}\}\) which means that \(dim(N(A))=0\)
\(det(A) \neq 0\)
\(A\) is invertible
The columns of \(A\) are a basis of \(\mathbb{R}^n\)
The rows of \(A\) are a basis of \(\mathbb{R}^n\)

If the matrix is not full rank then \(det(A)=0\) and \(dim(N(A))>0\).

3 Shiny app walkthrough

Herein, we will walk you through our shiny app that will allow you to fully understand linear transformations. Start by clicking the following link (or pasting it to your favorite web browser):

https://massonix.shinyapps.io/linear_transformations/

As you can see, there are 4 tabs:

Visualization, which has two components:
- First there is an input box where the user should specify the dimension and the matrix A that defines the linear transformation.
- Second there is the plot pane that shows the input and the output of the transformation.
Gauss-Jordan.
- It contains a calculator that performs Gauss-Jordan elimination column-wise. The algorithm can be run either step-wise or fast-forwarded to the final result.
- It provide us with two outputs: column echelon form (top) and transformation matrix (bottom).
- With this tab you can get an intuition of the Fundamental Theorem of Linear Algebra.
Null Space. It calculates and represents the null space of the input matrix.
Determinant. It calculates the determinant of the input matrix.

Exercise
Calculate the column space, rank, determinant, inverse matrix and null space of the following matrices and check the previous concepts.

\[ E_1 = \begin{bmatrix}-2&0 \\0&-2 \\\end{bmatrix} \]

\[ E_2 = \begin{bmatrix}1&3 \\3&9 \\\end{bmatrix} \]

\[ E_3 = \begin{bmatrix}1&2&3 \\2&4&6 \\3&6&9 \\\end{bmatrix} \]

4 Reverse engineer linear transformations

4.1 Basic transformations

Exercise: find the matrix that describes the following transformations of 2D space:

Remember that to find the matrix associated to the transformation you only need to find the transformations of the basis vectors.

Expands all lengths by a factor of 2 in all axes;
Shrinks all lengths by a factor of 1/3;
Rotates all vectors by \(\pi/3\) radians (60 degrees) about the origin (hint: use unit circle and this trigonometry calculator).
Reflects all vectors about the \(y\)-axis.
Projects all vectors onto the \(x\)-axis.

Exercise: find the matrix that describes the following transformations of 3D space:

Expands all lengths by a factor of 2 in all axes;
Shrinks all lengths by a factor of 1/2;
Reflects all vectors with respect to the \(\{x,y\}\) plane;
Projects all vectors onto the \(\{x,z\}\) plane;
Rotates all vectors by \(\pi/6\) radians (30 degrees) around the \(z\)-axis (hint: use unit circle)

For each transformation, you should answer these questions:

Are the column vectors linearly independent?
Is the matrix invertible?
Which is the dimension of the null space?
Which is the determinant of A?

We will leave an average of 2-3 min for each exercise, depending on the difficulty, then we will ask you to tell us which matrices you guessed.

4.2 Composition of basic transformations

If we have two matrices \(A\in \mathbb{R}^{mxn}\) and \(B\in \mathbb{R}^{kxm}\) associated to two linear transformations, then \(C=BA\in \mathbb{R}^{kxn}\) encodes the linear transformation obtained when applying first the transformation of \(A\) and then the transformation of \(B\).

\[ u \stackrel{A}{\longmapsto} (Au) \stackrel{B}{\longmapsto} B(Au) = (BA)u = Cu \] So every time we include “linear transformation” step, we are multiplying by a matrix on the left.

You can find a more thorough explanation in this 3blue1brown video.

Observation If \(A\) is a full rank square matrix and \(A^{-1}\) is the inverse matrix of the \(A\), \(A^{-1}\) reverts the effect of \(A\).

\[ u \stackrel{A}{\longmapsto} (Au) \stackrel{A^{-1}}{\longmapsto} A^{-1}(Au) = (A^{-1}A)u = Iu = u \]

Exercise

Using the matrices from the previous exercise, try to find the matrix that describes the following transformations:

(2D) First expands all lengths by a factor of 2 in all axes and then rotates all vectors by \(\pi/3\) radians (60 degrees) about the origin.
(2D) First rotates all vectors by \(\pi/3\) radians (60 degrees) about the origin and then Projects all vectors onto the x-axis.
(3D) First reflects all vectors with respect to the \(\{x,y\}\) plane and then projects all vectors onto the \(\{x,z\}\) plane.
(3D) First rotates all vectors by \(\pi/3\) radians round the \(x\) axis (matrix C), then rotates \(\pi/3\) radians round the \(y\) axis (matrix B) and then \(\pi/3\) radians round the \(z\) axis (matrix A).
In the previous examples, does the order of matrix multiplication matter?
In the exercise 4, is \(A(BC)\) equal to \((AB)C\)?

5 Matrix algebra with R

5.1 Introduction to R

R is an open-source and vectorized programming language. In R we never deal with scalars: single values are vectors of length 1:

x <- 5
length(x)

## [1] 1

It is also a functional programming language, which means that operations are centered around functions, and that it provides many tools for the creation and manipulation of functions (see this book for more). Most importantly, functions take inputs, and this inputs can be of many different classes:

class(x)

## [1] "numeric"

5.1.1 Basic data types

We have 3 basic data types: numerics, characters and logicals:

Numerics are needed to perform arithmetic operations:

y <- 9
z <- x + y
class(z)

## [1] "numeric"

Characters are what in other languages are known as strings:

quote <- "Gilbert Strang rocks!"
print(quote)

## [1] "Gilbert Strang rocks!"

class(quote)

## [1] "character"

Finally, logicals (known as booleans in other programming languages) are important in conditions:

(5 + 5) == 10

## [1] TRUE

(6 - 7) > 0

## [1] FALSE

We can convert between data types with the as.*() functions:

logical <- TRUE
as.character(logical)

## [1] "TRUE"

as.numeric(logical)

## [1] 1

5.1.2 Compound data types

When we have more than one element, we can think of compound data types. They can be classified using two criteria:

Dimension: 1D or 2D
Homogeneous or heterogeneous, defined by whether they store a single or multiple basic data types, respectively.

Combining both criteria, we get the following classification of R’s data structures (see this chapter for more):

	Homogeneous	Heterogeneous
1D	Atomic vector	List
2D	Matrix	Data frame

They are created with specific functions:

my_vector <- c(1, 2, 3)
my_list <- list(1, "Ferran", TRUE)
my_matrix <- matrix(c(1, 2, 3, 4), nrow = 2, ncol = 2, byrow = FALSE)
my_data_frame <- data.frame(name = c("Ferran", "Paula"), male = c(TRUE, FALSE))
my_vector

## [1] 1 2 3

my_list

## [[1]]
## [1] 1
## 
## [[2]]
## [1] "Ferran"
## 
## [[3]]
## [1] TRUE

my_matrix

##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4

my_data_frame

##     name  male
## 1 Ferran  TRUE
## 2  Paula FALSE

As you can imagine, these data structures are used for different purposes and taken by distinct functions. Here, we will focus on matrices.

This section aims to teach the fundamentals of R so that you will be able to perform linear algebra operations and visualize them.

5.2 Create, subset and combine vectors and matrices in R

We will show these operations for both atomic vectors and matrices. Let us start by creating a vector and a matrix:

# Create
x <- c(7,3,8)
A <- matrix(c(1, 2, 3, 4, 5, 6, 7, 8, 9), nrow = 3, ncol = 3, byrow = FALSE)
x

## [1] 7 3 8

##      [,1] [,2] [,3]
## [1,]    1    4    7
## [2,]    2    5    8
## [3,]    3    6    9

Get the dimensions:

length(x)

## [1] 3

dim(A)

## [1] 3 3

nrow(A)

## [1] 3

ncol(A)

## [1] 3

To ease subsetting we can name rows and columns as follows:

names(x) <- c("1", "2", "3")
rownames(A) <- c("a", "b", "c")
colnames(A) <- c("x", "y", "z")
x

## 1 2 3 
## 7 3 8

##   x y z
## a 1 4 7
## b 2 5 8
## c 3 6 9

There are 3 ways of subsetting a vector or a matrix: by index, by character or by logicals:

# Remember x = (7,3,8)
# Subset
## Numeric
x[1]

## 1 
## 7

x[c(2, 3)]

## 2 3 
## 3 8

x[c(1, 3, 3)]

## 1 3 3 
## 7 8 8

A[1:3, -2]

##   x z
## a 1 7
## b 2 8
## c 3 9

## Character
x["2"]

## 2 
## 3

A[c("a", "b"), ]

##   x y z
## a 1 4 7
## b 2 5 8

We can concatenate vectors and matrices as follows:

# Concatenate
y <- c(5, 6)
z <- c(x, y)
z

## 1 2 3     
## 7 3 8 5 6

rbind(A, x)

##   x y z
## a 1 4 7
## b 2 5 8
## c 3 6 9
## x 7 3 8

cbind(A, x)

##   x y z x
## a 1 4 7 7
## b 2 5 8 3
## c 3 6 9 8

5.3 Basic operations with vectors

# Vector multiplication element by element
# (7*7,3*3,8*8)
x*x

##  1  2  3 
## 49  9 64

# Dot product 
# 7*7+3*3+8*8
x%*%x

##      [,1]
## [1,]  122

5.4 Basic operations with matrices

# Transpose
B <- t(A)
B

##   a b c
## x 1 2 3
## y 4 5 6
## z 7 8 9

# Sum of matrices
A + A

##   x  y  z
## a 2  8 14
## b 4 10 16
## c 6 12 18

# Matrix multiplication element by element
A * A

##   x  y  z
## a 1 16 49
## b 4 25 64
## c 9 36 81

# Matrix multiplication
A %*% A

##    x  y   z
## a 30 66 102
## b 36 81 126
## c 42 96 150

t(x[1] %*% A[, 1]) + t(x[2] %*% A[, 2]) + t(x[3] %*% A[, 3])

##      [,1]
## [1,]   75
## [2,]   93
## [3,]  111

# Matrix-vector multiplication
A %*% x

##   [,1]
## a   75
## b   93
## c  111

6 Session Information

sessionInfo()

## R version 4.4.1 (2024-06-14)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 20.04.6 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=es_ES.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=es_ES.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=es_ES.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Europe/Madrid
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] knitr_1.48       BiocStyle_2.32.1
## 
## loaded via a namespace (and not attached):
##  [1] digest_0.6.37       R6_2.5.1            bookdown_0.40       fastmap_1.2.0       xfun_0.47           cachem_1.1.0        htmltools_0.5.8.1   rmarkdown_2.28      lifecycle_1.0.4     cli_3.6.3           sass_0.4.9          jquerylib_0.1.4     compiler_4.4.1      rstudioapi_0.16.0   tools_4.4.1         evaluate_0.24.0     bslib_0.8.0         yaml_2.3.10         BiocManager_1.30.25 jsonlite_1.8.8      rlang_1.1.4

Hands on session 1: Linear Transformations

09/10/2024