```{r setup, include=FALSE}
library(knitr) 
knitr::opts_chunk$set(echo = TRUE, out.width = "100%", fig.align='center', 
                      message=FALSE, warning = FALSE)
options(width = 1200)
```

# Introduction and objectives

This session will focus on linear maps. Specifically, we aim to accomplish the following objectives:

1. Relate the geometry of linear maps to the concepts of rank, null space, determinant and invertibility.
2. Create, subset, join, operate and visualize matrices in R.


<!-- # Revision of pre-requisites -->

<!-- Checkout the pre-requisites in the course website. -->



# Linear transformations

Simply put, we can think of a transformation as a function or program: something that takes an input vector and "spits" an output vector. Such a transformation is deemed "linear" if it fulfills two additional requirements:



1. Applying the transformation of a sum of vectors gives the same result as applying the sum of the respective transformations:

$$
L(u + v) = L(u) + L(v)
$$

2. Scaling a vector by a scalar $\lambda$ then applying the transformation gives the same result as applying the transformation of the vector first then scaling the result by $\lambda$:

$$
L(\lambda u) = \lambda L(u)
$$

In particular, applying a linear transformation to the zero vector gives always the zero vector as result.
$$
L(\vec{0})=\vec{0}
$$
&nbsp;


As we have previously studied, linear maps can be represented with matrices. To obtain the result of applying the transformation to a vector, we just need to multiply the matrix associated to the linear map with the vector.

$$
L(u)= Au 
$$

```{r echo=FALSE}
x <- matrix(c("u1", "u2"), nrow = 2)
i_hat <- matrix(c(1, 0), nrow = 2)
j_hat <- matrix(c(0, 1), nrow = 2)

i_hat_r <- matrix(c(2,0), nrow=2)
j_hat_r <- matrix(c(2,1), nrow=2)

write_matex2 <- function(x) {
  begin <- "\\begin{bmatrix}"
  end <- "\\end{bmatrix}"
  X <-
    apply(x, 1, function(x) {
      paste(
        paste(x, collapse = "&"),
        "\\\\"
      )
    })
  paste(c(begin, X, end), collapse = "")
}
```


Let us see an example:


```{r echo=FALSE}
E1 <- matrix(c(2, 0, 2, 1), 2, 2, byrow = FALSE)
v1 <- matrix(c(1,3), nrow=2)
v1r <- matrix(c(8,3), nrow=2)
```

We will consider the following matrix to define a linear map: 

$$
A = `r write_matex2(E1)`
$$

And the following vector:
$$
u = `r write_matex2(v1)`
$$

We obtain the result of applying the transformation to a vector by multiplying the matrix associated to the linear map with the vector.

$$
L(u)= Au = `r write_matex2(E1)``r write_matex2(v1)`=`r write_matex2(v1r)`
$$


In fact, each of the column vectors of the matrix are the output vector when we apply the transformation to each of the canonical basis vectors.



$$
`r write_matex2(E1)``r write_matex2(i_hat)`=`r write_matex2(i_hat_r)`  \hspace{15pt}  and   \hspace{15pt}
`r write_matex2(E1)``r write_matex2(j_hat)`=`r write_matex2(j_hat_r)`
$$

&nbsp;

&nbsp;


We can think of linear transformations as linear combinations of the column vectors of the matrix. 

To illustrate this, consider the following vector $u = `r write_matex2(x)` \in R^2$, which can be expressed as a linear combination of the vectors of the canonical basis:

$$
u = u_1 `r write_matex2(i_hat)` + u_2 `r write_matex2(j_hat)`
$$

Let us take the linear transformation $L$ of the vector $u$ and apply the two above mentioned properties of linear transformations:

$$
L(u) = L \left( u_1 `r write_matex2(i_hat)` + u_2 `r write_matex2(j_hat)`\right) = L \left(u_1 `r write_matex2(i_hat)`\right) + L \left(u_2 `r write_matex2(j_hat)`\right) = u_1 L\left(`r write_matex2(i_hat)`\right) + u_2 L\left(`r write_matex2(j_hat)`\right)
$$

&nbsp;


In our example: 
$$
u = `r write_matex2(v1)` = 1 `r write_matex2(i_hat)` + 3 `r write_matex2(j_hat)`
$$
$$
L(u) = 1 L\left(`r write_matex2(i_hat)`\right) + 3 L\left(`r write_matex2(j_hat)`\right) = 
1 `r write_matex2(i_hat_r)` + 3 `r write_matex2(j_hat_r)` = `r write_matex2(v1r)`
$$

&nbsp; 


**We can conclude that:**

1. Linear transformations can be expressed as a matrix-vector product.

$$
L(u) = Au
$$

2. Linear transformations are completely determined by the transformations of the vectors of the canonical basis.

$$
L(u) = u_1 L(e_1) + u_2 L(e_2)
$$

```{r echo=FALSE}
v2 <- matrix(c(5,6), nrow = 2)
B <- matrix(c(2, -1, 5, 3), nrow = 2, ncol = 2, byrow = FALSE)
```
**Exercise** 

1. Calculate the transformation of the vector $v = `r write_matex2(v2)` \in R^2$ with the linear map defined with the matrix $B$.
$$
B = `r write_matex2(B)`
$$

2. Determine if the following vector functions are linear transformations:

- $F_1(u_1,u_2)=(u_1+u_2,2u_1)$
- $F_2(u_1,u_2)=(0,1)$
- $F_3(u_1,u_2,u_3)=(0,0)$
- $F_4(u_1,u_2)=(u_1^2, u_1-u_2)$
- $F_5(u_1,u_2)=(u_1 \cos(\pi/4)-u_2 \sin(\pi/4),u_1 \sin(\pi/4)+u_2 \cos(\pi/4))$
- $F_6(u_1,u_2)=(u_1+2,2u_1,0)$




&nbsp;

&nbsp;

## Important definitions

Given an $nxn$ matrix $A$, we can define different concepts associated to it:

* **Column space**: vector space spanned by the column vectors of $A$, denoted $C(A)$.
* **Rank**: number of linearly independent columns of $A$, which coincides with the dimension of $C(A)$ and the dimension of $R(A)$.
* **Determinant**: scalar value that can be calculated recursively based on the elements of the matrix (see chapter 3 of the course notes).<br> It has the following property: the determinant is the factor by which a unit of length (n=1), area (n=2) or volume (n=3) is multiplied after transforming the space with the linear transformation encoded by $A$.
* **Inverse matrix**: matrix that, when multiplied with the original matrix, results in the identity matrix. If $A^{-1}$ is the inverse matrix of A, then $AA^{-1} = A^{-1}A = I$.
* **Null space**: denoted as $N(A)$ is the vector space of all vectors that are transformed to the zero-vector by means of $A$. In other words:

$$
N(A) = \left\{ w \in \mathbb{R}^n \hspace{5pt} |\hspace{5pt} Aw = \vec{0} \right\}
$$

![](../img/linear_transformations_shed_light.png)

<!-- Again, the main objective is **to develop the geometric intuition of linear transformations to see how all these concepts are connected**: -->

<!-- There are two of these relations that are useful to understand algebraically: -->

<!-- 1. Relation between rank and dim(N(A)): -->
## Important relations

**Fundamental Theorem of Linear Algebra**

For any $n$ x $m$ matrix $A$,
$$
dim(C(A)) + dim(N(A)) = m
$$

**Other relations**

An $n$ x $m$ matrix **$A$ is full rank if and only if the null space is the zero vector space**. Remember that a matrix is full rank also means that all column vectors are linearly independent. 

In a more condensed notation, if $\hspace{2pt} a_1, \ldots, a_m$ are the column vectors of $A$, then:
$$
a_1, \ldots, a_m \hspace{0.2cm} \textrm{are linearly independent} \Longleftrightarrow N(A) = \{\vec{0}\} 
$$

You can find the proof of this theorem in the [Khan Academy](https://en.khanacademy.org/math/linear-algebra/vectors-and-spaces/null-column-space/v/null-space-3-relation-to-linear-independence) videos in the prerequisites sections.

&nbsp;

Additionally, if the matrix **$A$ is a square matrix** of size $n$, the following are equivalent (see chapter 3 of the course notes): 

<!-- 2. Relation between dim(N(A)) and invertibility: -->

- $A$ is full rank, that means that all the column vectors are linearly independent.
- $N(A) = \{\vec{0}\}$ which means that $dim(N(A))=0$
- $det(A) \neq 0$
- $A$ is invertible
- The columns of $A$ are a basis of $\mathbb{R}^n$
- The rows of $A$ are a basis of $\mathbb{R}^n$

If the **matrix is not full rank** then $det(A)=0$ and $dim(N(A))>0$. 

&nbsp;



# Shiny app walkthrough

Herein, we will walk you through our shiny app that will allow you to fully understand linear transformations. Start by clicking the following link (or pasting it to your favorite web browser):

[https://massonix.shinyapps.io/linear_transformations/](https://massonix.shinyapps.io/linear_transformations/)

As you can see, there are 4 tabs:

1. **Visualization**, which has two components:
    - First there is an input box where the user should
specify the dimension and the matrix A that defines the linear transformation. 
    - Second there is
the plot pane that shows the input and the output of the transformation.
2. **Gauss-Jordan**. 
    - It contains a calculator that performs Gauss-Jordan elimination column-wise. The algorithm can be run either step-wise or fast-forwarded to the final result. 
    - It provide us with two outputs: column echelon form (top) and transformation matrix (bottom). 
    - With this tab you can get an intuition of the Fundamental Theorem of Linear Algebra.
3. **Null Space**. It calculates and represents the null space of the input matrix.
4. **Determinant**. It calculates the determinant of the input matrix.

<!-- To illustrate how all these concepts are connected, we start by fixing the rank (number of linearly independent columns) and see how that affects the rest of properties. We will use the following example matrices: -->

**Exercise** <br>
Calculate the column space, rank, determinant, inverse matrix and null space of the following matrices and check the previous concepts. 

```{r echo=FALSE}
E1 <- matrix(c(-2, 0, 0, -2), 2, 2, byrow = FALSE)
E2 <- matrix(c(1, 3, 3, 9), 2, 2, byrow = FALSE)
E3 <- matrix(c(1, 2, 3, 2, 4, 6, 3, 6, 9), 3, 3, byrow = FALSE)
```

$$
E_1 = `r write_matex2(E1)`
$$

$$
E_2 = `r write_matex2(E2)`
$$

$$
E_3 = `r write_matex2(E3)`
$$

<!-- After this, you should realize two important connections: -->

<!-- 1. If the matrix is not full rank the determinant is 0 -->
<!-- 2. dim(C(A)) + dim(N(A)) = n -->


# Reverse engineer linear transformations

## Basic transformations

**Exercise: find the matrix that describes the following transformations of 2D space:**

Remember that to find the matrix associated to the transformation you only need to find the transformations of the basis vectors. 

1. Expands all lengths by a factor of 2 in all axes;
2. Shrinks all lengths by a factor of 1/3;
3. Rotates all vectors by $\pi/3$ radians (60 degrees) about the origin (hint: use [unit circle](https://en.wikipedia.org/wiki/Unit_circle) and this [trigonometry calculator](https://www.rapidtables.com/calc/math/trigonometry-calculator.html)).
4. Reflects all vectors about the $y$-axis.
5. Projects all vectors onto the $x$-axis.

**Exercise: find the matrix that describes the following transformations of 3D space:**

1. Expands all lengths by a factor of 2 in all axes;
2. Shrinks all lengths by a factor of 1/2;
3. Reflects all vectors with respect to the $\{x,y\}$ plane;
4. Projects all vectors onto the $\{x,z\}$ plane;
5. Rotates all vectors by $\pi/6$ radians (30 degrees) around the $z$-axis (hint: use unit circle)

**For each transformation, you should answer these questions:**

- Are the column vectors linearly independent?
- Is the matrix invertible?
- Which is the dimension of the null space?
- Which is the determinant of A?

We will leave an average of 2-3 min for each exercise, depending on the difficulty, then we will ask you 
to tell us which matrices you guessed.

## Composition of basic transformations

If we have two matrices $A\in \mathbb{R}^{mxn}$ and $B\in \mathbb{R}^{kxm}$ associated to two linear transformations, then $C=BA\in \mathbb{R}^{kxn}$ encodes the linear transformation obtained when applying first the transformation of $A$ and then the transformation of $B$. 
<!-- In other words, whenever you are considering a product of matrices $C=AB$ as a single transformation: -->

<!-- $$ -->
<!-- Cx = (AB)x = y -->
<!-- $$ -->

<!-- In other words, the transformation $C$ is encoding the concatenation of the transformation $A$ after $B$, in the sense that first x is the input of $B$, and then the output is the input of A: -->

$$
u \stackrel{A}{\longmapsto} (Au) \stackrel{B}{\longmapsto} B(Au) = (BA)u = Cu
$$
So every time we include "linear transformation" step, we are multiplying by a matrix on the left.

You can find a more thorough explanation in [this 3blue1brown video](https://www.youtube.com/watch?v=XkY2DOUCWMU).

**Observation**
If $A$ is a full rank square matrix and $A^{-1}$ is the inverse matrix of the $A$, $A^{-1}$ reverts the effect of $A$.

$$
u \stackrel{A}{\longmapsto} (Au) \stackrel{A^{-1}}{\longmapsto} A^{-1}(Au) = (A^{-1}A)u = Iu = u
$$

**Exercise**

Using the matrices from the previous exercise, try to find the matrix that describes the following transformations:

1. (2D) First expands all lengths by a factor of 2 in all axes and then rotates all vectors by $\pi/3$ radians (60 degrees) about the origin.
2. (2D) First rotates all vectors by $\pi/3$ radians (60 degrees) about the origin and then Projects all vectors onto the x-axis.
3. (3D) First reflects all vectors with respect to the $\{x,y\}$ plane and then projects all vectors onto the $\{x,z\}$ plane.
4. (3D) First rotates all vectors by $\pi/3$ radians round the $x$ axis (matrix C), then rotates  $\pi/3$ radians round the $y$ axis (matrix B) and then  $\pi/3$ radians round the $z$ axis (matrix A).
5. In the previous examples, does the order of matrix multiplication matter?
6. In the exercise 4, is $A(BC)$ equal to $(AB)C$? 


# Matrix algebra with R

## Introduction to R

R is an open-source and **vectorized** programming language. In R we never deal with scalars: single values are vectors of length 1:

```{r}
x <- 5
length(x)
```

It is also a **functional** programming language, which means that operations are centered around functions, and that it provides many tools for the creation and manipulation of functions (see [this book](http://adv-r.had.co.nz/) for more). Most importantly, functions take inputs, and this inputs can be of many different classes:

```{r}
class(x)
```

### Basic data types

We have 3 basic data types: **numerics, characters and logicals**:

**Numerics** are needed to perform arithmetic operations:

```{r}
y <- 9
z <- x + y
class(z)
```

**Characters** are what in other languages are known as strings:

```{r}
quote <- "Gilbert Strang rocks!"
print(quote)
class(quote)
```

Finally, **logicals** (known as booleans in other programming languages) are important in conditions:

```{r}
(5 + 5) == 10
(6 - 7) > 0
```

We can convert between data types with the **`as.*()`** functions:

```{r}
logical <- TRUE
as.character(logical)
as.numeric(logical)
```


### Compound data types

When we have more than one element, we can think of compound data types. They can be classified using two criteria:

- Dimension: 1D or 2D
- Homogeneous or heterogeneous, defined by whether they store a single or multiple basic data types, respectively.

Combining both criteria, we get the following classification of R's data structures (see [this chapter](http://adv-r.had.co.nz/Data-structures.html) for more):

|    | Homogeneous   | Heterogeneous |
|----|---------------|---------------|
| 1D | Atomic vector | List          |
| 2D | Matrix        | Data frame    |

They are created with specific functions:

```{r}
my_vector <- c(1, 2, 3)
my_list <- list(1, "Ferran", TRUE)
my_matrix <- matrix(c(1, 2, 3, 4), nrow = 2, ncol = 2, byrow = FALSE)
my_data_frame <- data.frame(name = c("Ferran", "Paula"), male = c(TRUE, FALSE))
my_vector
my_list
my_matrix
my_data_frame
```

As you can imagine, these data structures are used for different purposes and taken by distinct functions. Here, we will focus on matrices.

This section aims to teach the fundamentals of R so that you will be able to perform linear algebra operations and visualize them. 

<!-- We will start by teaching the basic data structures, categorized by dimensionality (1D: atomic vectors + lists; 2D: matrices + data.frames) and whether they can contain only one value type (homogeneous: atomic vectors and matrices) or more than one (heterogeneous: lists and data.frames). Moreover, here I showcase an example of the main functions we should cover in the hands-on session: -->


## Create, subset and combine vectors and matrices in R

We will show these operations for both atomic vectors and matrices. Let us start by creating a vector and a matrix:

```{r}
# Create
x <- c(7,3,8)
A <- matrix(c(1, 2, 3, 4, 5, 6, 7, 8, 9), nrow = 3, ncol = 3, byrow = FALSE)
x 
A
```


 - Get the dimensions:

```{r}
length(x)
dim(A)
nrow(A)
ncol(A)
```


- To ease subsetting we can name rows and columns as follows:

```{r}
names(x) <- c("1", "2", "3")
rownames(A) <- c("a", "b", "c")
colnames(A) <- c("x", "y", "z")
x
A
```

- There are **3 ways** of subsetting a vector or a matrix: by index, by character or by logicals:

```{r}
# Remember x = (7,3,8)
# Subset
## Numeric
x[1]
x[c(2, 3)]
x[c(1, 3, 3)]
A[1:3, -2]

## Character
x["2"]
A[c("a", "b"), ]
```


We can concatenate vectors and matrices as follows:

```{r}
# Concatenate
y <- c(5, 6)
z <- c(x, y)
z
rbind(A, x)
cbind(A, x)
```

## Basic operations with vectors

```{r}
# Vector multiplication element by element
# (7*7,3*3,8*8)
x*x

# Dot product 
# 7*7+3*3+8*8
x%*%x
```


## Basic operations with matrices

```{r}
# Transpose
B <- t(A)
B

# Sum of matrices
A + A

# Matrix multiplication element by element
A * A

# Matrix multiplication
A %*% A
t(x[1] %*% A[, 1]) + t(x[2] %*% A[, 2]) + t(x[3] %*% A[, 3])


# Matrix-vector multiplication
A %*% x
```

# Session Information

```{r}
sessionInfo()
```


