From today, I will post the study notes fo Linear Algebra (following the MIT 18.06). In the meantime, I am going to finish the assignments in this class and post my solutions.

The web page and videos are available on http://web.mit.edu/18.06/www/

The assignments are available on http://web.mit.edu/18.06/www/Fall2022/ and https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/pages/assignments/

The fundamental problem of linear algebra is to solve a system of linear equations. First, we will talk about the most normal and nice case, that is “n linear equations, n unknowns”.

Row picture

Take a example

$$ \begin{cases} 2&x-&y=0 \\ -&x+2&y=3 \end{cases} $$

Then, we can easily get the matrix form, the cofficent of matrix, the vector of unknowns and vector of right hands number.

$$ \left[ \begin{matrix} 2 & -1 \\ -1 & 2 \end{matrix} \right] \left[ \begin{matrix} x \\ y \end{matrix} \right] = \left[ \begin{matrix} 0 \\ 3 \end{matrix} \right] $$

Actually, we often note the above matrix as ${ A }$, the vector of unknowns as ${ \boldsymbol{x} }$ and the right-hand vector ${ b }$, then we get

$$ A \boldsymbol{x} = b $$

Then we will draw the row picture, we pick one row at a time and draw it in the x-y plane. We can get the solution ${ x=1, y=2 }$, which is the point that lies on both lines.

Column picture (#)

Column picture is the key point. We can treat the above equation as follow.

$$ x \left[ \begin{matrix} 2 \\ -1 \end{matrix} \right] + y \left[ \begin{matrix} -1 \\ 2 \end{matrix} \right] = \left[ \begin{matrix} 0 \\ 3 \end{matrix} \right] $$

So, our goal is how to combine the two vectors in the right amounts to get the right-hand vector. Actually, this process called “Linear Combination”, what we do is to find a right linear combination of “columns” to fit the problem. Let’s check the solution ${ (1,2) }$ in the column picture.

If we pick all the ${ x }$ and all the ${ y }$, we can get any right-hand vector, that means the all the combination of these two vectors can cover the whole plane.

Take another eample of 3 equations and 3 unknowns.

$$ \begin{cases} 2&x&-&y& &=0 \\ -&x+&2&y-&z &=-1\\ &&-&y+4&z &=4 \end{cases} $$

Let’s transform it to the matrix form.

$$ A = \left[ \begin{matrix} 2&-1&0 \\ -1&2&-1\\ 0&-1&4 \end{matrix} \right] , b = \left[ \begin{matrix} 0 \\ -1\\ 4 \end{matrix} \right] $$

In the row picture, each equation in the above system determine a plane in three-dimention space. And, two of them determine a line in 3D space.

For the column picture, it’s easy to get the solution is ${ x=0,y=0,z=1 }$.

Question: Can I solve ${ A\boldsymbol{x}=b }$ for every ${ b }$?

OR the question is: Do the linear combinations of the columns fill tree dimentional space?

In this case, the answer is absolutely “yes”! But in some case, like the three columns of ${ A }$ lies in one plane of 3D space. We are in trouble.

Matrix form

Coming back to the formula ${ Ax =b }$

Let’s see how a matrix times a vector.

Take a example, the following method treat ${ Ax }$ as a combination of columns of ${ A }$

$$ \left[ \begin{matrix} 2 & 5 \\ 1 & 3 \\ \end{matrix} \right] \left[ \begin{matrix} 1 \\ 2 \\ \end{matrix} \right] = 1\left[ \begin{matrix} 2 \\ 1 \\ \end{matrix} \right] + 2\left[ \begin{matrix} 5 \\ 3 \\ \end{matrix} \right] =\left[ \begin{matrix} 12 \\ 7 \\ \end{matrix} \right] $$

OR, we can do it by dot production

$$ \left[ \begin{matrix} 2 & 5 \\ 1 & 3 \\ \end{matrix} \right] \left[ \begin{matrix} 1 \\ 2 \\ \end{matrix} \right] =\left[ \begin{matrix} \left[ \begin{matrix} 2 & 5 \end{matrix} \right] \left[ \begin{matrix} 1 \\ 2 \end{matrix} \right]\\ \left[ \begin{matrix} 1 & 3 \end{matrix} \right] \left[ \begin{matrix} 1 \\ 2 \end{matrix} \right]\\ \end{matrix} \right] = \left[ \begin{matrix} 2\times 1 + 5\times 2 \\ 1 \times 1 + 3 \times 2 \\ \end{matrix} \right] $$