matrix rank and linear dependence

definition

the rank of a matrix is the maximum number of independent vectors within any axes.

example

$$\left[\begin{array}{ccc}2& 4& 7\\ 1& 2& 5\\ 3& 6& 1\end{array}\right]$$

here $c_2=2c_1$, rank is $r=2$.

express a column as a linear weighted combination (LWC) of another

$c_1$ as a LWC of $c_2$ and $c_3$

$$\begin{array}{ccc}4b+7c=2& \to & \mathbf{b}=\frac{1}{2}\\ 2b+5c=1& \to & \mathbf{c}=0\\ 6b+c=3& \to & 3=3\end{array}$$

then $c_1$

$$\begin{array}{ccccc}4b& & 7c& & 2\\ 2b& +& 5c& =& 1\\ 6b& & 1c& & 3\end{array}$$

$c_2$ as a LWC of $c_1$ and $c_3$

$$\begin{array}{ccc}2a+7c=4& \to & \mathbf{c}=0\\ 1a+5c=2& \to & \mathbf{a}=2\\ 3a+c=6& \to & 6=6\end{array}$$

then $c_2$

$$\begin{array}{ccccc}2a& & 7c& & 4\\ 1a& +& 5c& =& 2\\ 3a& & 1c& & 6\end{array}$$

$c_3$ as a LWC of $c_1$ and $c_2$

$$\begin{array}{ccc}2a+4b=7& \to & b=\frac{7}{4}-\frac{1}{2}a\\ a+2b=5& \to & \frac{7}{2}=5\\ 3a+6b=1& \to & \frac{21}{2}=1\end{array}$$

$c_1$ and $c_2$ are on the same line, they are collinear, thus they can never be combined to obtain $c_3$

$$\begin{array}{ccccc}2a& & 4b& & 7\\ 1a& +& 2b& \ne & 5\\ 3a& & 6b& & 1\end{array}\forall c_1,c_2\in R$$

a row as a linear weighted combination (LWC) of another

$r_1$ as a LWC of $r_2$ and $r_3$

$$\begin{array}{ccc}b+3c=2& \to & \mathbf{b}=\frac{19}{14}\\ 2b+6c=4& \to & 4=4\\ 5b+c=7& \to & \mathbf{c}=\frac{3}{14}\end{array}$$

then $r_1$

$$\begin{array}{ccc}b& 2b& 5b\\ & +& \\ 3c& 6c& c\\ & =& \\ 2& 1& 3\end{array}$$

$r_2$ as a LWC of $r_1$ and $r_3$

$$\begin{array}{ccc}2a+3c=1& \to & \mathbf{a}=\frac{14}{19}\\ 4a+6c=2& \to & 2=2\\ 7a+c=5& \to & \mathbf{c}=-\frac{3}{19}\end{array}$$

then $r_2$

$$\begin{array}{ccc}2a& 4a& 7a\\ & +& \\ 3c& 6c& c\\ & =& \\ 1& 2& 5\end{array}$$

$r_3$ as a LWC of $r_1$ and $r_2$

$$\begin{array}{ccc}2a+b=3& \to & \mathbf{b}=-\frac{19}{3}\\ 4a+2b=6& \to & 6=6\\ 7a+5b=1& \to & \mathbf{a}=-\frac{14}{3}\end{array}$$

then $r_3$

$$\begin{array}{ccc}2a& 4a& 7a\\ & +& \\ 1b& 2b& 5b\\ & =& \\ 3& 6& 1\end{array}$$

what is happening?

dependency—as rank of which it is the complement—is a property of the whole set of vectors that constitute a matrix.

the column set contains two collinear columns, and their relationship is apparent—even more because in the example it’s an integer relationship, whereas in the rows set the dependency is more subtle, because it is distributed between the three row vectors.

in the column set one column can not be expressed as a combination of the other two, which being collinear can move only within the line they both span and can be reciprocally be instead expressed as a multiple of the other;

in the row set there’s no single row that can be expressed as a multiple of another, but all three can in turn be expressed by a combination of the other two.

the dependency is thus an inherent property of the matrix that can express itself as various relationships between the row and column vectors.