Linear Algebra
Vectors
Can be represented by,
( a b . . ) \begin{pmatrix} a\\b\\.\\. \end{pmatrix} ⎝ ⎜ ⎜ ⎜ ⎛ a b . . ⎠ ⎟ ⎟ ⎟ ⎞ or ai +bj +ck + ...
Linear Combination
v 1 , v 2 , v 3 , . . . , v n ∣ v n ∈ R n v_1,v_2,v_3,\ ...,v_n | v_n \in \R^n v 1 , v 2 , v 3 , . . . , v n ∣ v n ∈ R n
of two or more vectors is basically a summation of the vectors where each of them are multiplied with a scalar.
c 1 v 1 + c 2 v 2 + c 3 v 3 + . . . + c n v n ∣ c n ∈ R c_1v_1+c_2v_2+c_3v_3+\ ...+c_nv_n | c_n \in \R c 1 v 1 + c 2 v 2 + c 3 v 3 + . . . + c n v n ∣ c n ∈ R
Span
The span is the set of all the linear combination of two or more vectors.
e.g If we have two vectors where a → ≠ k b → \overrightarrow{a}\neq k\overrightarrow{b} a = k b where k ∈ R k\in\R k ∈ R (∴ \therefore ∴ the vectors are linearly independent ) then the s p a n ( a , b ) = R 2 span(a,b)=\R^2 s p a n ( a , b ) = R 2
s p a n ( a → ) span(\overrightarrow{a}) s p a n ( a ) is a line.
s p a n ( v 1 , v 2 , v 3 , . . . , v n ) = { c 1 v 1 + c 2 v 2 + c 3 v 3 + . . . + c n v n ∣ c i ∈ R f o r 1 ≤ i ≤ n } span(v_1,v_2,v_3,\ ...,v_n)=\{c_1v_1+c_2v_2+c_3v_3+\ ...+c_nv_n |c_i\in\R\ for\ 1\leq i\leq n\} s p a n ( v 1 , v 2 , v 3 , . . . , v n ) = { c 1 v 1 + c 2 v 2 + c 3 v 3 + . . . + c n v n ∣ c i ∈ R f o r 1 ≤ i ≤ n }
s p a n { ( 2 3 ) , ( 4 6 ) } = c 1 ( 2 3 ) + c 2 ( 4 6 ) = c 1 ( 2 3 ) + 2 c 2 ( 2 3 ) = ( c 1 + 2 c 2 ) ( 2 3 ) = c 3 ( 2 3 ) ( ∴ a l i n e ) \begin{aligned} span\bigg\{\begin{pmatrix}2\\3\end{pmatrix}, \begin{pmatrix}4\\6\end{pmatrix} \bigg\} &= c_1\begin{pmatrix}2\\3\end{pmatrix}+ c_2\begin{pmatrix}4\\6\end{pmatrix}\\ &=c_1\begin{pmatrix}2\\3\end{pmatrix}+ 2c_2\begin{pmatrix}2\\3\end{pmatrix}\\ &= (c_1+2c_2)\begin{pmatrix}2\\3\end{pmatrix}\\ &=c_3\begin{pmatrix}2\\3\end{pmatrix} (\therefore a\ line) \end{aligned} s p a n { ( 2 3 ) , ( 4 6 ) } = c 1 ( 2 3 ) + c 2 ( 4 6 ) = c 1 ( 2 3 ) + 2 c 2 ( 2 3 ) = ( c 1 + 2 c 2 ) ( 2 3 ) = c 3 ( 2 3 ) ( ∴ a l i n e )
To check if a set of vectors spans R 3 \R^3 R 3 ,
c 1 ( x 1 x 2 x 3 ) + c 2 ( y 1 y 2 y 3 ) + c 3 ( z 1 z 2 z 3 ) = ( a b c ) c_1 \begin{pmatrix} x_1\\x_2\\x_3 \end{pmatrix} +c_2 \begin{pmatrix} y_1\\y_2\\y_3 \end{pmatrix} +c_3 \begin{pmatrix} z_1\\z_2\\z_3 \end{pmatrix}= \begin{pmatrix} a\\b\\c \end{pmatrix} c 1 ⎝ ⎛ x 1 x 2 x 3 ⎠ ⎞ + c 2 ⎝ ⎛ y 1 y 2 y 3 ⎠ ⎞ + c 3 ⎝ ⎛ z 1 z 2 z 3 ⎠ ⎞ = ⎝ ⎛ a b c ⎠ ⎞
If ∄ \nexists ∄ a valid solution of c 1 , c 2 , c 3 c_1,\ c_2,\ c_3 c 1 , c 2 , c 3 for any given set of a, b and c values, then the set of vectors do not span R 3 \R^3 R 3
Inner Product
a = ( a 1 a 2 a 3 ) , b = ( b 1 b 2 b 3 ) a . b = a 1 b 1 + a 2 b 2 + a 3 b 3 \textbf{a} =\begin{pmatrix} a_1\\a_2\\a_3 \end{pmatrix} , \textbf{b} = \begin{pmatrix} b_1\\b_2\\b_3 \end{pmatrix}\\ \textbf{a}.\textbf{b}=a_1b_1+a_2b_2+a_3b_3 a = ⎝ ⎛ a 1 a 2 a 3 ⎠ ⎞ , b = ⎝ ⎛ b 1 b 2 b 3 ⎠ ⎞ a . b = a 1 b 1 + a 2 b 2 + a 3 b 3
Using matrix notation can also write it as,
a T b = a 1 b 1 + a 2 b 2 + a 3 b 3 \textbf{a}^T\textbf{b}=a_1b_1+a_2b_2+a_3b_3 a T b = a 1 b 1 + a 2 b 2 + a 3 b 3
where,
( a 1 a 2 a 3 ) ( b 1 b 2 b 3 ) \begin{pmatrix} a_1&a_2&a_3 \end{pmatrix} \begin{pmatrix} b_1\\b_2\\b_3 \end{pmatrix} ( a 1 a 2 a 3 ) ⎝ ⎛ b 1 b 2 b 3 ⎠ ⎞
Knowing all this, we can say,
a T a = a 1 2 + a 2 2 + a 3 2 ∴ a T a = ∣ ∣ a ∣ ∣ \textbf{a}^T\textbf{a} = a_1^2+a_2^2+a_3^2\\ \therefore\sqrt{\textbf{a}^T\textbf{a}} = ||\textbf{a}|| a T a = a 1 2 + a 2 2 + a 3 2 ∴ a T a = ∣ ∣ a ∣ ∣
Sum of all elements of a vector:
1 T a = a 1 + a 2 + a 3 \textbf{1}^T\textbf{a} = a_1+a_2+a_3 1 T a = a 1 + a 2 + a 3
Mean of vector:
( 1 n ) T a = a 1 + a 2 + a 3 n \left(\frac{1}{n}\right)^T\textbf{a} = \frac{a_1+a_2+a_3}{n} ( n 1 ) T a = n a 1 + a 2 + a 3
Inner product with a unit vector:
e i T a = a i e_i^T\textbf{a} = a_i e i T a = a i
Linear Independence
In a set of vectors, If any one vector can be represented as a linear combination of the others, then the set is linearly dependent.
To check for Linear Independence for a set of vectors,
c 1 ( x 1 x 2 x 3 ) + c 2 ( y 1 y 2 y 3 ) + c 3 ( z 1 z 2 z 3 ) = ( 0 0 0 ) c_1 \begin{pmatrix} x_1\\x_2\\x_3 \end{pmatrix} +c_2 \begin{pmatrix} y_1\\y_2\\y_3 \end{pmatrix} +c_3 \begin{pmatrix} z_1\\z_2\\z_3 \end{pmatrix}= \begin{pmatrix} 0\\0\\0 \end{pmatrix} c 1 ⎝ ⎛ x 1 x 2 x 3 ⎠ ⎞ + c 2 ⎝ ⎛ y 1 y 2 y 3 ⎠ ⎞ + c 3 ⎝ ⎛ z 1 z 2 z 3 ⎠ ⎞ = ⎝ ⎛ 0 0 0 ⎠ ⎞
If the only solution to this is not c 1 = c 2 = c 3 = 0 c_1=c_2=c_3=0 c 1 = c 2 = c 3 = 0 , then the set is not Linearly Independent
Linear Superposition
If, for a function, f ( α x + β y ) → α f ( x ) + β f ( y ) f(\alpha x+\beta y)\to \alpha f(x) + \beta f(y) f ( α x + β y ) → α f ( x ) + β f ( y ) then, the function is linear.
Affine Functions
Functions such that, f ( α 1 x 1 + α 2 x 2 + . . . + α n x n ) = f(\alpha_1x_1+\alpha_2x_2+...+\alpha_nx_n)= f ( α 1 x 1 + α 2 x 2 + . . . + α n x n ) = is linear, and α 1 + α 2 + . . . + α n = 1 \alpha_1 +\alpha_2+...+\alpha_n=1 α 1 + α 2 + . . . + α n = 1