Solving Linear Equation

Posted 2022-04-01 sweetzxl

1. Gauss Elimination Method

\((1\times 1)\) \(ax_1=b\) \(\rightarrow\) \(x_1=\fracba\)
\((2 \times 2)\) \(\begincasesa_11x_1+a_12x_2=b_1 \\ a_21x_1+a_22x_2=b_2\endcases\)

Ex1:
\[ \begincasesx_1+2x_2=5 \3x_1+4x_2=6\endcases \rightarrow x_1=-4,x_2=4.5\]

\((n \times n)\)
\[ \begincasesa_11x_1+a_12x_2+\cdots+a_1nx_n=b_1 \a_21x_1+a_22x_2+\cdots+a_2nx_n=b_2\\vdots\a_n1x_1+a_n2x_2+\cdots+a_nnx_n=b_n\\endcases\]

Matix
\[A=\beginbmatrix a_11 & a_12 & \cdots & a_1n\a_21 & a_22 & \cdots & a_2n\\vdots\a_11 & a_12 & \cdots & a_nn \endbmatrix, x=\beginbmatrixx_1 \cdots x_n\endbmatrix, b=\beginbmatrixb_1 \cdots b_n\endbmatrix, Ax=b\]

Matrix form extracts the must essential information .

Matrix form of Ex1:
\[\beginbmatrix1 & 2 \\3 & 4\endbmatrix \beginbmatrixx_1 \\ x_2 \endbmatrix=\beginbmatrix5\\6\endbmatrix\]

Augment matrix
\[\beginbmatrix1 & 2 & 5 \\3 & 4 & 6\endbmatrix \rightarrow \beginbmatrix1 & 2 & 5 \\0 & -2 & -9\endbmatrix \rightarrow \beginbmatrix1 & 2 & 5 \\0 & 1 & 4.5\endbmatrix \rightarrow \beginbmatrix1 & 0 & -4 \\0 & 1 & 4.5\endbmatrix \]

Systematic method of Solving for Ex1:
\[ \begincasesx_1+2x_2=5 \3x_1+4x_2=6\endcases \rightarrow \begincasesx_1+2x_2=5 \0-2x_2=-9\endcases \rightarrow \begincasesx_1+2x_2=5 \x_2=4.5\endcases\]

Know as Gauss elimination Method：

• write as in matrix form
• Use row operations: multiply all elements in a row by the s number, subtract (add) row from another row (These operations do not change the solution)
• Apply row-by-row to bring the matrix to a form which is easy to solve

Matlab: To solve \(Ax=b\), using \(x=A\backslash b\)

>> A = [1 2;3 4]

A =

     1     2
     3     4

>> b = [5 6]'

b =

     5
     6

>> x = A\b

x =

   -4.0000
    4.5000

>> format long
>> x = A\b

x =

  -3.999999999999999
   4.499999999999999

2. Direct Methods and Iterative Methods

Bring to an easily solved form:

• Diagonal: \(\beginbmatrix1 & & 0\\ & \ddots & \\0 & & 1 \endbmatrix\)
• Upper triangular: \(\beginbmatrix a_11 & a_12 & \cdots & a_1n\\ & & a_22 & \cdots & a_2n\\ \vdots\\ & & & \cdots & a_nn\\ \endbmatrix\)
• Lower triangular :

For upper triangular (back substitution):

\[a_nnx_n=b_n \rightarrow x_n = \fracb_na_nn\] \[a_n-1,n-1x_n-1+ a_n-1,nx_n=b_n-1\] repeat... \[x_i=\fracb_i-\sum^n_j=i+1a_ijx_ja_ii\]

For lower triangular (forward substitution):

\[a_11x_1=b_1 \rightarrow x_1 = \fracb_1a_11\] \[a_2,1x_1+ a_2,2x_2=b_2\] repeat... \[x_i=\fracb_i-\sum^i-1_j=1a_ijx_ja_ii\]

General Methods:

• Gauss Elimination
• Pivoting
• Gauss-Jordan
• LU-factorization

Starting from 1st, a suitable multiple of the current row (pivot row) is subtracted form all subsequent rows, the result is an upper triangular

Ex2: \[\begincases2x_1+x_2-x_3=1\\x_1+2x_2+x_3=8\\-x_1+x_2-x_3=-5\endcases\]

matrix:
\[\beginbmatrix2 & 1 & -1 & 1\\ 1 &2 & 1 & 8\\ -1 & 1 & -1 & -5 \endbmatrix \rightarrow \beginbmatrix2 & 1 & -1 & 1\\ 0 &1.5 & 1.5 & 7.5\\ 0 & 1.5 & -1.5 & -4.5 \endbmatrix\]

\[\rightarrow \beginbmatrix2 & 1 & -1 & 1\\ 0 &1.5 & 1.5 & 7.5\\ 0 & 0 & -3 & -12\endbmatrix \rightarrow \begincases-3x_2 = -12\\x_3=4\endcases \rightarrow \begincasesx_1 =2 \\x_2=1\\x_3=4\endcases\]

It is a problem if this element is zero. If a pivot is small number, this may lead to loss of accuracy.

Exchange rows when necessary:
\[\beginbmatrix 1 & 2 & 5\\ 3 & 4& 5 \endbmatrix, \] \[\beginbmatrix 0 & 2 & 5\\ 3 & 4& 5 \endbmatrix, \]

Pivoting is the procedure of rearranying rows to make sure Pivot is not zero and as large as possible in magnitude.

Ex3: \[ \begincases0.0003x_1 + 12.34x_2=12.343 \\0.4321 x_1+x_2=5.321\endcases\] Using Gauss:

• \(\frac0.43210.0003=1440\) (round-off error, since \(0.0003*1440 = 0.4320\))
• \[ \begincases0.0003x_1 + 12.34x_2=12.343 \\ 0.0001 x_1+17770x_2=-17760\endcases \rightarrow x_2 = -\frac1776017770=0.9994\] \[ \rightarrow x_1 = \frac12.343-12.34*0.99940.0003=33.33\]

3. Gauss-Jordan Method (Applicable for calculating inverses)

\[\beginbmatrixa_11 & a_12 & \cdots & a_1n& b_1\\ \vdots & & & \vdots\\ a_n1 & a_n2 & \cdots & a_nn & b_n\endbmatrix\]
normalized and pivot row by dividing by the
normalize by the pivot; subtract the pivot row to elimate all non-zero in the column.

Ex1. \[\beginbmatrix4 & 1 & 2 & 21\\ 2 & -2 & 2 & 8\\ 1 & -2 & 4 & 16\endbmatrix \rightarrow \beginbmatrix1 & -2 & 4 & 16\\ 2 & -2 & 2 & 8 \\4 & 1 & 2 & 21 \endbmatrix \rightarrow \beginbmatrix1 & -2 & 4 & 16\\ 0 & 2 & -6 & -24 \\0 & 9 & -14 & -43 \endbmatrix\] \[ \rightarrow \beginbmatrix1 & -2 & 4 & 16\\ 0 & 1 & -3 & -12 \\0 & 9 & -14 & -43 \endbmatrix \rightarrow \beginbmatrix1 & 0 & -2 & -8\\ 0 & 1 & -3 & -12 \\0 & 0 & 13 & 65 \endbmatrix \rightarrow \beginbmatrix1 & 0 & -2 & -8\\ 0 & 1 & -3 & -12 \\0 & 0 & 1 & 5 \endbmatrix\] \[ \rightarrow \beginbmatrix1 & 0 & 0 & 2\\ 0 & 1 & 0 & 3 \\0 & 0 & 1 & 5 \endbmatrix\]

Suitable for matrix inverse

\(AB=I\), \(A\beginbmatrixb_1 & b_2 & b_3 \endbmatrix \rightarrow \beginbmatrixa_11 & a_12 & a_13 & 1 & 0 & 0\\ a_21 & a_22 & a_23 & 0 & 1 & 0\\ a_31 & a_32 & a_33 & 0 & 0 & 1\endbmatrix \rightarrow \beginbmatrix1 & 0 & 0 &c_11 & c_12 & c_13 & \\ 0 & 1 & 0 & c_21 & c_22 & c_23 &\\ 0 & 0 & 1& c_31 & c_32 & c_33 \endbmatrix\)

4. LU-decomposition Method

\(A=LU\), where \(L\) means lower triangular, \(U\) means upper triangular.

If this is achieved, then, solve \(Ax=b\) is equivalent to solve \(LUx=b\)
Work in 2 steps: \(Ux=y,Ly=b\), solve \(Ly=b\) then \(Ux=y\).

Two approaches:

• Gauss Elimination;
• Crout‘s Method

1) Gauss Elimination:

\[\beginbmatrix a_11 & a_12 & \cdots & a_1n\a_21 & a_22 & \cdots & a_2n\\vdots\a_11 & a_12 & \cdots & a_nn \endbmatrix=\beginbmatrix1 & & & & \\ m_21 & 1 & & & \\ m_31 & m_32 & 1 & &\endbmatrix \beginbmatrixa'_11 & a'_12 & \cdots & a'_1n \\ & a'_22 & \cdots & a'_2n\endbmatrix\]

general formulation: \(m_ij=\fraca'_ija'_jj\)

Ex4: \(\beginbmatrix1 & 2\\3 & 4\endbmatrix=\beginbmatrix1 & 0\\3 & 1\endbmatrix\beginbmatrix1 & 2\\0 & -2\endbmatrix\)

2) Crout‘s Method

\[\beginbmatrix a_11 & a_12 & \cdots & a_1n\a_21 & a_22 & \cdots & a_2n\\vdots\a_n1 & a_n2 & \cdots & a_nn \endbmatrix=\beginbmatrixL_11 & & & & \\ L_21 & L_22 & & & \\ \vdots & \vdots & \ddots & \endbmatrix \beginbmatrix1 & U_12 & U_13 & \cdots & U_1n \\ & 1 & U_23 & \cdots &U_2n\\ & & \ddots & & \vdots \\ & & & & 1\endbmatrix \]

Ex5:

\[\beginbmatrix a_11 & a_12 & a_13\a_21 & a_22 & a_23\a_31 & a_32 & a_33 \endbmatrix=\beginbmatrix L_11 & 0 & 0\L_21 & L_22 & 0\L_31 & L_32 & L_33 \endbmatrix\beginbmatrix 1 & U_12 & U_13\0 &1 & U_23\0 & 0 & 1 \endbmatrix=\beginbmatrix L_11 & L_11U_12 & L_11U_13\L_21 & L_21U_12+L_22 & L_21U_13+ L_22U_23\L_31 & L_31U_12+L_32 & L_33+ L_31U_13+ L_32U_23 \endbmatrix\]