幂迭代法求特征值和特征向量

Posted 2020-11-25 fahaizhong

幂迭代法求第k大的特征值和特征向量

数学表述

设矩阵A
[ A = left[ egin{matrix} X_{11}&ldots&X_{1n} ewline X_{21} & ldots& X_{2n} ewline &vdots& ewline X_{n1}&ldots &X_{nn} ewline end{matrix} ight] ]
求其最大特征值对应的特征向量(b_k = [v_1,v_2,ldots,v_n]^T):
[b_{k+1} = frac{Ab_k}{||Ab_k ||}..........(1)]

证明：见英文wiki Power_iteration

代码：

import numpy as np
def power_iteration(A, num_simulations: int):
    # Ideally choose a random vector
    # To decrease the chance that our vector
    # Is orthogonal to the eigenvector
    b_k = np.random.rand(A.shape[1])
  
    for _ in range(num_simulations):
        # calculate the matrix-by-vector product Ab
        b_k1 = np.dot(A, b_k)
 
        # calculate the norm
        b_k1_norm = np.linalg.norm(b_k1)
 
        # re normalize the vector
        b_k = b_k1 / b_k1_norm
 
    return b_k

例子

[ A = left[ egin{matrix} 2&1 ewline 1&2 ewline end{matrix} ight] ]

求其特征值：
[ |A-lambda I| = left| egin{matrix} 2-lambda &1 ewline 1 &2-lambda ewline end{matrix} ight| = 3 - 4lambda + lambda^2 ]
求得最大特征值(lambda_2 = 3)，另一特征值(lambda_1 = 1)
(lambda_2 = 3) 时：
[ |(A-lambda I)V| = 0 ]

[ (A-3I)V = left[ egin{array} {cccc} -1 &1 ewline 1 &-1 ewline end{array} ight] left[ egin{array} {cccc} v_1 ewline v_2 ewline end{array} ight] = left[ egin{array} {cccc} 0 ewline 0 ewline end{array} ight] ]

求得(v_1 = v_2),归一化后，(V_{lambda=3} = [0.707,0.707]^T)

验证代码：

A = np.array([[2,1],[1,2]])
V = power_iteration(A, 100).reshape(1,-1)
V

array([[0.70710678, 0.70710678]])

求得特征向量后，特征值
[ lambda = VAV^T........(2) ]
代码

lbd = np.dot(V,np.dot(A,V.T))
lbd 

array([[3.]])

扩展

求次大特征向量与特征值:
[ B = A - frac{lambda}{||V||^2}V^TV ........(3) ]

[ b_{k+1}' = frac{Bb_k'}{||Bb_k' ||} ...........(4) ]

代码

B = A - lbd / np.linalg.norm(V)**2 * np.dot(V.T,V)
V_2 = power_iteration(B, 100)
lbd1 = np.dot(V_2.T,np.dot(B,V_2))
lbd1

1.0000000000000002

以此类推，我们可以得到任意第k大特征值。这在很多机器学习方法中，取前k重要特征有着重要的作用。
欢迎访问个人博客。转载请注明出处。