误差反向传播 - 神经网络

Posted 2023-03-13

【中文标题】误差反向传播 - 神经网络

【英文标题】：Error Backpropagation - Neural network

【发布时间】：2013-12-23 18:01:33

【问题描述】：

我正在尝试为神经网络的错误反向传播编写代码，但我的代码需要很长时间才能执行。我知道神经网络的训练需要很长时间，但单次迭代也需要很长时间。

多类分类问题！

Total number of training set = 19978
Number of inputs = 513
Number of hidden units = 345
Number of classes = 10

下面是我的完整代码：

X=horzcat(ones(19978,1),inputMatrix);         %Adding bias
M=floor(0.66*(513+10));         %Taking two-third of imput+output

Wji=rand(513,M);
aj=X*Wji;

zj=tanh(aj);                      %Hidden Layer output

Wkj=rand(M,10);
ak=zj*Wkj; 

akTranspose = ak';
ykTranspose=softmax(akTranspose);      %For multi-class classification
yk=ykTranspose';                       %Final output

error=0;

%Initializing target variables
t = zeros(19978,10);
t(1:2000,1)=1;
t(2001:4000,2)=1;
t(4001:6000,3)=1;
t(6001:8000,4)=1;
t(8001:10000,5)=1;
t(10001:12000,6)=1;
t(12001:14000,7)=1;
t(14001:16000,8)=1;
t(16001:18000,9)=1;
t(18001:19778,10)=1;

errorArray=zeros(100000,1); %Stroing error values to keep track of error iteration 
errorDiff=zeros(100000,1);
for nIterations=1:5
    errorOld=error;

    aj=X*Wji;                %Forward propagating in each iteration
    zj=tanh(aj);

    ak=zj*Wkj;

    akTranspose = ak';
    ykTranspose=softmax(akTranspose);
    yk=ykTranspose';
    error=0;
    %Calculating error
    for n=1:19978                                      %for 19978 training samples
        for k=1:10                                      %for 10 classes
            error = error + t(n,k)*log(yk(n,k));         %using cross entropy function  
        end
    end
    error=-error;

    Ediff = error-errorOld;

    errorArray(nIterations,1)=error;     
    errorDiff(nIterations,1)=Ediff;

    %Calculating dervative of error wrt weights wji
    derEWji=zeros(513,345);
    derEWkj=zeros(345,10);

    for i=1:513
        for j=1:M;
            derErrorTemp=0;
            for k=1:10
                for n=1:19978
                    derErrorTemp=derErrorTemp+Wkj(j,k)*(yk(n,k)-t(n,k));
                    Calculating derivative of E wrt Wkj%
                    derEWkj(j,k) = derEWkj(j,k)+(yk(n,k)-t(n,k))*zj(n,j);
                end
            end
            for n=1:19978
                Calculating derivative of E wrt Wji
                derEWji(i,j) = derEWji(i,j)+(1-(zj(n,j)*zj(n,j)))*derErrorTemp;
            end
        end
    end


    eta = 0.0001;                    %learning rate

    Wji = Wji - eta.*derEWji;          %updating weights
    Wkj = Wkj - eta.*derEWkj;
end

【问题讨论】：

您是否尝试使用调试分析器查看哪个命令占用了大部分时间？ --> ***.com/questions/14381978/…

【参考方案1】：

即使在JIT 的帮助下，Matlab 中的 for-loop 也非常耗时。尝试通过矢量化它们来修改您的代码，而不是将它们组织成 3 循环甚至 4 循环。例如，

for n=1:19978                                      %for 19978 training samples
        for k=1:10                                      %for 10 classes
            error = error + t(n,k)*log(yk(n,k));         %using cross entropy function  
        end
end

可以改为：

error = sum(sum(t.*yk)); % t and yk are both n*k arrays that you construct

您可以尝试对其余代码执行类似的工作。针对不同情况对数组使用点积或乘法运算。