使用卡尔曼滤波器进行背景减法和前景检测

Posted 2023-02-21

【中文标题】使用卡尔曼滤波器进行背景减法和前景检测

【英文标题】：Background subtraction And foreground detection using Kalman Filter

【发布时间】：2016-08-02 21:55:08

【问题描述】：

我需要使用卡尔曼滤波器将视频中的背景与前景分开。有人可以给我一些资源或代码示例吗？

更新：我在这里找到了一个很好的例子Traffic detection。它非常适合流量检测，但我想重新调整它以用于人员提取。我发现了一些需要调整的变量，例如：

1. alpha = learning constant
2. K= #of guassians in the mixture
3. T = min. portion of background
4. initVariance = initial variance
5. pixelThresh = threshold condition for computing adaptive process on pixel

这里是主文件的概述（如果您只是想要一个概述）

---

function foregroundEstimation(movie)

%Load video into memory and prepare output video for writing

v = VideoReader(movie);
numFrames = 150;
foregroundVideo = VideoWriter('foreground.avi');
open(foregroundVideo);

%video constants

initFrame = read(v,1);
global height;
global width;
height = size(initFrame,1); 
width = size(initFrame,2);



%initialize GMM parameters: (based upon Stauffer notation)
%http://www.ai.mit.edu/projects/vsam/Publications/stauffer_cvpr98_track.pdf

% alpha = learning constant
%  K= #of guassians in the mixture
% T = min. portion of background
% initVariance = initial variance
% pixelThresh = threshold condition for computing adaptive process on pixel

alpha=0.05;  % 0.05
K=5;         % 5
global T;    % T = 0.8
T=0.8;

global initVariance;
initVariance=75; % 75
pixelThresh=45; % 45
referenceDistance = 40; % 40 %shortcut to speed up processing time. Compare current pixel to pixel referenceDistance frames back and skip adaptive process if similar. Downside is doesn't collect background evidence as well.
sideWeight = (K-1)/(1-T);
global matchThres;
matchThres = sqrt(3);
global ccThreshold;
ccThreshold = 9000;  % 5000
global deltaT;
deltaT = 1;
global numParticles;
numParticles = 100;

trackingOn = 0; % will superimpose tracking color marker on detected vehicles in output video. tackingOn should equal 1 or 0

prevCentSize = 0;


%structures to pass information between frames for detection purposes.
field = 'f';
filterValue = [];[];;
prevFilter = struct(field,filterValue);
modelValue = [];prevFilter;
prevModel = struct(field,modelValue);

%initiailze video process components
initColorBoxes();


foreFrame = zeros(height,width,3);
backFrame = zeros(height,width,3);

%map represents pixels at a given frame to perform adaptive process
pixThreshMap = ones(height,width);


%individual pixel process components
pixel = zeros(3,1);
pixMean = zeros(3,1,K);
pixVar = ones(1,K);
pixWeight = zeros(1,K);

%global pixel process components
globalWeight = (ones(height,width,K)/sideWeight);
globalWeight(:,:,1) = T;
%globalWeight = (ones(height,width,K)/K);
globalMean = zeros(height,width,3,K);
globalVar = ones(height,width,K);




%=====================================================
%Extract Foreground and Background by K-mixture model
%=====================================================
%initialize g-mixture model
globalVar = globalVar*initVariance;

for k=1:K
globalMean(:,:,1,k)=initFrame(:,:,1);
globalMean(:,:,2,k)=initFrame(:,:,2);
globalMean(:,:,3,k)=initFrame(:,:,3);
end;


distVec = zeros(numFrames,1);

%adaptive g-mixture background segmentation
for frameIndex=2:numFrames
%get current frame and the refernece frame
%tic;
frameIndex
currentFrame = double(read(v,frameIndex));



if (frameIndex<=referenceDistance)
referenceFrame= double(read(v,1));
else
referenceFrame= double(read(v,frameIndex-referenceDistance));
end;

frameDifference = abs(currentFrame - referenceFrame);

%creates map of pixel locations where we will perform adaptive process. Based
%upon threshold that detects low change regions based on previous frame in
%order to save computation.
pixThreshMap = min(sum(+(frameDifference(:,:,:)>pixelThresh),3),1);



for index=1:3
  backFrame(:,:,index)=(+(pixThreshMap(:,:)==0)).*currentFrame(:,:,index);      
end;

%extract the parts considered "stable background" from current frame

%reset foreground frame
foreFrame = ones(height,width,3)*255;

%gaussian mixture matching & model updating



[i,j]=find(pixThreshMap(:,:)==1);


%loop through every pixel location where adaptive process should be performed
for k = 1:size(i,1)

    pixel = reshape(currentFrame(i(k),j(k),:),3,1);
    pixMean = reshape(globalMean(i(k),j(k),:,:),3,1,K);
    pixVar = reshape(globalVar(i(k),j(k),:,:),1,K);
    pixWeight=reshape(globalWeight(i(k),j(k),:),1,K);

    %update gaussian mixture according to new pix value
    match=matchingCriterion(pixMean,pixVar,pixel);
    matchWeight = 0;

    if(match>0)
        %match found so update weights/normalize
        pixWeight = (1-alpha)*pixWeight;
        pixWeight(match)= pixWeight(match) + alpha;
        pixWeight = pixWeight/sum(pixWeight);
        matchWeight = pixWeight(1,match);

        %NOTE ALPHA SHOULD BE REPACED WITH SOME KIND OF RHO EVENTUALLY
        %WHERE RHO IS PRODUCT OF ALPHA AND CONDITIONAL PROBABILITY MEASURE

        %update variance
        pixVar(:,match) = (1-alpha)*pixVar(:,match) + ...
        alpha*(pixel - pixMean(:,:,match))'*(pixel-pixMean(:,:,match));

        %update mean
        pixMean(:,:,match) = (1-alpha)*pixMean(:,:,match) + alpha*pixel;



    else
        %no match currently found.
        %replace one with lowest sigma value

        rankVector = pixWeight./sqrt(pixVar(1,:));
        [mini minIndex] = min(rankVector);

        pixMean(:,:,minIndex) = pixel;  
        pixVar(:,minIndex) = initVariance;  
    end

    %rerank all pixel components
    rankCriterionVector = pixWeight./sqrt(pixVar(1,1,:)); 
    [rankSort rankIndex] = sort(rankCriterionVector);

    pixMean = pixMean(rankIndex);
    pixVar = pixVar(rankIndex);
    pixWeight = pixWeight(rankIndex);


    %repopulate global structures with updated values
    globalWeight(i(k),j(k),:) = pixWeight(:,1);
    globalMean(i(k),j(k),:,:) = pixMean(:,1,:);
    globalVar(i(k),j(k),:,:) = pixVar(:,:,:);

    %now need to perform the background segmentation based upon weight
    %threshold

    bgIndex = segmentBackground(pixWeight);
    if(ismember(matchWeight, pixWeight))
        matchIndex = find(pixWeight == matchWeight,1);
    else
        matchIndex = 0;
    end

    if((matchIndex>=bgIndex) || (matchIndex == 0))

        %also check against initFrame for match
        %NOTE CHANGE
        if(initMatch(initFrame(i(k),j(k),:),pixel) == 0)
          foreFrame(i(k),j(k),:) = pixel;   
        end 

    end


end


%Now write foreground frame to foreground estimation video
contrastFrame = foreFrame/max(abs(foreFrame(:)));

%remove all connected components associated with foreground objects that are smaller than what we expect a vehicle should be.
[cleanFrame centroids]= connectedComponentCleanup(contrastFrame);

if(trackingOn == 1)
if(size(centroids,1) > prevCentSize)
   prevModel = addModel(prevModel, centroids, height, width);
elseif (size(centroids,1) < prevCentSize)
   prevModel = removeModel(prevModel, centroids, height, width);
end


if(size(centroids,1) > 0)
    %implies there is a car in frame for tracking
    [curModel orderedCentroids] = vehicleTracking(centroids, prevModel);
    prevModel = curModel;
    trackFrame = colorBox(cleanFrame, curModel,orderedCentroids, height, width);  
else
    trackFrame = cleanFrame;
end

else
   trackFrame = cleanFrame; 
end

writeVideo(foregroundVideo,trackFrame); 
prevCentSize = size(centroids,1);
end

---

谢谢。

【问题讨论】：

相关例子：- mathworks.com/help/vision/examples/…

【参考方案1】：

基本上这不是卡尔曼滤波器的任务，看看opencv中的背景减法。 之后可以使用卡尔曼滤波器来跟踪帧之间的对象，您可以在这里查看示例 http://studentdavestutorials.weebly.com/kalman-filter-with-matlab-code.html

【讨论】：

是的，我想使用卡尔曼滤波器来跟踪前景中的对象，因为我正在处理视频文件...

顺便说一句，在这种情况下，您不受卡尔曼滤波器的限制，实际上它是非常基本的方法。您可以寻找视频对象跟踪挑战来寻找高级挑战votchallenge.net/challenges.html