In this problem you will perform block matching motion estimation between two consecutive video frames.
Follow the instructions below to complete this problem. (1) Download the two video frames from frame_1 and frame_2.
The frames/images are of height 288 and width 352. (2) Load the frame with file name "frame_1.jpg" into a 288×352 MATLAB
array using function "imread", and then convert the array type from 8-bit integer to real number using function "double" or "cast" (note that the range of intensity values after conversion is between 0 and 255). Denote by I1 the
converted MATLAB array. Repeat this step for the frame with file name "frame_2.jpg" and denote the resulting MATLAB array by I2 .
In this problem, I2 corresponds
to the current frame, and I1 corresponds
to the previous frame (i.e., the reference frame). (3) Consider the 32×32 target
block in I2 that
has its upper-left corner at (65,81) and
lower-right corner at (96,112) .
Note this is MATLAB coordinate convention, i.e., the first number between the parenthesis is the row index extending from 1 to 288 and
the second number is the column index extending from 1 to 352 .
This target block is therefore a 32×32 sub-array
of I2 .
(4) Denote the target block by Btarget .
Motion estimation via block matching searches for the 32×32 sub-array
of I1 that
is "most similar" to Btarget .
Recall in the video lectures we have introduced various forms of matching criteria, e.g., correlation coefficient, mean-squared-error (MSE), mean-absolute-error (MAE), etc. In this problem, we use MAE as the matching criterion. Given two blocks B1 and B2 both
of size M×N ,
the MAE is defined as MAE(B1,B2)=1M×N∑Mi=1∑Nj=1|B1(i,j)−B2(i,j)| .
To find the block in I1 that
is most similar to Btarget in
the MAE sense, you will need to scan through all the 32×32 blocks
in I1 ,
compute the MAE between each of these blocks and Btarget ,
and find the one that yields the smallest value of MAE. Note in practice motion search is only performed over a certain region of the reference frame, but for the sake of simplicity, we perform motion search over the entire reference frame I1 in
this problem. When you find the matched block in I1 ,
enter the following information: (1) the coordinate of the upper-left corner of the matched block in MATLAB convention. This requires two integer numbers; (2) the corresponding MAE value, which is a floating-point number. Enter the last number to two decimal
points. As an example for format of answer, suppose the matched block has upper-left corner located at (1,1)
, and the corresponding MAE is 10.12, then you should enter 1 1 10.12 (the three numbers are separated by spaces)
clear;
clear all;
clear clc;
I1 = double(imread('frame1.jpg'));
I2 = double(imread('frame2.jpg'));
[height,width] = size(I1);
Btarget = I2(65:96,81:112);
MAE = 1000000;
x = 0;
y = 0;
for i=1:height-31
for j=1:width-31
diff = Btarget - I1(i:i+31,j:j+31);
tsum = sum(sum(abs(diff)));
if(tsum < MAE)
x = i;
y = j;
MAE = tsum;
end
end
end
MAE = MAE/(32*32);