Stitch two key frames
Take two images (frame 270 and 450) as examples.</p>
To stitch two overlapping video frames, first identify keypoints in both images, and match those keypoints.
| Key Points | Key Points Matched |
|---|---|
 |
 |
Then, compute the homography, a projective transformation, and project one frame to another by maketform and imtransform, one image to another. For example, after the transformation, the red square on the left should match the red square on the right.
Square in Frame 270 |
Square in Frame 450 |
|---|---|
 |
 |
For simplicity, just replace zero pixels in one image with pixels from another image, without a better blending function.
Panorama using five key frames
Use the same techniques on reference frames 90, 270, 450, 630, 810, to project them onto frame 450. In particular, to map frame far from frame 450, like 90, compute one project to a nearer frame 270 first, to minimize the error.
Map the video to the reference plane
Use the same techniques for all 900 frames, and use 5 reference frames as imtermediate reference. Use imageFolder2mpeg.m to output a movie.
I have tried to compute the homography between every consecute frames, but the result is not as good as this one.
Create background panorama
To remove moving objects from the video and create a background panorama, first detect all background pixels in each frame by finding the median value pixel. Use the median pixel value for each pixel of the panorama.
In order to find the median ignoring black pixels, changing all zeros into NaN, using matlab function nanmedian and changing them back to zeros give the desired result. I use three channels separately to avoid finding medians for matrices with too many dimensions, but the trade-off is that there are unexpected colors near the border.
Create background movie
Apply the reversed transformation in Part 3 on the background paranorma to obtain a movie like the original one, but with background only.
Create foreground movie
Foreground pixels are those differ from background ones. Use the same technique in Part 4 and 5 to obtain a foreground movie.
Generate a wide video
Basically the same as Part 5, but with a wider width.