Motion Estimation Using Convolutional Neural Networks (Figures)

First Frame 0 Ground Truth 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 Second Frame 0 0 500 100 200 300 400 500 Horn Schunck, EPE = 5.14 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlowNetC, EPE = 3.48 0 50 0 FlowNetS, EPE = 3.6 0 0 100 200 Figure 13: Motion estimation results for an example test image 300 400 500 First Frame 0 Ground truth, Prediction MSE = 906.6 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Initial Prediction MSE = 1545.9 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 944.3 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 926.7 0 50 0 FlownNetS, Prediction MSE = 1143.4 0 0 100 Figure 14: MCPE results for an example test image 200 300 400 500 First Frame 0 Ground Truth 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 Second Frame 0 0 500 100 200 300 400 500 Horn Schunck, EPE = 4.75 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlowNetC, EPE = 3.76 0 50 0 FlowNetS, EPE = 4.14 0 0 100 200 Figure 15: Motion estimation results for an example test image 300 400 500 First Frame 0 Ground truth, Prediction MSE = 936.5 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Initial Prediction MSE = 1750.7 0 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 1262.3 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 1238.9 0 50 0 FlownNetS, Prediction MSE = 1453.3 0 0 100 Figure 16: MCPE results for an example test image 200 300 400 500 First Frame 0 50 50 100 100 150 150 200 200 250 250 300 300 350 350 0 100 200 300 400 500 Second Frame 0 0 Horn Schunck 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 100 400 500 200 300 400 500 400 500 FlowNetC 0 50 0 FlowNetS 0 0 100 200 300 Figure 17: Motion estimation results for Miss America - neighboring frames First Frame 0 Initial Prediction MSE = 30.5 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 Second Frame 0 0 500 100 200 300 400 500 Horn Schunck, Prediction MSE = 5.9 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 38.1 0 50 0 FlownNetS, Prediction MSE = 4.1 0 0 100 200 Figure 18: MCPE results for Miss America - neighboring frames 300 400 500 First Frame 0 50 50 100 100 150 150 200 200 250 250 300 300 350 350 0 100 200 300 400 500 Second Frame 0 0 Horn Schunck 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 100 400 500 200 300 400 500 400 500 FlowNetC 0 50 0 FlowNetS 0 0 100 200 300 Figure 19: Motion estimation results for Miss America - 4 frames apart First Frame 0 Initial Prediction MSE = 183.4 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 Second Frame 0 0 500 100 200 300 400 500 Horn Schunck, Prediction MSE = 51.2 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 39.2 0 50 0 FlownNetS, Prediction MSE = 39.1 0 0 100 Figure 20: MCPE results for Miss America - 4 frames apart 200 300 400 500 First Frame 0 Initial Prediction MSE = 205.7 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 Second Frame 0 0 500 100 200 300 400 500 Horn Schunck, Prediction MSE = 75.1 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 487.0 0 50 0 FlownNetS, Prediction MSE = 87.3 0 0 100 200 Figure 21: MCPE results for an example video - neighboring frames 300 400 500 First Frame 0 Initial Prediction MSE = 726.2 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Second Frame 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 314.1 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 361.5 0 50 0 FlownNetS, Prediction MSE = 348.9 0 0 100 200 Figure 22: MCPE results for an example video - 5 frames apart 300 400 500 First Frame 0 Initial Prediction MSE = 1173.2 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Second Frame 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 546.6 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 472.4 0 50 0 FlownNetS, Prediction MSE = 622.9 0 0 100 200 Figure 23: MCPE results for an example video - 10 frames apart 300 400 500 First Frame 0 Initial Prediction MSE = 1350.1 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Second Frame 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 792.2 0 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 609.0 0 50 0 FlownNetS, Prediction MSE = 728.9 0 0 100 200 Figure 24: MCPE results for an example video - 30 frames apart 300 400 500 First Frame 0 Initial Prediction MSE = 2288.0 0 50 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 0 100 200 300 400 500 Second Frame 0 0 0 100 200 300 400 500 Horn Schunck, Prediction MSE = 1350.2 0 50 50 100 100 100 150 150 150 200 200 200 250 250 250 300 300 300 350 350 350 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 FlownNetC, Prediction MSE = 1258.8 0 50 0 FlownNetS, Prediction MSE = 1567.1 0 0 100 200 Figure 25: MCPE results for an example video - 50 frames apart 300 400 500