Abstract
This paper presents a new algorithm for enforcing temporal coherence on depth estimation from multi-view videos of dynamic scenes as well as the first substantial quantitative evaluation of the improvement in depth estimation accuracy due to temporal coherence. The proposed algorithm is generally applicable and practical since it bypasses explicit scene flow estimation, which has a very large state space, and relies only on optical flow which is used to impose soft constraints on depth estimation for the next frame. As a result, our algorithm is applicable to scenes with large depth and motion ranges. The output is a sequence of depth maps that can be used for novel view synthesis among other applications. While it is intuitive that enforcing temporal coherence should improve the accuracy of depth estimation, this improvement has never been assessed quantitatively due to the lack of data with ground truth. To overcome this limitation we use the image prediction error as the criterion and show that the benefits of temporal coherence are significant on a diverse set of multi-view video sequences.