Skip to Main content Skip to Navigation
Journal articles

A Lightweight Neural Network for Monocular View Generation with Occlusion Handling

Simon Evain 1, 2 Christine Guillemot 1, 2
2 Sirocco - Analysis representation, compression and communication of visual data
Inria Rennes – Bretagne Atlantique , IRISA-D5 - SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE
Abstract : In this article, we present a very lightweight neural network architecture, trained on stereo data pairs, which performs view synthesis from one single image. With the growing success of multi-view formats, this problem is indeed increasingly relevant. The network returns a prediction built from disparity estimation, which fills in wrongly predicted regions using a occlusion handling technique. To do so, during training, the network learns to estimate the left-right consistency structural constraint on the pair of stereo input images, to be able to replicate it at test time from one single image. The method is built upon the idea of blending two predictions: a prediction based on disparity estimation, and a prediction based on direct minimization in occluded regions. The network is also able to identify these occluded areas at training and at test time by checking the pixelwise left-right consistency of the produced disparity maps. At test time, the approach can thus generate a left-side and a right-side view from one input image, as well as a depth map and a pixelwise confidence measure in the prediction. The work outperforms visually and metric-wise state-of-the-art approaches on the challenging KITTI dataset, all while reducing by a very significant order of magnitude (5 or 10 times) the required number of parameters (6.5 M).
Document type :
Journal articles
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-02428602
Contributor : Christine Guillemot <>
Submitted on : Monday, May 17, 2021 - 11:04:10 AM
Last modification on : Wednesday, May 19, 2021 - 3:10:55 AM

File

A_Lightweight_Neural_Network_f...
Files produced by the author(s)

Identifiers

Citation

Simon Evain, Christine Guillemot. A Lightweight Neural Network for Monocular View Generation with Occlusion Handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, inPress, pp.1-14. ⟨10.1109/TPAMI.2019.2960689⟩. ⟨hal-02428602v2⟩

Share

Metrics

Record views

26

Files downloads

16