Cascade Mesh-RCNN: Depth Map Estimation Using Fully Convolutional Residual Network

Wong, Yung Siang (2021) Cascade Mesh-RCNN: Depth Map Estimation Using Fully Convolutional Residual Network. Final Year Project (Bachelor), Tunku Abdul Rahman University College.

Text Wong Yung Siang.pdf Restricted to Registered users only Download (4MB)

Abstract

For deep learning-based object reconstruction, the model does not perform well in reconstructing objects with high texture levels. This is because the input image doesn’t include enough information for the ground truth. Therefore, the fine details of the object are badly reconstructed in the output 3D representation. Besides, the size of the object reconstructed in the 3D grid does not correspond to the true metric size of the object’s actual size in the real world. This is due to the depth data is lost in the 2D image. There is a way to estimate the object size by given the focal length and the object distance from the camera. However, it is difficult to provide the distance data due to the inaccurate image metadata provided in the database. In this thesis, the proposed method is cascading a depth estimation network into the Mesh-RCNN and predict the corresponding depth map when given an input image. Meanwhile, the predicted object mask from the mask prediction branch can be used as a mask array to mask the depth map. After the masked process, the remaining values in the depth map are then averaged to get the mean distance of the object from the image view. In a nutshell, the depth prediction network is cascaded into the Mesh-RCNN and able to predict the depth map of the given image.