3D reconstruction from monocular images

Demonstration of a workflow that reconstructs real-world-scale 3D food meshes and scene geometry from single 2D meal images to estimate portion and volume.

Overview

A workflow that takes in 2D pictures of a meal and creates real-world accurate 3D meshes and a scene. Why?

3D reconstruction from monocular images is a rapidly evolving research area in computer vision with significant applications in food image analysis. The ability to reconstruct 3D food models from single 2D eating occasion images in real-world physical units allows users to share food experiences in three dimensions and provides crucial information about food portions, facilitating the tracking of individual nutrition intake. However, 3D reconstruction presents unique challenges that make it particularly valuable to evaluate the robustness and capability of existing computer vision algorithms.

Tech stack