VolE: A point-cloud framework for food 3D reconstruction and volume estimation.

Accurate food volume estimation is crucial for medical nutrition management and health monitoring applications. However, existing methods for estimating food volume are often constrained by the use of monocular data. They typically rely on specialised hardware such as 3D scanners, gather sensor-specific information such as depth data, or depend on camera calibration with reference objects. In this paper, we present VolE, a novel framework that leverages mobile device-driven 3D reconstruction to estimate food volume. VolE captures images and camera locations in free motion to generate precise 3D models, thanks to AR-capable mobile devices. To achieve real-world measurement, VolE is a reference- and depth-free framework that leverages food video segmentation for food mask generation. We also introduce a new food dataset encompassing the challenging scenarios absent in the previous benchmarks. Our experiments demonstrate that VolE outperforms the existing volume estimation techniques across multiple datasets by achieving 2.22% MAPE, highlighting its superior performance in food volume estimation. The source code is available at https://umairharon.github.io/VolE .