Overview

Accurate estimation of food nutrition plays a vital role in promoting healthy dietary habits and personalized diet management. However, most existing food datasets primarily focus on Western cuisines and often rely on depth sensors, limiting their applicability in real-world scenarios.

To address these challenges, we introduce OmniFood8K, a large-scale multimodal dataset containing 8,036 real-world food scenes with detailed nutritional annotations and multi-view images. Our dataset significantly expands coverage of Chinese cuisine and supports practical RGB-only nutrition estimation.

Dataset Overview

Overview of the OmniFood8K dataset: data collection process and category distribution.

Dataset Highlights

Additionally, we construct NutritionSynth-115K, a large-scale synthetic dataset introducing compositional variations while preserving accurate nutritional annotations, enabling robust model training.

Method

We propose an end-to-end framework for predicting nutritional information from a single RGB image.

• Scale-Shift Residual Adapter (SSRA): Refines predicted depth maps to enforce global scale consistency and preserve local structural details.
• Frequency-Aligned Fusion Module (FAFM): Hierarchically aligns RGB and depth features in the frequency domain.
• Mask-based Prediction Head (MPH): Dynamically emphasizes key ingredient regions for improved prediction accuracy.

Experiments

Extensive experiments across OmniFood8K and Nutrition5K datasets demonstrate that our method outperforms existing approaches, providing a practical solution for daily dietary assessment.