Food Calorie Estimation from Photos using CNN

Diet-logging apps fail when every meal must be typed and portion-guessed by hand; photo-based estimation removes that friction. The task splits into two coupled problems: recognizing the dish, and estimating its energy content. The Food-101 dataset (Bossard, Guillaumin and Van Gool, ETH Zurich) gives 101,000 labeled food images across 101 dishes — the standard classification benchmark — while calorie values come from a curated per-dish nutrition table. A shared convolutional backbone with two heads (classification + regression) learns visual features useful to both tasks at once. The honest engineering lives in the error handling: the demo shows dish confidence next to the calorie number, states the measured error band, and never presents the estimate as medical advice.

1. The meal photo is resized to 224x224 and normalized for the EfficientNet-B0 backbone.
2. The backbone extracts multi-scale food features — texture, color, shape.
3. Head A (Dense(101) + softmax, cross-entropy) predicts the dish; head B (Dense(1) + ReLU, MSE) predicts kcal per 100 g.
4. Joint training uses a weighted loss: 0.6 × classification + 0.4 × regression.
5. The kcal/100 g output is scaled by the dish reference weight and the user's portion multiplier.
6. The nutrition table supplies protein/carbs/fat for the predicted dish; the demo renders the full breakdown.

Tech stack:

• Python 3.10, TensorFlow/Keras or PyTorch
• EfficientNet-B0 (ImageNet pre-trained)
• Food-101 dataset (ETH Zurich)
• Curated per-dish nutrition table
• Flask (demo backend)
• HTML/CSS/JavaScript (demo UI)
• Git

Dataset & Model Details

• Dataset: Food-101 — 101,000 images, 101 dish classes (75,750 train / 25,250 test); source: L. Bossard et al., ETH Zurich. Calorie values from a curated per-dish nutrition table joined by class label; the regression head trains on portion-annotated samples.
• Model input: 224x224x3 image. Outputs: 101-way dish softmax + single kcal/100 g regression value.
• Architecture: EfficientNet-B0 → Head A Dense(101)+softmax → Head B Dense(1)+ReLU; weighted joint loss.
• Metric: dish top-1 accuracy + calorie MAE on the test split, measured after training. Design target: MAE within ±15%; the shipped report states the measured error.

• [Two-headed architecture] One EfficientNet-B0 backbone feeds a 101-class dish classifier and a calorie regression head, trained with a weighted joint loss.
• [Portion-aware estimates] The user picks 0.5x/1x/2x servings; the regression head's per-100-g output scales to the serving.
• [Macro breakdown] Protein, carbs and fat computed from the per-dish nutrition table for the predicted dish.
• [Confidence-transparent UI] Dish confidence displayed next to the calorie figure, so uncertain classifications are visible, not hidden.
• [Curated nutrition table] Per-100-g values for all 101 dishes ships with the project and is editable.
• [Measured error band] Calorie MAE reported from the held-out test split — the report states the band plainly.
• [Viva-ready documentation] Report, PPT and Q&A covering multi-task learning, regression metrics and honest failure modes.

• Python training notebook (two-head training, joint loss, evaluation)
• Curated per-dish nutrition table (editable)
• Trained model weights
• Test-split evaluation: top-1 accuracy + calorie MAE from the actual run
• Web demo (photo upload, portion picker, macro breakdown)
• Project report PDF (methodology, error analysis, limitations)
• PPT presentation for final review
• Viva Q&A preparation document

• Estimates are approximate: hidden oil, sauces and true portion size are the biggest error sources.
• Mixed plates (thali with 5 items) are read as the dominant dish; multi-dish segmentation is future scope.
• This is a diet-logging aid, explicitly not medical or clinical nutrition advice.
• Dishes outside the 101 Food-101 classes get the closest match, which may be wrong — confidence display mitigates this.
• Fine-tuning benefits from a GPU; CPU training is possible but slow on 101k images.