How AI Food Recognition Works

Introduction

When we tell people our system uses AI to identify food waste, the first question is usually: “How accurate is it?” It’s a fair question. But it’s also the wrong starting point for understanding what AI food recognition actually does and why it matters.

The real question isn’t “Is it perfect?” The real question is: “Does it give me data I can act on?” And the answer to that is unequivocally yes—even when the AI makes mistakes.

This guide explains how our AI food recognition works, why we’ve designed it the way we have, what its limitations are, and how you can get the most value from the data it produces.

Key Points

• Our AI recognises 2,400+ different food items
• We achieve approximately 80% accuracy across all deployments
• When the AI isn’t confident, items go to “Mixed Waste”
• You can flag corrections and add custom foods to improve accuracy

How AI Recognition Works

Every time someone throws food waste into a monitored bin, a sequence of events happens in milliseconds.

The Recognition Process

Step 1: Image Capture

The camera detects motion and captures an image the moment food enters the bin. The scale simultaneously records the weight change. This pairing of image + weight is fundamental to the system.

Step 2: Neural Network Analysis

The image is processed by a convolutional neural network trained on millions of food images. The model compares what it sees against its database of 2,400+ food categories, looking for visual patterns it recognises.

Step 3: Confidence Scoring

For each potential match, the model calculates a confidence score—essentially, how sure it is about the identification. This happens for every category in the database, resulting in a ranked list of possibilities.

Step 4: Classification Decision

If the top match exceeds our confidence threshold, the item is classified as that food. If no match is confident enough, the item goes to “Mixed Waste.” This threshold is calibrated to balance accuracy against data completeness.

The 2,400+ Food Database

Our recognition model is trained on a comprehensive database of over 2,400 food items commonly found in commercial kitchens:

• Raw ingredients: Vegetables, fruits, proteins, grains, dairy
• Prepared foods: Common dishes, sauces, sides, desserts
• Bakery items: Breads, pastries, cakes
• Beverages: Coffee grounds, tea, liquid waste
• Trimmings: Peels, stems, bones, fat

This is a global model—the same AI serves all our deployments. It’s trained on data from kitchens worldwide, which makes it robust across different cuisines and kitchen types.

The Mixed Waste Category

“Mixed Waste” is what the system assigns when it’s not confident enough to classify an item. Some people see high Mixed Waste and think the AI is failing. Actually, it’s doing exactly what it should: being honest about uncertainty.

The Alternative Would Be Worse

We could tune the system to always make a guess, even when it’s uncertain. The result? More “confident” classifications that are actually wrong. We’d rather show you honest uncertainty than false precision.

What Triggers Mixed Waste

• Low confidence scores: The AI’s top guess doesn’t meet the confidence threshold
• Multiple items at once: Several different foods thrown together in one disposal
• Unusual items: Foods not well-represented in the training data
• Poor visibility: Bad lighting, obstructed camera, or blurry images

Using Mixed Waste as a Diagnostic

Your Mixed Waste percentage is actually useful information:

Mixed Waste Level	Interpretation
Under 15%	Great conditions. Highly granular data.
15-30%	Normal range. Good category-level data.
Over 30%	Worth investigating. May indicate issues.

Common Mislabeling Scenarios

No AI system is perfect. Understanding where the model commonly struggles helps you interpret your data correctly.

Visually Similar Foods

The AI sees what the camera sees—it can’t taste or smell:

• Rice vs couscous vs quinoa
• Chicken vs pork vs turkey
• Different pasta shapes
• Sauces and liquids

Prepared vs Raw

The same ingredient looks very different raw, cooked, or mixed into a dish:

• Raw chicken breast vs grilled chicken vs chicken in a curry
• Fresh vegetables vs roasted vs stir-fried
• Individual ingredients vs combined in a plated dish

Environmental Factors

Lighting: Poor lighting changes how colours appear, making identification harder.

Camera cleanliness: Grease, steam residue, or debris on the camera lens degrades image quality.

Partial visibility: If food lands in a corner of the bin or is partially obscured, the AI has less visual information to work with.

Flagging & Corrections

When you spot a mislabeled item in your dashboard, you can flag it. This isn’t just correcting your own data—it helps improve the model for everyone using the system.

Your Corrections Matter

Every flag you submit becomes training data for the next model update. You’re directly contributing to improved accuracy—not just for your site, but for all sites.

How to Flag an Item

1. Find the item: Navigate to the waste log or detailed view
2. Click the flag icon: Select the flag or “report” option
3. Select the correct category: Choose from the list or search
4. Submit: Your correction updates immediately

When to Flag

Focus on:

• High-volume items: Corrections on frequently wasted foods have the biggest impact
• High-value items: Expensive proteins or specialty items worth tracking accurately
• Recurring errors: If the same item keeps getting misclassified, flag it

Adding Custom Foods

Our database covers 2,400+ common foods, but every kitchen has its specialties. Custom foods let you extend the system with items specific to your menu.

When to Add a Custom Food

• A signature dish specific to your restaurant
• Regional or ethnic specialties not in the global database
• Items that keep getting misclassified
• Specific preparations you want to track separately

Best Practices for Custom Foods

Use clear, specific names: “House Marinara Sauce” is better than “Sauce.”

Choose the right category: Assign to appropriate parent categories for correct grouping in reports.

Don’t over-customise: Too many hyper-specific items can make reporting harder to interpret.

Why 80% Accuracy Is Actually Good

When people hear “80% accurate,” some wonder if that’s good enough. Let’s put it in context.

The Alternative: Manual Logging

Before AI-powered systems, food waste tracking meant manual logging. In practice:

Metric	Manual Logging
Compliance rates	Often under 20%
Data quality	Inconsistent, error-prone
Staff burden	High (slows service)
Sustainability	Abandoned within months

The choice isn’t between 80% AI accuracy and 100% manual accuracy. It’s between 80% AI accuracy and effectively 0% usable data.

Trends Matter More Than Items

Individual item accuracy matters less than you might think. What matters is:

• Consistent measurement over time: Even with some errors, week-over-week trends are reliable
• Category-level accuracy: Even if “chicken” is sometimes “turkey,” you know your protein waste total
• Total waste volume: The scale never lies—you always know exactly how much went in the bin

Statistical Significance

At 80% accuracy across thousands of data points, the errors tend to average out. If the AI occasionally calls rice “couscous,” it also occasionally calls couscous “rice.” Over time, your category totals converge on reality.

Best Practices for Better Accuracy

Optimise Lighting

The AI needs good light to see clearly. Ensure the bin area is well-lit without harsh shadows or glare.

Keep the Camera Clean

Kitchen environments produce steam and grease. Wipe the camera lens periodically with a soft, clean cloth.

One Item at a Time (When Possible)

The AI identifies best when it sees one type of food at a time. Dumping mixed plate scrapings in one go makes identification harder.

Add Custom Foods for Your Menu

If you have signature items or regional specialties, add them as custom foods for more accurate tracking.

Flag Recurring Errors

If you notice the same item repeatedly getting misclassified, flag it to help retrain the model.

Reading Reports with AI in Mind

Understanding how the AI works helps you interpret your waste reports correctly.

Trust Aggregates Over Individual Items

Your weekly summary of “Protein: 45kg” is more reliable than any individual “2.3kg chicken” entry.

Focus on Trends

The most valuable insights come from week-over-week or month-over-month comparisons:

• Is total waste going up or down?
• Are certain categories consistently high?
• Did an intervention have an effect?
• Are there patterns by day of week or time of day?

Use Mixed Waste as a Signal

If your Mixed Waste percentage suddenly increases:

• Check the camera lens—may need cleaning
• Check lighting—bulb may have changed or failed
• Check for new menu items that might need to be added as custom foods

Don’t Over-Interpret Outliers

If you see a single unusual entry, check the image before drawing conclusions. It might be a mislabel worth flagging, or it might be a one-off event.

Sources & References

1. Deep Learning for Food Image Recognition - arXiv, Cornell University
2. Food Recognition: A New Dataset, Experiments, and Results - IEEE
3. The State of AI in Food Service - McKinsey & Company
4. Computer Vision in the Food Industry - Trends in Food Science & Technology
5. Human-in-the-Loop Machine Learning - Stanford HAI