One of the options to solve the data entry could be:
- good quality automatic scanning of receipts (not only individual barcodes) from shops using OCR possibly supported with image recognition for double-checking (can happen that products will be mis-labelled or without quantities, etc)
- when ordering on-line, the receipt should be available, so should be also much easier
Yet, not always one will have a meaningful receipt available...
Solved the data entry and being able to predict own's supply needs would be also great to have a up-to date management of the inventory. Here are even more challenges on the tracking of the available goods at home, where these are and how many items (and in what state, expiration date, etc) would require most probably implementing different solutions from IoT (connected cameras, sensors, etc.).
Then, having a connected home with own groceries supplies under control, one can then automate further the shopping process with feeding-back the information about own's demand to on-line groceries one is subscribed to. This can enable customer subscription plans, and for retailer keeping a possible continuous flow of goods. This could be really really useful especially for upcoming months, when it seems like we are expected to spend a bit more time at home rather than usual, hopefully not fighting in the local shops for the last rolls of the new white paper gold.
$(".icon-button-children").each((index, item) => ($(item).click())) // expand all items
$(".LinesEllipsis ").each((index, item) => console.log(item.innerText)) // record the stuff
The other big challenge I never resolved was how you'd account for e.g. using 1/4 cup of flour out of a bigger volume. Or taken to its extreme, cooking oil. How do you know how much your 'splash' is? You can't predict the remaining volume without a lot of fiddling to measure it and that defeats the purpose.
In the end, I opted for manual databases too, but they're pain to keep up-to-date. I still think there's a lot of value in a database for all-or-nothing style ingredients, but it was enough to deter me. I'm glad someone is less lazy.
Definitely. Or, even, just taking some snacks to another room, and later placing it back to an 'intelligent' cupboard, let's say. In extreme cases, one may need be under a constant tracking at home.
But, simplifying, another option may be to use an improved voice assistant when cooking. One would need then to interact with the VA to note and confirm the stuff used. Improved VA, in essence, would try to 'understand' the activities and interact, instead of being a passive one (as most of currently are), that one needs to say commands. This would require adding it access to camera and maybe other sensors available at 'smart' home. Definitely a lot of interesting and challenging problems to tackle not forgetting too about the user privacy part.
In the household setting, the most realistic system are we r would be to keep track of the addition (purchase/replenishment) ołl the containers/packages of each product and subtraction when the entire container is used-up and discarded.
Obviously, this system would require maintaining at least one spare container for each product and reorder replacement when the remaining inventory count is 1 or whatever number of stock of a specific product one would want to maintain.
Partial expenditures (splashes or table spoons of cooking oil, pinch of silt, a number of eggs from the pack, cups of sugar or flour from the container, a serving of cereal from box etc.) of the product from the container wouldn’t be tracked. The inventory would only be updated when the container/package has been fished up or purchased and added to the home inventory.
For those items which one wishes not stock more than one container or packaging, a solution would be to label such containers, so that you can re-order the product (via Alexa or Google Assistant or Siri) when the remaining amount of the product is below certain level, depending on the product. I.e., you wouldn’t stock more than one container of milk or eggs. But when you got below half full, you would know to reorder.
If certain products are used exclusively by the kids, they should learn to talk to Alexa or let parents know that they would soon run out of the product (they would learn if they want to have the product available).
If one would follow this system of continuous replenishment, it might be unnecessary to keep the track of the available stock/inventory.
But updating of the inventory database could be done automatically, using designated commands to the voice assists which invoke IFTT engine action sequence in which the product is ordered or added to the weekly order or shopping list and updated in the inventory database as being ordered. Further, the delivery of the orders could be automatically detected by the IFTT email analysis extension, but for accessing the order in the notification and updating the inventory of items delivered in a given order, one would need to write a bit of the code, for each of the suppliers, i.e., Amazon, Jet, Target, Chewy etc.
The high turnover items with no reserve (fresh groceries) do not justify keeping track of their inventory, just remember to reorder them or add to the shopping list and complete the recurring task of doing grocery shopping, of which the voice assistants and personal devices could be reminding you about.
I think the key is the meal planning. The recipes need to bulk manage the inventories and list as meals are prepared and upcoming. The recipes would need to be cleaned up and normalized.
My solution would be to read frames from a smartphone's camera until a barcode is detected. This can be achieved with eg Firebase ML, on-device. If a barcode database lookup gives a product, put it on the in/out list. If not, send the frame to a product recognition vision service. This could be Google Cloud Vision AI, but they don't give you access to their product set that backs Google Lens.
Finally, provide controls to adjust the number of items on the in/out list
I thought about OCRing supermarket receipts too but these differ so greatly in layout etc. per country I figured it's not the way to go
Edit: Problem you run into as not all of this can be done on-device is privacy concerns of coursem just my thoughts on what interface I'd like to have
I just tested Product Open Data using the barcode from a can of Coconut Milk from my cupboard, and it returned the correct product (including a photograph) and the name of the brand and their owner.
Edit: updated to add Open Food Facts, given that Open Product Data appears no-longer-maintained
Do you know of any alternative data sources?
Edit: since writing these comments, I've learned that Product Open Data may be unmaintained.
 - https://discuss.okfn.org/t/can-we-resurrect-product-open-dat...
I did try Open Food Facts which correctly identified the same product for me -- and allows anyone to enter missing products, a bit like Wikipedia.
User /u/homarp mentions an app which can integrate Open Food Facts with Grocy.
 - https://world.openfoodfacts.org/
 - https://news.ycombinator.com/item?id=22648741
People usually shop at the same places and the same things, so they could identify the products in the receipts once and then only update the new ones. Also,You could crowdsource it.
It turns out that with an Amazon Prime account and shopping at Whole Foods the entire history is available down the the SKU, quantity, and date. Discovered this when going to order a delivery for the first time in the COVID-19 outbreak - aside from the generic shopping selection, we can also pick from our own previous purchases.
I haven't checked if there are other methods to access the data (e.g., with the app, or some history list), but there's the potential for at least some screen-scraping, and maybe they'll make the history available in a downloadable file if we pester them? It'd certainly help both this app and making their stores a bit 'stickier'.
On the other end of the system, I'm hoping to implement a way to bulk load recipes into grocy, using the open source recipe-scrapers library.
 - https://github.com/hhursev/recipe-scrapers
Also, costs would simply explode making the whole solution a non-starter for a lot of households.