When Amazon Go was announced, many of us wondered how the service actually works. Here's the teaser from the Amazon Go page:
Our checkout-free shopping experience is made possible by the same types of technologies used in self-driving cars: computer vision, sensor fusion, and deep learning. Our Just Walk Out Technology automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your Amazon account and send you a receipt.
This solution has it all: identity management, object recognition, theft prevention, and, of course, commerce. Haomiao Huang, writing for Ars Technica, has an educated guess on what's happening:
As you enter the store, you swipe in. Cameras running deep learning vision algorithms see you and track you as you move about the store. Each time you pick up an item or put it down, cameras recognize that action. Observations from multiple cameras are combined using Bayes’ Rule, delivering a reading of what you took. All the while, the system tracks all the possible combinations of items that you might have in your possession. Every time you pass by a door or a gate, an RFID scanner takes a reading, which lets the system narrow down the list. When you finally leave, the system can look through what it thinks you might have, pick the most probable guess, and charge you accordingly.
All of this is made possible by the advent of deep learning, cloud computing, and probabilistic reasoning. Amazon Go is something that couldn’t have happened even five years ago.