It's mobile, informative, and smart.
ever found yourself in the supermarket staring at a shelf full of different
cereal boxes, wishing someone could just point out the one with the best price,
lowest sugar content, and the best reviews? New solutions for smarter retail will
soon give customers the same type of information they get online when
researching or comparing products – delivered inside the store as they shop.
A new augmented reality mobile shopping app being developed
by researcher scientists at IBM’s lab in Haifa, Israel, is about to change the
way we shop in stores. When shoppers use their smart phone or tablet video camera
to pan over products on the shelf, the application will instantly display recommendations and offers based on their specific
"We're going way beyond simple facial recognition
for products to provide superimposed information that points out the products
shopper prefer – whether based on previous purchases, price, consumer rating,
sodium content, environmentally friendly packaging, or other
considerations," said Amnon Ribak, project leader for the augmented
For example, a shopper
looking for a high-quality facial moisturizer can specify important
characteristics of the product, such as having a sun protection factor (SPF) of
15, is hypoallergenic, was not tested on animals, and that is on sale.
As the shopper points a
smart device camera at the shelf of moisturizers in the pharmacy, the app
recognizes the merchandise and displays information on the device’s screen,
superimposed on the product images. It also highlights information based on the
shopper's stated preferences, and can offer coupons or special discounts that
"The idea of
standing in an aisle in the supermarket and having your mobile device point out
the gluten-free cookies you need can be a real time saver," Ribak said.
"This has the potential to completely change the shopping experience from
one of hunting, reading, and searching to simply picking up those products you
To develop this new technology, Ribak leads a team of
research scientists whose image processing expertise has already contributed
advances in such areas as license plate recognition at night or inclement
weather for the Swedish toll road system; identification of malignant tumors in
medical images; and optical recognition for ancient texts in libraries across
Product recognition doesn't require bar codes or RFID tags. It is done using a blend of different approaches, including image recognition for the packaging that matches the colors and shapes to a database collection, optical character recognition techniques, as well as on-shelf context and positioning.
The researchers wrote algorithms that combine techniques used in facial recognition, color and shape matching, and associations with surrounding products. The app can take into account the mobile device’s camera
angle, and distance from a shelf to help distinguish between products.
"Our first mission was to create our own
mini-supermarket in the lab, so we could test the various approaches and
challenges involved," Ribak said. "We've already submitted a number
of patent applications based on the new techniques we discovered to overcome
the challenges in recognizing products with less than ideal lighting, shadows,
Aside from assisting shoppers, the new application
could also help retail managers organize their stores, instantly point out
what's missing on the shelves, or summarize which products were on sale during
the week. As more and more shoppers use the application, it will give retailers
deeper insight into how their customers shop – which is an analytics goldmine
for optimized shelf and store arrangement.
Read more about IBM's e-commerce technology on the Smarter Planet blog.
Labels: mobile, retail, shopping, virtual reality