Amazon Update Alexa to Next level AI (Show and Tell) recognition Object

Devices

"Alexa, what am I holding?"

Show and Tell is an Alexa feature designed to help blind and low vision customers identify grocery items.

Alexa-enabled devices are already a big help around the house for Grijalva, from changing the channel on her Fire TV to using her voice to turn off lights and appliances. Now, an Echo Show feature mak easier to identify household pantry items.

With Show and Tell, blind and low vision customers can hold up an item to the Echo Show camera and ask, “Alexa, what am I holding,” and Alexa helps identify the item through advanced computer vision and machine learning technologies for object recognition.

“It’s a tremendous help and a huge time saver because the Echo Show just sits on my counter, and I don’t have to go and find another tool or person to help me identify something. I can do it on my own by just asking Alexa,” Grijalva said.

Amazon believes in starting from the customer and working backward, a philosophy also applied to invent and using our technology resources for good. That means paying attention to what all of our customers are telling us.

“The whole idea for Show and Tell came about from feedback from the blind and low vision customers,” said Sarah Caplener, head of Amazon’s Alexa for Everyone team. “We heard that product identification can be a challenge and something customers wanted Alexa’s help with. Whether a customer is sorting through a bag of groceries or trying to determine what item was left out on the counter, we want to make those moments simpler by helping identify these items and giving customers the information they need in that moment.”

From early research and exploration through product development and testing, Caplener’s team collaborated with the Vista Center for the Blind and Visually Impaired in Santa Cruz, California, where Grijalva is the assistive technology manager. For Show and Tell, Grijalva enlisted other blind and low vision customers for user studies, providing feedback to the Alexa for Everyone team. “By working together, we create an amazing product for everyone,” said Grijalva.