This morning, the eagerly anticipated AiPoly app was made available in the iOS app store. This app uses artificial intelligence to identify objects through the iPhone’s camera. While several object recognition services exist already, AiPoly uses a live video stream to capture data, and processes images on the phone itself, rather than sending data to a web based service. This means that object recognition can happen fairly quickly, and that you can point the phone at various objects to get a constantly updating stream of information, rather than having to take individual photos. The app’s creators warn against using it as a mobility tool. In other words, don’t try to replace your white cane or dog guide with this app. It should be used to explore the environment in combination with existing skills and tools.
In its first iteration, this app shows a great deal of potential for future development. Being able to scan a room or outdoor space with the phone’s camera and hear about nearby objects could be immensely useful. The app has little to no learning curve for the user, just press a button and start pointing and listening. However, in our first day tests, it is worth noting that the artificial intelligence used here is an emerging technology with lots of room to grow.
The app often misidentified objects, calling a hand a foot, a wall a refrigerator, and not knowing what to call a sink or a trash can. A restroom sign was identified only as “Shape.”
However, the app also called attention to objects around the office. “Calendar” was announced, and I reached out to find that indeed, a bulletin board with posted documents was immediately to my left.
The app allows users to teach the AI interface by typing descriptions of objects. As its database grows, this could become a remarkable assistive technology. Imagine if the app could recognize a restroom or exit sign, a nearby trash can, or an interesting attraction along one’s path of travel.
We will continue to test AiPoly here at ILRC, and because the app is free, you can download it and experiment along with us.