Oko's Visionary Approach to Pedestrian Navigation
Enhancing Mobility for the Visually Impaired with AI and Accessibility Innovation
The Winner of the Apple Design Award 2024
In early June this year, Apple unveiled the winners of the 2024 Apple Design Awards, celebrating 14 outstanding apps and games. Among these remarkable achievements is Oko, the recipient of the Inclusivity award. This category recognises apps that offer exceptional experiences for all, supporting individuals from diverse backgrounds, abilities, and languages. Oko is an app specifically crafted for visually impaired individuals, focusing on enhancing pedestrian navigation through the utilisation of Apple technologies such as VoiceOver and Dynamic Type.
Addressing the Core Issue
When considering pedestrian navigation for visually impaired persons, the immediate concern often revolves around routing. Typically, the primary challenge is guiding individuals accurately from point A to point B. This typically leads to technical considerations involving path networks, GPS, and related aspects.
However, Oko takes a unique approach by concentrating on a crucial aspect of the pedestrian navigation journey—the interpretation of pedestrian signal lights.
The journey began a few years ago with the founders—Michiel Janssen, Vincent Janssen, and Willem Van de Mierop—computer science graduates with expertise in AI, who were inspired by a family friend who is visually impaired. This friend shared the difficulties faced while crossing streets.
We all know that crossing streets safely entails paying attention to pedestrian signals present at most intersections—the walk and stop signals. These signals indicate when pedestrians should wait and when it's safe to cross the road.
For visually impaired individuals, Accessible Pedestrian Signals (APS) play a vital role. APS devices provide auditory cues to visually impaired pedestrians, indicating when they have the right-of-way to cross at a signalised intersection and in which direction they can move.
APS devices are installed on poles at sidewalk corners near crosswalks. The device plays a low locator tone to help pedestrians locate the APS. Each APS has a button with a raised arrow pointing in the direction of the crosswalk. Audible tones (sometimes with voice message) are used to indicate the direction in which the pedestrian has the right-of-way.
Despite being a seemingly effective solution, users encounter unexpected challenges.
The installation of APS presents various issues, as highlighted in a report by the US Access Board in 2009, listing 15 common problems. While the concept is promising, improper implementation poses hurdles for users. For instance, poles with APS placed too far from the crossroad or at an inconvenient height can hinder accessibility; APS placed wrongly pointing the wrong crossroad directions during fixes after it being accidentally knocked down. Moreover, the process of installing APS demands time and resources, leading to delays in implementation. These challenges are not confined to a specific region but represent a global trend.
Exploring Alternative Solutions
The team behind Oko delved into the possibility of leveraging AI for aiding visually impaired individuals in crossing streets. By utilising the rear camera of the user's iPhone, they devised a system that translates visual inputs from standard pedestrian signals into alternative forms of feedback (audio, haptic, or visual) with the assistance of AI and Apple's neural engine.
“That’s what the app does, it does one thing and did it very well. Oko is not a replacement of cane or guide dog skills as they are still very important. Oko simply translate the visual signal to other feedback form.”
— Seth Wilson, Product Manager at Oko, suffer from Optic Nerve Hypoplasia
With AI's support, the app doesn't require an internet connection for signal translation, filters out irrelevant images, and eliminates the need for city infrastructure permissions.
The app simplifies the process—users hold the phone against their chest with the camera facing the street, rotate to align their body, receive the feedback, and confidently navigate the crossing.
“Mobility is the biggest barrier of living an independent life.” — Michiel Janssen, Co-founder at Oko
Looking Ahead
Building on the AI framework, Oko aims to enhance its capabilities by incorporating GPS and computer vision to improve routing. By recognizing additional visual signals, such as bus numbers and destinations, the app intends to provide users with more comprehensive feedback.
Oko also plans to foster a community where users can report unusual situations along routes, like construction sites. This collaborative effort will facilitate the sharing of photos of pedestrian crossings to enhance the app's recognition capabilities and potentially introduce new features.
The potential of Oko to empower visually impaired users in enhancing their mobility and independence is truly inspiring, and I eagerly looking forward witnessing its continued evolution.
Reference