UNIVERSITY PARK, Pa. — A team of researchers at Penn State has developed a new smartphone application designed to assist visually impaired individuals in navigating their surroundings. Named NaviSense, the tool utilizes artificial intelligence (AI) to identify objects based on spoken prompts, providing real-time guidance through the phone’s audio and vibrational features. The development of NaviSense reflects both the advancements in assistive technology and the ongoing needs of the visually impaired community.
NaviSense was presented at the Association for Computing Machinery’s SIGACCESS ASSETS ’25 conference in Denver, held from October 26 to 29, where it received the Best Audience Choice Poster Award. The researchers’ findings were published in the conference proceedings, underscoring the application’s innovative approach to enhancing the user experience compared to existing visual aid options.
According to Vijaykrishnan Narayanan, a professor of electrical engineering and the lead investigator of the NaviSense project, traditional visual-aid programs often require users to connect with in-person support teams. This can be inefficient and may raise privacy concerns. While some automated services exist, he noted a significant drawback: “Previously, models of objects needed to be preloaded into the service’s memory to be recognized. This is highly inefficient and gives users much less flexibility when using these tools.”
The NaviSense application addresses these limitations by integrating large-language models (LLMs) and vision-language models (VLMs), which are advanced AI technologies capable of processing extensive data to respond to user inquiries. By connecting to an external server that hosts these models, NaviSense can dynamically learn about its environment and identify items without the need for prior object loading. “Using VLMs and LLMs, NaviSense can recognize objects in its environment in real-time based on voice commands, without needing to preload models of objects,” Narayanan explained, marking a significant advancement in assistive technology.
Prior to developing the tool, the research team conducted a series of interviews with visually impaired individuals to better understand their specific challenges. Ajay Narayanan Sridhar, a doctoral student in computer engineering and the lead student investigator on the NaviSense project, emphasized the importance of this feedback, stating, “These interviews gave us a good sense of the actual challenges visually impaired people face.”
NaviSense operates by searching an environment for a requested object, efficiently filtering out items that do not match the user’s verbal request. In cases where the application requires clarification, it engages users with follow-up questions to narrow down the search parameters. Sridhar noted that this conversational capability offers a level of convenience and flexibility that many existing tools lack.
The development of NaviSense illustrates the potential for AI to transform assistive technologies, providing visually impaired users with tools that are not only more effective but also more responsive to individual needs. As this technology progresses, it could set new standards in the realm of accessibility, enabling greater independence for individuals with visual impairments.
Looking ahead, the advancements presented by NaviSense may pave the way for further innovations in assistive technologies, potentially inspiring similar applications that leverage AI to address a wide range of accessibility challenges. This progress highlights the crucial intersection of technology and inclusivity, reaffirming the commitment to enhancing the quality of life for all individuals, regardless of their visual capabilities.
Blackbaud Unveils AI Innovations and Intelligent Workflows in Latest Product Update
OpenAI Launches ChatGPT Shopping Assistant to Enhance Product Discovery and Recommendations
AI Transforms Health Care Workflows, Elevating Patient Care and Outcomes
Tamil Nadu’s Anbil Mahesh Seeks Exemption for In-Service Teachers from TET Requirements
Top AI Note-Taking Apps of 2026: Boost Productivity with 95% Accurate Transcriptions
