Who is working on AR for blind people

Question

After watching the video on facial recognition using ML + Meta glasses I started wondering if we could use LLMs + Vision models to help blind people augment their surroundings.While I admittedly have no experience with the blind or the methods taught to them that assist them in navigating the world, I have a hunch there's potential to leverage the current LLM/ AI stack to improve upon those methods.Are their any cool companies or open source projects experimenting with this?

JieJie · Accepted Answer

My friend uses Oko on iOS to tell her when the walk light is on, and when the numbers are counting down. There is also a navigation aid that she doesn't quite feel comfortable using yet, but I will say the Oko app being able to tell her (quite precisely) what is going on with the stoplight has been a big confidence booster for her getting out into the busy world.
https://www.ayes.ai
The leader in the field is BeMyEyes, of course. They've been working with Microsoft to integrate GPT-4o vision models into their app, with some great success. What we haven't seen yet is the move to live-video image recognition that could come from something like an OrCam or Meta glasses (they recently announced a partnership with Meta). I'm guessing there are serious safety issues with the model missing important information and leading someone vulnerable astray.
https://www.bemyeyes.com https://www.bemyeyes.com/blog/be-my-eyes-meta-accessibility-...
OrCam has a new product (woe upon those of us who have the paltry OrCam MyEye2) that the Meta glasses will be competing against at an eye-watering > $4K price point, that seems to do less.
https://www.orcam.com/en-us/orcam-myeye-3-pro
As with the hearing aid industry which recently went over-the-counter causing prices to plummet, the vision aid product category is in temporary disarray as inexpensive new technologies makes their way into a premium-price market.

dmofp · Answer

I love this question. There are a ton of AR and AI startups looking for a market / problem to apply their technology to. Meanwhile the market for visual aids & accessibility tools is large, starved of innovation and is willing to pay!AR + AI for visual aid / accessibility. Do it somebody.