Today: beacons on every wall and a survey team.
- Bluetooth beacons fixed around the building, each with a battery to replace
- A vendor visit to survey and calibrate before anyone can use it
- Often a separate app to install and an account to create
OpenCV AI Competition 2026 entry
Voice directions for blind visitors, in any building, after a 15-minute walk.
Two volunteers walk the floor with their phones. Then a blind or low-vision visitor's phone works out where it is, within centimetres on benchmark video, and speaks one step at a time. No beacons, nothing to install.
338Mpeople with blindness or serious vision loss
15 minto map a floor, with 2 phones and 0 beacons
1.4 cmcamera located on benchmark video of a small room
$4/dayruns a building on one AWS server
FreeRuns in the phone's browserUsed with a cane or guide dog
Why it matters
Inside a building, a blind visitor still has to find someone to ask.
GPS stops at the front door. Indoor guidance today needs beacons on the walls and a paid survey, so almost no library, clinic or college has it.
43 millionpeople are blind
295 millionmore have moderate or severe vision loss
15 minto map a floor with Waymark: 2 phones, 1 printed sheet, 0 beacons
Global estimates for 2020: Bourne et al., The Lancet Global Health, 2021.
Waymark: two phones and one sheet of paper.
- A printed A4 ChArUco board by the entrance gives every walk the same origin and real scale
- Two volunteers walk the floor for about 15 minutes and mark rooms, doors, stairs and lifts
- Visitors open a free web page, with nothing to install and no account
How it works
Map a floor once. Every blind visitor after that gets spoken directions.
All of the vision runs in OpenCV 5 on one AWS Graviton4 instance. The phones just send camera frames and read out the replies.
Mapping
1. Two volunteers map the floor in 15 minutes.
Both volunteers start at the board, so their walks share one origin and real metric scale from the first frame. The server refines each walk with bundle adjustment, fuses the 3D points they both saw, and draws a floor plan with a route graph.
- A coordinator splits the unmapped corridors between walkers and tells each one where to go
- Volunteers tap to mark rooms, doors, stairs, lifts and exits as they pass
- Routes follow paths people actually walked
ChArUco + solvePnPALIKED + LightGlueBundle adjustmentHoughLinesP floor plan
Locating
2. The visitor’s phone knows where it is, every second.
About once a second the visitor's phone sends one camera frame and its compass heading. The server finds a shortlist of nearby keyframes, matches features to the map's 3D points, and solves for position and heading.
- Median 1.77 cm and 1.98° on the 7-Scenes relocalisation protocol
- 352 ms per frame on one Graviton4 instance
- Replies come as speech and vibration and work with screen readers. Visitor frames are not stored
VLAD shortlistMAGSAC++ PnPLightGlue when weak
Guidance agent
3. When it isn’t sure, it stops the visitor and calls a person.
Every frame, the agent looks at how confident the fix is and decides what to say. Each decision is logged with what it saw and why it acted, and the helper console shows that log live.
- Asks the visitor to pan when the camera can't see enough, and to slow down when unsure
- After leaving the route three times, it re-routes with fewer turns
- Stops before stairs unless it is certain
- Flags corridors that no longer match the map, and calls a human helper when it can't place the visitor
Try it here
Hear what Waymark says when it isn’t sure where you are.
These controls run the same rules as the guidance agent on the server. Move the confidence down, put stairs ahead, or walk off the route.
What the guide tells the visitor
- Says
- Walk straight for 6 metres, then turn left.
- State
guiding- Haptic
straight- Why
- On route, confident fix.
Why it said that
- Below 0.35: ask for a slow pan
- 3 misses in a row: ask to stop walking
- 6 misses: hand over to a helper
- Off route: stop and re-route
- Stairs within 3 m and below 0.8: stop
- 0.35 to 0.6: say "Slowly." first
- Otherwise: the next instruction
Results
1.4 cm on replayed benchmark video of a small room, with 92% of frames located.
That is the median over a replayed visit through the live server (Microsoft 7-Scenes, a small room). A larger office room reached 16 cm. Every number here comes from a script in the repo. Mapping and relocalisation use the Microsoft 7-Scenes "heads" scene. Timing is the median of 15 runs on Graviton4 (c8g.xlarge).
1.2cm
Two volunteers’ maps line up to within 1.2 cm of each other after merging.
Worst keyframe 4.2 cm. Reprojection error drops from 28.4 px to 2.3 px. Only the first and last 10 frames of each walk get a reference pose, the same way the board anchors a real walk.
Median keyframe position error at each stage
Live tracking13.5 to 23.2 cm
Board-anchored bundle adjustment2.0 to 7.5 cm
Merging both walkers1.2 cm
Bars are scaled to the worst walker at each stage.
1.77cm
A phone finds where it is from one photo, with no hint, to within 1.77 cm.
within 5 cm, 5° 95%failed 0 / 100
HLoc publishes 1.0 cm on this scene and is still more accurate. Waymark gets close without a retrieval network.
352ms
from photo to spoken direction, on one AWS Graviton4 server.
ALIKED features take about 308 ms. Everything else, matching and PnP included, takes about 45 ms.
4.6mm
The map stays this accurate even when 15% of matches are wrong.
solve time 0.4 s
Tests with blind and low-vision visitors: results added today.
Blind and low-vision participants are walking real routes with a sighted spotter beside them and their own cane or dog. Arrival, checkpoint error and every agent decision are still being measured, so there are no numbers here yet. The numbers above come from replayed benchmark video.
Checking the latest evaluation run…
Architecture
The phone sends a photo. The server sends back one direction.
Phones do no vision work. They send a JPEG about once a second and speak the reply, so any recent phone browser can take part.
Client
Phone browser
Rear camera, compass, speech and vibration. Volunteers use the mapping page, visitors the guidance page.
Edge
API Gateway
HTTPS for the camera, then forwards to the worker with an origin secret.
Compute
Graviton4 c8g.xlarge running OpenCV 5
- ChArUco board pose with solvePnP
- ALIKED keypoints, LightGlue at keyframes
- Triangulation and bundle adjustment
- Merging walkers, floor plan, route graph
- VLAD shortlist and PnP to locate visitors
- Guidance agent with a traced decision per frame
DynamoDB
Decision traces
S3
Maps and walk frames
CloudWatch
Logs, dashboard, budget alarm
The stack is defined in AWS CDK and managed through SSM, with no SSH access. Full details are in the technical report.
Safety
It adds directions. The cane or guide dog still leads.
Waymark says so before every visit. The safety rules live in server code, and a visitor can't talk the agent out of them.
- It does not see obstacles. The map has walls, doors and stairs. People, trolleys and wet floors are for the cane or dog.
- It stops before stairs unless the fix is confident, and points the visitor to the handrail.
- It says when it's unsure and asks the visitor to pan, slow down or wait.
- A person is one tap away. Call a helper is always on screen, and the agent calls one itself after six frames without a fix.
- If the connection drops, the phone tells the visitor to stop and wait.
Questions
Does it replace a cane or guide dog? No. Other questions, answered.
How does a blind visitor open it and choose where to go?
Through the screen reader they already use every day: VoiceOver on iPhone or TalkBack on Android. Every control is a labelled button, and the first question asks whether you use a screen reader, so directions are not spoken twice. To open Waymark, the visitor taps a link the venue sends before the visit, or holds the phone to an NFC tag beside the entrance board. Either way, the link opens that building directly (go.html?building=…), with nothing to find or aim at. To choose where to go, they swipe through the list of places, or dictate with the keyboard microphone: "take me to the lift". While walking, Repeat, Where am I? and Call a helper are large buttons.
Does it replace a cane or guide dog?
No. Waymark knows where the walls, doors and stairs are. It knows nothing about people, bags on the floor or a door left open. The visitor keeps their cane or dog, and Waymark tells them which way to go.
What happens when the building changes?
When frames are clear but stop matching the map, the agent flags that area for re-mapping and the helper console shows it. A volunteer then re-walks that corridor. The map does not update itself.
Which phones does it work on?
Any recent iPhone or Android phone with a rear camera and a current browser. All the vision runs on the server, so the phone only sends a photo about once a second. It works with VoiceOver and TalkBack, and without them.
What does it cost to run?
It's free for visitors and buildings. One c8g.xlarge instance costs about $3.83 a day and answers each visitor frame in about a third of a second. The smaller c8g.large costs half that at 0.6 seconds a frame.
What happens to the camera images?
Volunteer walk frames are kept for 30 days so a walk can be replayed, then deleted. Visitor frames are not stored at all. There are no accounts.
How accurate is it?
On the 7-Scenes benchmark, a merged map is within 1.2 cm at the median and visitors are located to 1.77 cm. Real buildings with glass, blank walls or poor light are harder, and the agent says so when they are. Field test numbers come next.
Does it work across floors?
Not yet. Each map covers one floor, and a route ends at a lift or staircase.
See the map. Then follow it on a phone.
The sample building has two volunteer walks already merged. Open it on a laptop, then open the guidance app on a phone.
Map a space yourself with the mapping appPrint the entrance board
