How it works
Five stations.One assistant.
SPAI runs in Apple Vision Pro while a technician works. It speaks each step, watches gloves, hands and instruments through the headset camera, and logs everything, without anyone touching a keyboard at the sink.
Real footage recorded inside Apple Vision Pro, Spatial Computing Innovation Lab, Full Sail University.
Compliance runs
on memory.
Sterile processing keeps surgical instruments safe, but compliance is mostly tracked by hand, after the fact. There’s no real-time check while the work happens, so mistakes surface later, if at all.
of hospital surveys cited the reprocessing standard as the most-cited standard in 2019.
The Joint Commission, 2019of surgical cases had at least one instrument error in one review.
PMC, 2024per minute of operating-room time. Delays from a bad tray are expensive.
Childers & Maggard-Gibbons, JAMA Surgery, 2018The five
stations
SPAI follows a tray from the dirty side to sealed and ready. Each station has its own spoken script, its own checks, and a clear “complete” or “send back.”
- 01
Decontamination
PPE on first, then guided cleaning steps for the dirty side.
Gloves on · contamination risk - 02
Inspection
Check each instrument for damage and debris before it moves on.
Instruments present · step confirmed - 03
Tray assembly
Match the count sheet, confirm the right tray, complete or send back.
Tray loaded · instrument count - 04
Packaging
Wrap or container the set for sterilization.
Gloves on · step confirmed - 05
Seal validation
Verify packaging and seals before the set is released.
Step confirmed · session logged
Under
the hood
Four parts, each doing one job. Detection runs on the headset with Core ML; a small backend handles logging and heavier checks; a state machine decides whether the step is allowed to move on.
Vision Pro
Main-camera access through Apple’s Enterprise API. Panels, voice and hand tracking.
FastAPI
Receives frames and events, stores session logs and compliance records.
YOLOv8
Two trained models: gloves + hands, and surgical instruments. Exported to Core ML.
FSM
A finite-state machine checks each step against the protocol and halts on a violation.
Computer vision has matched human auditors 96.8% of the time on hand-hygiene monitoring (Singh et al., JAMIA, 2020).
The models,
honestly
Numbers you can check, and the limits that come with them. Improving both is what the pilot is for.
| Model | Score | Trained on | What it does | Known limits |
|---|---|---|---|---|
| v2 · Gloves + hands | 93.6 | First-person headset footage | Gloved vs bare hand at every step | Needs more blue and white nitrile gloves and real SPD lighting |
| v4 · Instruments | 92.1 | ASTRA surgical instruments dataset | Counts instruments; tray loaded or empty | Labels every tool “instrument”: it counts, it doesn’t name them yet |
Scores are mAP50 on each model’s validation set. Both are YOLOv8n.
When something goes wrong
A bare hand at the wrong moment halts the workflow and sounds an alert from the direction of that panel.
The trainee acknowledges it, fixes it, and carries on. The event log keeps the record either way.