The landscape of Extended Reality (XR) hardware is undergoing a significant shift as startups move beyond standard interaction models to solve long-standing technical hurdles. At the VR/AR Expo China held in Shanghai, a Western startup named Immersix demonstrated a novel approach to eye-tracking technology that deviates from the industry-standard pupil-tracking method. By focusing on the unique features of the human retina rather than the position of the pupil, the company claims to have developed a system that offers superior accuracy, lower power consumption, and a solution to the persistent problem of frequent recalibration in Mixed Reality (MR) and Virtual Reality (VR) headsets.
The Technical Shift: From Pupil to Retina Tracking
Most contemporary eye-tracking solutions, including those integrated into high-end devices like the Apple Vision Pro and Meta Quest Pro, rely on pupil tracking. These systems typically use a ring of infrared (IR) LEDs to create "glints" on the cornea, which a camera then monitors to calculate the eye’s position relative to the headset’s lenses. While effective, this method is highly sensitive to "headset slippage"—the minute movements of the device on the user’s face—which often necessitates frequent recalibration to maintain accuracy.
Immersix has introduced a paradigm shift by tracking the retina’s internal structure. The hardware module for this system is notably minimalist, consisting of a single camera and a solitary IR LED per eye. Instead of focusing on the exterior surface of the eye, the camera is designed to look through the pupil to capture the unique patterns of blood vessels on the retina. Because the retina is a fixed biological structure that does not change over time, it provides an absolute "ground truth" for the eye’s orientation, independent of the headset’s physical position on the user’s face.
The Calibration Process and Persistent Mapping
One of the most significant pain points in current XR eye-tracking is the calibration phase, which users often find tedious. The Immersix solution requires a specialized initial calibration that functions similarly to a biometric scan. During this stage, the user is prompted to follow a series of markers on a display. As the eye moves, the camera captures various segments of the retina, which the software then stitches together to create a comprehensive "feature map" or "retinal fingerprint."
In technical demonstrations, this process involved a "sweeping" motion where a user moved a pointer across a screen to "erase" a digital rectangle. The software intentionally introduces a slight lag to the pointer, forcing the user to move their eyes into extreme lateral positions. This ensures the camera can document the peripheral regions of the retina, resulting in a 360-degree internal map. Once this map is generated, it remains valid for years, as the retinal vascular pattern is stable throughout adulthood.
Following this one-time setup, the system performs active tracking by matching the small portion of the retina visible at any given moment against the stored master map. This pattern-matching algorithm allows the system to determine the exact orientation of the eye with sub-degree accuracy.
Performance Metrics and Hardware Efficiency
The implications for hardware design are substantial. By reducing the number of IR LEDs from a full circle to just one, Immersix significantly lowers the power draw of the eye-tracking module. This is a critical factor for the next generation of lightweight AR glasses, where battery life and thermal management are primary constraints.
The company reports the following performance specifications for its current evaluation kits:
- Accuracy: Sub-degree precision in eye orientation detection.
- Refresh Rate: Up to 120Hz, matching the high-frequency requirements for foveated rendering.
- Robustness: High resistance to headset slippage; the system maintains tracking even if the frames are moved or removed and replaced.
- Hardware Footprint: Minimalist camera and LED configuration, suitable for integration into slim eyeglass frames.
During field tests at the Shanghai expo, the system demonstrated high reliability in UI navigation tasks. Users were able to interact with a grid of circular icons, with the system instantly highlighting the specific element being looked at. Even after intentionally shifting the frames on the face, the tracking remained consistent, a feat that typically causes standard pupil-tracking systems to fail or require a "quick-fix" recalibration.
Broader Impact on the XR Industry
The move toward retina-based tracking addresses several industrial requirements simultaneously. For enterprise and professional use cases, the stability of the tracking is paramount. In surgical simulations or industrial training, a loss of eye-tracking precision due to headset movement can disrupt the entire experience.
Furthermore, this technology aligns with the growing need for "Foveated Rendering"—a technique where only the area the user is looking at is rendered in high resolution. For foveated rendering to be effective without being noticeable to the user, the tracking must be both extremely fast (low latency) and highly accurate. Immersix’s claim of 120Hz tracking suggests that their solution is capable of meeting the demands of high-performance GPU optimization.
The technology also opens doors for seamless biometric authentication. Since the system already creates a retinal map for tracking purposes, that same data can serve as a secure "Retina ID" for logging into devices, making payments, or accessing sensitive data within a virtual environment. This dual-purpose functionality could simplify the hardware stack for future headsets.
Privacy Considerations and Data Security
As with any technology involving biometric data, the shift to retina tracking raises significant privacy questions. Retinal patterns are as unique as fingerprints and are considered sensitive personal information. Storing a high-resolution map of a user’s retina poses a potential security risk if the data were to be intercepted or uploaded to a cloud server.
To address these concerns, industry analysts suggest that such data must be stored locally on the device’s Secure Enclave, similar to how Apple handles FaceID or TouchID data. Immersix has indicated that the data is intended for local processing, ensuring that the "retinal fingerprint" never leaves the hardware. As the technology moves toward commercialization, transparency regarding data encryption and storage will be vital for consumer adoption.
Future Outlook and Availability
Immersix is currently positioning itself as an OEM (Original Equipment Manufacturer) partner, offering evaluation kits to major tech companies involved in the production of AR and VR hardware. By providing a devkit that can be integrated into prototypical frames, the startup is allowing companies like Meta, Pico, and others to test the viability of retina tracking for their upcoming product cycles.
The transition from a prototype demonstrated at a trade show to a mass-produced consumer product involves several hurdles, including manufacturing at scale and ensuring compatibility with various eye shapes and conditions (such as cataracts or high myopia). However, the successful demonstration in Shanghai suggests that the core algorithm and hardware configuration are mature enough for serious consideration by the industry’s major players. If adopted, retina tracking could become the new standard for "set-it-and-forget-it" eye interaction, finally removing one of the most persistent friction points in the XR user experience.
About the Author
