Pico Neo 3 Pro Eye, Tobii eye-tracking access from a PC

[Bob Hoskins]

Hi all, good evening.

I have a Pico Neo 3 Pro Eye on loan from a client, and was wondering if there’s a way, over USB or Wifi, to use Tobii SDK to read the gaze data from a PC, in order to implement dynamic foveated rendering in my game engine. Is this possible?

I can’t seem to get a clear answer from the folks at Pico, but it’s clear there is no way to do this through their official runtime and PC VR streaming solution, which relies on SteamVR and doesn’t expose the appropriate ext gaze interaction extension through OpenXR.

I don’t necessarily need to use OpenXR to access the gaze coordinates, and can integrate the Tobii SDK, but only if it’s possible and will work, naturally. I don’t have time to dig deep into the SDK just to find out, so I’m asking here.

Otherwise I suppose my only option is to use it in standalone mode only, and forward the eye-tracking data over Wifi, which would be rather disappointing. I honestly can’t seem to get any Tobii based headset makers to even acknowledge that PC VR even exists, let alone has developers interested in integrating eye-tracking into their apps/games at a low level. But we do exist, and I’m here asking for advice and help on the matter. Even the new Pimax Crystal, which apparently does support eye-tracked foveated rendering on PC, can’t tell me if, as a developer, I can access this eye data myself to implement it myself (amongst other things, like menu interactions or PC-PC or PC-NPC eye-contact).

In a side note, I heard of a youtuber, ThrillWu, is making some demo videos about eye tracking addons to add it to different headsets, like the Index, or the new BigScreen, so I was wondering if it’s using a Tobii product I can purchase and use in code. I don’t mind buying a dedicated eye-tracker, so long as I can use it in any headset I want to, from my PC. I asked this same question a few years ago from Johann while he was working here, and I was told that there is no way, I hope that’s changed by now.

thanks for your time, I look forward to the answer.

[Chi Zhang]

The question also puzzles me for long. As far as I knew from staffs in Tobii China, Tobii provide eye trackers to Pico and HTC. But Pico only exposes OpenXR API and hides the access to tobii sdk (as u mentioned) which HTC used to provide. However recently HTC cannot let developers get eye data from tobii eye trackers. Don’t know if they will do the same as Pico.

[Grant [Tobii]]

Hi @bobhoskins/

It’s surprising you didn’t get any answer from Pico on this, as they have an article on their website that shows how to implement this – https://developer-global.pico-interactive.com/document/unity/unity-xr-plat-FoveatedRender

Let us know if this suits your needs!

Regarding Integration Components, yes this is possible.. please send me a PM and I will put you in touch with the correct Sales Person.

[Bob Hoskins]

Hi Grant, thanks for the response.

As far as I can tell, Unity only supports DFR on Pico 3 Neo Pro Eye in standalone builds, not PC VR.

Pico’s PC VR OpenXR runtime/compositor uses SteamVR (indeed I think it’s just a SteamVR plugin, not sure). I don’t know if SteamVR even forwards their eye-tracking data through OpenXR this way, or even can. It would have to add an app layer extension I believe (but I’m not at all sure about this). I need native SDK access and they have virtually no Native SDK documentation, sadly. I checked. All I want is Tobii data from a PC through a standalone connected over USB-C or Wifi. I know that Pimax Crystal has DFR support on PC, too, so I know it’s theoretically possible on the Pico Neo 3 as well, I was just hoping to be able to use this headset but it seems like they already end-of-lifed it, for all intents and purposes.

If you could dig in to my Tobii SDK options for this that would be great. I don’t see why it’s so hard to use this feature on PC, they seem to be deliberately kneecapping their own product. Or just don’t care.

[Author]

Posts