Link eye-tracking pointer to the mouse

[Thomas Anquetil]

Hello there,

First and foremost I am new to this forum, so I’d like to apologize if I’m asking the question at the wrong place. I’m working with Unity so I thought this section would do the job.

I have to work on Unity with HTC Vive Pro, to build an interface so that a user can interact with a 3D environment using only his eyes. Now, this should not be new on such a forum, but this is not my point.

Due to the current health situation around the World (I’m in France, and we’ve gone lockdown again since last friday) I cannot work at my school anymore. The issue is that means I cannot have the headset to work. So my question is simple : is there a way to kind of simulate the headset without actually having it ? In this specific situation, that would help me a lot. I recall that I work with Unity.

Regards,
Thomas Anquetil

[Grant [Tobii]]

Hi @thomasaqtl, thanks for your enquiry.. Indeed you are in the right place to ask!

You will be please to know there is a mouse emulator we provide that can mimick the eye gaze data received which should satisfy your requirements:

You can read more about this @ https://vr.tobii.com/sdk/develop/unity/documentation/tobii-settings/#mouse-provider

Check it out and let us know how you get on. Best Wishes.

[Thomas Anquetil]

Hi, thank you for your quick reply.

I’m a bit confused as to how to make it work with SteamVR. Because I have to use SteamVR but since I do not have headset it asks me to setup my headset, which I do not have. How to go around with this ?

Regards,
Thomas Anquetil

[Grant [Tobii]]

Hi @thomasaqtl, could you kindly clarify exactly what SDK you are developing with? The above mentioned mouse-emulator tool is for use with the Tobii XR SDK and needs to be used within this environment.

[Thomas Anquetil]

Hi, I apologize for being not clear. Let me be a little more specific.

Whatever the scene I’m working with (let’s say Example_HandEyeCoordination in TobiiXR examples), in Tobii XR_Initializer (from the Inspector), I put “Mouse” at the top, in the Eye Tracking Provider section, then I launch the game mode in Unity. BUT to work with Unity in VR I need a configured Headset in SteamVR to run the simulation. The problem is I have no headset. So I cannot event launch the game mode (well, actually I can, but nothing happens).

Is the Mouse Provider supposed to work only if a headset is used and only to replace the Eye Tracking device ?

I hope I have been clear this time.

[Grant [Tobii]]

Hi @thomasaqtl, yes you are correct that the mouse emulator will only work in scene mode for simulating gaze data and does not emulate a camera in VR.

Additionallty, the Hand Eye Coordination scene specifically will not work without a VR setup and controls. So I’m afraid that to actually run the scene in a proper game mode, you will indeed need a regular VR Headset. Hopefully can operate effectively under these restrictions. Best Wishes.

[SHU WEI]

Hello just want to follow up on this topic-

I am currently developing a VR experience with eye tracking to record gaze behaviours. I have issues using the mouse provider to simulate gaze for dev testing within the Unity editor.

I have followed the official guidance and this post, but still can’t have it work. Here is my process: 1. Use the example scene “Example_DevTools” as a test scene and select “Mouse” as the eye tracking provider from Tobii XR Initializer object 2. Disable XR support and use the (Old) Input System in my Project Settings 3. Press Run with no error message (I expect the mouse provider should work at this point, however, it doesn’t do anything in both the scene view and focused game view). Is there anything I’m missing?

Also to my understanding, my deployment device (Pico Neo 3 Pro Eye) does not support Tobii eye-tracking debugging within the Unity editor (even after I used their preview tool app), I wonder do you have any suggestions on dev and testing pipeline for Pico with Tobii (e.g. using the mouse provider for agile testing)?

Thanks in advance,
Shu

[Grant [Tobii]]

Hi @swei1215, were you able to run the mouse emulation within the XR SDK sample scene.. ie the 3 cubes? It should be enough to simply put mouse emulator at the top of the hierarchy without any need to disable an XR settings.

Indeed, you are right that I’m afraid the Pico Preview Tool does not yet support the Tobii XR SDK.. but it does support the pico eye tracking metrics, so a possible workaround is to use the pico data stream for debug and switch to Tobii for release.

[SHU WEI]

Thanks Grant, the mouse emulator doesn’t work for me in the GettingStarted sample scene too.

I noticed that although I chose “Mouse” as Eye tracking provider (Standalone Eye Tracking Provider), the selection highlight disappears when I press play and the debug message logs “Starting TobiiXR with (Tobii.XR.NoseDirectionProvider) as provider for eye tracking.” I presume the setting selection is not passed successfully, do I need to specify in the TobiiXR.cs or TobiiXR_Initializer.cs script with something like “Internal.Provider = new MouseProvider();” to make it work?

Thanks in advance,
Shu

[SHU WEI]

Hi Grant, just an update to say I have the mouse simulator working now. Dragging “Mouse” on the top of the provider list and then restarting Unity helps.

Thanks for the suggestion about the debug workaround of using the Pico data stream, I will give it a go.

Shu

[Grant [Tobii]]

Glad to hear you got it working, thanks for the update!

[Author]

Posts