Get Eye-screen-coordinates and match it with a Unity game object

[Furkan Genc]

Hi,

I am really new here and I need help to simply get eye coordinates using Tobii Rex in Unity and match it with a Unity game object. How/Where should I start ?

[Jenny [Tobii]]

Hi Furkan,

The kind of data you want is called Gaze Point data in EyeX. It is the location on the screen where the user is looking.

In the EyeX Unity package there is an EyeX component called GazePointDataComponent that you can add to your game object. From the component you can then read the LastGazePoint property in your update loop to get the last EyeXGaxePoint. The EyeXGazePoint struct has properties to get the gaze point in Screen, Viewport, GUI and Display coordinates.

There are also EyeX components that are directly coupled to looking at the game object. For example you can add a GazeAwareComponent which has a property that tells you if the user is currently looking at the game object or not.

Check out this quick and simple tutorial on how to get started in Unity and have a cube that spins when you look at it:
http://developer.tobii.com/getting-started-tobii-eyex-sdk-for-unity/

[Furkan Genc]

I use the code below but it moves just opposite on the X coordinate (it goes left when I look right and it goes right when I look left). Unfortunately, I cannot find the mistake here;

void Update()
	{
		var displaySize = _eyeXHost.DisplaySize;
		var screenBounds = _eyeXHost.ScreenBounds;
		var gazePoint = _gazePointDataComponent.LastGazePoint;

		var gazePointOnDisplayX = gazePoint.Display.x;
		var gazePointOnDisplayY = gazePoint.Display.y;

		if (displaySize.IsValid &&
		    screenBounds.IsValid && screenBounds.Value.Width > 0 && screenBounds.Value.Height > 0 &&
		    gazePoint.IsValid)
		{
			var normalizedGazePoint = new Vector2(
				(float)((gazePointOnDisplayX - screenBounds.Value.X) / screenBounds.Value.Width),
				(float)((gazePointOnDisplayY - screenBounds.Value.Y) / screenBounds.Value.Height));
			
			var gazePointOnDisplayPlaneMm = new Vector2(
				(float)((0.5 - normalizedGazePoint.x) * displaySize.Value.Width),
				(float)((0.5 - normalizedGazePoint.y) * displaySize.Value.Height));
			
			
			_rendererComponent.transform.position = new Vector3(
				gazePointOnDisplayPlaneMm.x * Scale,
				gazePointOnDisplayPlaneMm.y * Scale,
				0);
			
			_rendererComponent.enabled = true;
		}
		else
		{
			_rendererComponent.enabled = false;
		}
	}

Could you help me please ?

[Furkan Genc]

Ok I got it, there is no neccessary to help any more thanks.

[Na Cui]

Hi,
Can you help me?

I need to implement a programming, read the gaze coordinates too.

Can you tell me how do you implement it?

Thank you!

[Abirami Muthurathinam]

Hi Furkan Genc,
I too have the same problem now to find the coordinate point of the gaze. Can you please help me suggesting that how did you do your work?

It would be very useful for my work

Thank you!

[Grant [Tobii]]

Hi @abirami,

The technique used to access to the coordinate point was explained further up in this thread.. did this not explain adequately what you were trying to achieve?

Perhaps you were no longer referring to accessing the data within Unity and rather within a standard EyeX SDK program?

The means to do this is well explained within the user manual. If you could be more explicit with the issues that you are having, I will try to point you in the right direction.

[Abirami Muthurathinam]

Hi,
Thank you very much for your reply. I tried using the EyeX SDK with the instruction set but I don’t work on it for the moment. The instruction manual was pretty useful with my basic work.

I am a Masters student presently doing telecommunications engineering. I am interested in learning about eye tracking systems. I am working on this for my Master’s project with the basic sources I have. I am trying to understand how to find the XY co-ordinates on screen. I use a single hd camera and one IR light source to make a remote eye tracking system. I am using an open source (Haytham) software to track the pupil movement.

I simply use a PDF document to read and test the software. The software dose not identify the gaze point on screen, say, the exact word location where I am looking at. Rather it produces only XY coordinates for pupil movement (recorded by the camera) displayed on screen.
I would like to find the co-ordinates of the words where I am looking at, on screen so that I can find the saccade eye movement pattern. I would like to know, is it possible to find the original gaze point on screen using the coordinates recorded by the software?

The project mainly depends on this data, so if I get to know about some ideas regarding this, that would be really very helpful with my progression. I sincerely request for suggestions regarding my query.

I would be very grateful for your help.
Thank you very much

Abirami

[Grant [Tobii]]

Hi @abirami,

In the Developers Guide for Unity, Page 27 the Gaze point data is discussed.

As you might have noticed, there are a couple of filtering options with respect to the Gaze Stream.

May I ask why the standard Gaze Stream (filtered/unfiltered) is not sufficient for your needs?
This should supply you the XY coordinates of where the user is looking on screen.

If this issue is that you wish to store and analyse the data, then that is another case and must be
accomplished with the Tobii Pro range of eye tracker as the storing of eye data is against the user
agreement included with the EyeX SDK.

[Rasa Bhattarai]

Hello,
I am a beginner in using both Tobii and unity. I used unity to build a small game where a game object moves around in circular motion.
I have Tobii Eye X and I used TobiiAPI.GetGazePoint to get the x and y co-ordinates. My questions are :-

1. The co-ordinates as shown in the console window: How to interpret those co-ordinates ? In normal computers we have (0,0) on the upper left. So is it same (0,0 on upper left) when we run the unity game and try to check the co-ordinates with eye tracker ?
2. I want to save the co-ordinates as obtained from the eye tracker into a file. What is the procedure to do so ?

Really looking forward for the answers.
Thank you tobii team and other users

[Grant [Tobii]]

Hi @rasa, thanks for your query. You will be pleased to know we provide a detailed description of the co-ordinate system we used in the Unity SDK in the documentation pages which you can find @ https://tobii.github.io/UnitySDK/manual#gaze-point-data

The GazePoint.Viewport returns the gaze point in Viewport coordinates, where (0,0) is the bottom-left and (1,1) is the top-right corner of the UnityEngine.Screen. The values can be higher than 1 and lower than 0 if the user looks a bit outside the physical bounds of the monitor.

With regards, to saving the co-ordinates obtained from the eye tracker to a file, this is expressly prohibited by the Unity SDK licence agreement and requires the purchase of an ‘analytical licence’ to do. Any long term storage of gaze data is not permitted without this special licence.

You can read more @ https://analyticaluse.tobii.com/?utm_source=developer.tobii.com

If you would like a quote for this licence, please send a speculative email to [email protected] with a description of your project.

Please let me know if I can provide any further information for you.

[Rasa Bhattarai]

Hello,
Thanks for the information.
So does it mean that if I want to store the eye gaze co-ordinates on some .txt file or any other file then I need the purchase of analytical license ?

Also, does the use of analytical license give some different storage method(connection with cloud or sth etc.) to store the data or is it the same concept like using File handling in c# ?

Thank you once again.

[Grant [Tobii]]

Hi @rasa, yes that’s correct. Any long term storage of the gaze data does indeed require purchase of the analytical licence to do so.

The licence does not provide a specific means by which to store your data (cloud, file, etc) but rather provides you the means to collect and analyse your data without violating the Tobii Unity SDK licence agreement.

Please let me know if you require any further clarification.

[Akin Gunduz]

Hi,
I have the necessary licenses. I use “Tobii.Research” on .Net. But I have a problem with the coordinate system.
e.RightEye.GazePoint.PositionOnDisplayArea.X
This code gives me a point between (0,0) and (1,1). But I need pixel values between (0,0) and (1920,1080). How do I get these pixel values?

[Grant [Tobii]]

Hi @akingunduz, thanks for your query. indeed the gaze data will be displaed by default in co-ordinates normalised with reference to the current screen in use.

What this signified for example is that the point value (1,1) would represent pixel co-ordinates (1920,1080), (0.5,0.5) would be (0.5*1920,0.5*1080)=(960,540), etc

So hopefully you can see the simple conversion necessary of multiplying the normalised units by the screen pixel dimensions should you want the gaze data in absolute pixel format.

Please let us know if we can be of any further assistance.

[Author]

Posts