Stream Engine Gaze point values

[Zalo]

Hello,

I am creating a C# program with StreamEngine library and using interop calls.

Currently, I am trying to match the gaze point values received (tobii_gaze_point_t) from the eye tracker to the pixels on the screen. However, the gaze point values range from (0-1000, 0-1000) and that does not match the x,y coordinates of the screen.

Is calibration in my program needed to solve this issue of offset values or how others have dealt with this?

[Jonas Högström]

the gaze point values should be normalized over your screen area (0,0) represents top left corner and (1,1) is the bottom right corner. To get correct values you need to have done a correct screen setup. You typically don’t have to do this yourself if the tracker is a consumer tracker such as 4C or ET5. Once you get correct coordinates you need to multiply the values with the screen width/height in pixels if that is what you need. With the defailt license I acutally think you can’t change the screen setup. Use the tools that comes with the eye tracker for this.

User calibration is not really related to this problem, but to get correct gaze data you also need to make sure your user is correctly calibrated. Again, this isn’t something you have to (or can) implement yourself for consumer devices.

[Zalo]

I understand. Thank you for your input 🙂

[Christophe Zimmer]

Hi,

I also created a program in C# and I would like to have the coordinates of the gaze and apply your method, namely multiply x by the width resolution of the screen and the y value by the resolution of the height of the screen.
I do not obtain consistent values : ​​I have a resolution of 1920X1080 and yet I obtain coordinates greater than 2200 pixels in x value. Can you give me an example of code in C#? Here is a snippet of my code :

private static void OnGazePoint(ref tobii_gaze_point_t gazePoint, IntPtr userData)
{
// Check that the data is valid before using it
if (gazePoint.validity == tobii_validity_t.TOBII_VALIDITY_VALID)
{
Console.WriteLine($”Gaze point: {gazePoint.position.x*1920}, {gazePoint.position.y*1080}”);
}
}

Thanks you very much for your help,
Best regards,

Christophe

[Jonas Högström]

The code is not wrong. There are several reasons that you might get coordinates outside of the pixels of the screen:
* You are looking outside of the screen – look on the screen
* The user calibration is incorrect – perform a new user calibration
* the screen setup is incorrect – perform a new screen setup

The eye tracking signal isn’t perfect. typically the error is larger near the edges of the screen too, and there will always be occasional samples with significantly incorrect coordinates (due to some edge case during the image processing or computation in the eye tracker).

If you plot the computed coordinate on the screen, does it typically match where you are looking? Is there a constant offset? is there a scale issue? Either of these indicate that your personal calibration or screen setup is off. Very noisy signals could indicate problems with eye tracking for that particular person. Glasses and contact lenses, heavy make up, long eye lashes and partially closed eyes all lead to a degradation in signal quality.

[Christophe Zimmer]

Hi Jonas,
Thank you very much for your quick response, indeed I think it is because I am looking outside the screen.
I will try to program a visual cursor which would be displayed according to the coordinates returned by the API.
It should be easier to test if I look in the right place.
Have a good day,
Christophe

[Christophe Zimmer]

Hello,

I may have noticed when I program in the Tobii tutorial mode in C# (Stream Engine Demo) under visual studio 2022 with .NET in debug mode a crash of the application with a sudden exit of the program without error code feedback.

On the other hand, when I launch in release mode, the application waits for a keyboard key to be pressed correctly out of the program. Are you having this problem and where might the problem be coming from (misallocation of memory in debug mode with .NET in Visual Studio?). Can you enlighten me on that?

Thank you very much,

Christophe

[Jonas Högström]

I have not tried the sample applications, but crashes like that sounds like there is something that goes wrong in the native side of the application (in streamengine.dll) rather than the managed side, but it could of course be triggered by a bad call to the interop methods. If it differs between debug and release, it could indicate that there is a timing-factor, or that the GC behaves differently in debug and release for example.

Do you do any lengthy operations in the gaze callback? You should typically avoid doing any thing that requires dispatching to the UI thread or anything that takes longer than a few milliseconds to avoid blocking the sending thread for too long.

[Christophe Zimmer]

Dear Jonas

Thank you for your answer

You state what I am thinking of.

I am working with Christophe on that project. I am an experience software engineer, experienced in C# but also in system and low level programming.

I think there is an issue in the low level code (what I am calling the driver — the dll) – it might be a stack overflow or whatever mis behavior with memory.

Yes of course there is something different between release execution and execution with the VS debugger – nevertheless I need the software operate with the debugger. This difference may also include time behavior.

I don’t believe of a problem in Interop as I already worked with it and it did fine.

You asked if the gaze call back was doing a lot of job — no it does nothing but display x and y to the console
Nevertheless I can add some code to avoid to enter again in the call back if it is still in operation (how do you call that in English : “recursive call” ? “reentrance ?”) — just in case that the debugger is slowing down the Console.WriteLine()

BTW I put a try catch to exit nicely the program, and this failed too — the programs ends off all of a sudden.

But I think you should look at the low level code.

Best regards

Jean-Paul
Ingénieur Conseil (Développement C# / Delphi)

[Jonas Högström]

Exactly what demo application are you referring to above? I’d like to run the same code on my end and see if I can reproduce your crash. I have used all versions of the streamengine dll in debugmode from C# without any native crashes, so I think this crash is caused by something incorrect on the managed side (possibly combined with missing validation of input data on the native side), if it is code you wrote yourself that you can share, please PM me.

[Jonas Högström]

Writing gaze coordinates to the console should not be a problem to do in the gaze callback.

It would also help if you could let me know:
* What Tobii device you are using
* Firmware version
* Runtime service version
* SEC version

[Author]

Posts