Blink detection

[Martin Kopecek]

Hi falks,

Is it possible somehow easily evaluate the blinking? I mean all examples about SDK are controls the shift, zoom, etc. but I need confirm the command by blink of eyes and do not use the mouse or keyboard…

Is this command, function…somewhere in the EyeX Engine? Optionally where can I find it?

Thanx Martin

[Jenny [Tobii]]

Hi Martin,

There is currently no explicit blink detection exposed through the EyeX Engine API.

You could implement a blink detection filter yourself by using the eye position data stream (TX_INTERACTIONBEHAVIORTYPE_EYEPOSITIONDATA). When the eye lid is closed there is no eye position data, so blinking corresponds to the gaps in the stream of eye position data for the respective eye. The gaps in the data can be calculated from the timestamps of the data packages. This way, blink detection can be made per eye.

[Martin Kopecek]

Hi Jenny,

thanx for answer,

regards Martin

[Fiona]

This means that blinking can only be detected once a new gaze point has been registered though, right?
I’m asking because I need to determine whether a user is currently looking at the screen (or how long ago he last did). One idea I had was to use the difference between the current time and the last gaze point’s timestamp. However, the timestamp format appears to correspond neither to the Unix nor the C# nor the Unity Time.time time formats. Could you tell me what the timestamp is based on? Or is there another way of telling whether there currently is gaze data available?

[Robert [Tobii]]

Hi,

You could also use a data stream called PresenceData, see the MinimalStatusNotifications sample in the EyeX SDK for C/C++ (equivalent syntax for C#). You will get events whenever the user closes her eyes (long blink) or looks away, and also when the eyes are visible again. If you want to know for how long the user has been non-present you can always save the current time as a variable when you get the non-present event.

EDIT 19 Nov 2014: The UserPresence engine state (formerly known as PresenceData) should not be used for blink detection. The purpose of the state is to tell whether the user is present in front of the screen or not. Even though a current implementation may approximate this with the detection of the user’s eyes, the underlaying implementation may at any time change to not be depending on whether the user’s eyes are open or closed.

[Fiona]

Thanks a lot, I will give it a try.

[Martin Kopecek]

Thanx a lot!

Martin

[Gal Sont]

Thanks, works like a charm!

[Dwayne]

Hi Jenny,

You suggested that the Eye Position data stream could be used to do blink detection.

I am noticing that the EyePosition.Next Event seems to fire separate events into the stream for each eye and that the EyePositionEventArgs contains either a valid LeftEye.IsValid value or a RightEye.IsValid value but does not include valid values for both eyes at the same time, when both are open. Naturally if neither eye is open, then the event does not fire.

As you suggested this is enough to detect blinks in individual eyes from the delta between individual Timestamp values.

I did find however that with a stream of independent eye events that detection of when a particular eye is closed was a little awkward with the current event arg data. A dispatch timer provided me a messy work around this by providing a way of timing out an eye tracking variable (aka programmatically turning off an eye indicator at a predetermined interval) if its datetime tick elapsed value exceeded a given timer interval. This however slows down my gaze indicator transition animation and/or causes my eye open indicators to flicker a lot. Whereas the system eye indicators that Tobii provides near the bottom of the screen are nice an steady.

Besides considering the possibility of adding native blink detection event support, it might also be nice if future versions of this sdk could enhance the EyePosition stream event to also pass a LostEyePosition event which would make both blink detection and eye open status more manageable in the client. Other possible ways to address this might be to include something in the individual event arg that indicated the current state of the other eye, and that way each EyePositionStream event had enough information in it to determine the state of both eyes at once. This last idea however would not help with detecting when the user closes both eyes since no event would fire and some sort of background timer strategy in the client would still be necessary to detect that scenario.

Perhaps you folks already have plans to fix this issue, but I thought I would throw something out there while I was wrestling with it in the alpha version of the SDK.

Thanks.

[Robert [Tobii]]

Hi Dwayne,

The Eye Position data stream should fire events also when both eyes are open. I wrote a little test app for that, based on the MinimalGazeDataStream.

namespace MinimalEyePositionStream
{
    using EyeXFramework;
    using System;

    public static class Program
    {
        public static void Main(string[] args)
        {
            using (var eyeXHost = new EyeXHost())
            {
                eyeXHost.Start();

                using (var eyePositionDataStream = eyeXHost.CreateEyePositionDataStream())
                {
                    eyePositionDataStream.Next += (s, e) => Console.WriteLine("IsValid: (Left :{0}, Right: {1}) @{2:0}", e.LeftEye.IsValid, e.RightEye.IsValid, e.Timestamp);

                    Console.WriteLine("Listening for eye position data, press any key to exit...");
                    Console.In.Read();
                }
            }
        }
    }
}

Most of the time when my eyes are open, I see IsValid: (Left:True, Right:True). However, this data is a bit glitchy and requires filters and timers to be useful in blink detection.

Have you tried using the EyeXHost.UserPresenceChanged event instead? That gives a more robust event when eyes disappear and come back again. If you can compare the event timestamp between UserPresense=false and UserPresence=true and detect a blink based on that, it might be an easier alternative.

But as you say, native blink detection in the EyeX Engine would be easiest. It is on the backlog, but the priority is not so high at the moment.

[Dwayne]

Hi Robert,

Ah yes, I see you are correct.
I gave up on seeing the “both valid” condition to quickly while debugging.
If I dump a stream of write lines to the debug window I do indeed see a mix of events fired where both, left or right eyes are showing as valid when both eyes are open and being seen by the sensor.

So yes, I would agree that the data stream does seem a bit glitchy right now and not reliable enough on its own to sense when either eye is closed or when both eyes are closed.

I do have code that toggles a bound dependency property to indicate when a user is present or not but I did not end up using the UserPresenceChanged event for my blink detection logic. My current logic detects both individual left/right eye blinks as well as both eye blinks. The messy aspect of it is that it required the dispatch timer firing in the background to timeout my eye open variables which introduces an additional performance hit that would have been nice to avoid.

So I do have something that is working, I just wanted to provide some feedback based on what I was experiencing with the alpha SDK. Now that you have clarified that the stream is a mix of both eye and individual eye events, my feedback would be that it would be nice if the events more reliably reported back on both eyes each time. However, I am sure there are lots of factors involved and I appreciate that some of those factors may make it more challenging to do so.

Thanks!

[cysiek]

Hi Robert,
is there any chance for native blink detection in EyeX SDK or in Gaze SDK? If yes, when can we excpect api with such feature? Alternatively is there any new method for eye-blink detection using Tobii EyeX Controller? This feature is crucial for my research and just a “hope” that it will work better than in EyeTribe (in which eye blink detection is rather poor) caused me purchased Tobii EyeX Controller, so i hope it was a good decision.

[Karl R Shifflett]

HI Tobii Support,

It’s been a few years now, have you implemented blink or eye close detection?

Thanks,

Karl

[Grant [Tobii]]

Hi @kdawg1406, Apologies for the inconvenience but Blink Detection is not something implemented within the Tobii Core SDK functionality although you can of course create your own detection algorithm based around loss of tracking data within a small specific time window.

[Karl R Shifflett]

Hi Grant,

Thank you for the reply.

Best,

Karl

[Author]

Posts