Tobii EyeX is a complete eye tracking solution that includes both hardware and software to create the Tobii Eye Experience: natural human-computer interactions using a combination of eye-gaze and other user input.

Tobii_Eye_Experience_EyeXBox_297x185

The Tobii EyeX Controller is an eye tracking device that uses near-infrared light to track the eye movements and gaze point of a user.

The Tobii EyeX Engine is a piece of software that works like an OS extension for eye tracking. It knows how to configure and talk to the eye tracker, react to system changes, combine the user’s eye-gaze and other input and interpret them as user interactions, and it mediates between multiple applications that are using eye tracking simultaneously. It also contains a whole lot of eye tracking smartness harnessed over the 10+ years of eye tracking experience at Tobii. This is offered as filtered data streams and built-in heuristics that are specialized in figuring out what the user is actually looking at.

Tobii EyeX Interaction is a piece of software built on top of the EyeX Engine, and offers a set of basic eye-gaze interactions available out-of-the-box in the Windows environment.

The EyeX SDK provides you with access to the EyeX Engine API. It includes code samples, demo scenes, dlls, documentation and code for integration into a selected set of game engines and GUI frameworks.

Streams, states and behaviors

The EyeX Engine API can basically be divided into three categories: Streams, States and Behaviors.

The Streams provide nicely filtered eye-gaze data from the eye tracker transformed to a convenient coordinate system. The point on the screen where your eyes are looking (gaze point), and the points on the screen where your eyes linger to focus on something (fixations) are given as pixel coordinates on the screen. The positions of your eyeballs (eye positions) are given in space coordinates in millimeters relative to the center of the screen.

The States provide information about the current state of the EyeX system. There are a couple of more dynamic states related to the user: the user presence state (if there is a user in front of the screen or not) and the gaze tracking state (if the eye tracker is currently able to track the user’s eye-gaze or not). There are also states that are related to the system setup, such as the size in millimeters of the display the eye tracker is currently setup for, the currently selected user calibration profile, or if the eye tracker is currently available.

The Behaviors are higher level interaction concepts. You can compare them with familiar mouse interactions like click and scroll. The interaction behaviors are related to regions on the screen rather than single points. The regions typically correspond to GUI components like clickable buttons or scrollable panels. The concept of a gaze-aware region is a region that knows if a user is looking at it or not. The EyeX Engine uses advanced heuristics and filtering to decide which object a user is trying to interact with and how. This gives a much more stable and precise interaction than one that simply uses the coordinate of the latest gaze point.

How the EyeX Engine knows what the user is looking at

  1. The user looks at the screen. The EyeX Controller calculates the coordinates of the user’s gaze point. The EyeX Engine receives the normalized screen coordinates and transforms them to pixel coordinates on the screen.

    EyeXControllerGazePointCoordinates

  2. The EyeX Engine uses information from the client application to understand what kind of visual object the user is looking at and might want to interact with. Let’s say the rectangles on the screen are buttons with the ‘Activatable’ behavior. That means that they can be clicked using eye gaze. The EyeX Engine now decides which button the user is looking at and then expects the user to indicate that he or she wants to click the button.
  3. The client application might offer the user some different ways of actually clicking the button using eye-gaze, but it will always be some combination of looking at the thing you want to click, and then giving some secondary input to trigger the click. Let’s say that in this client application the user can trigger a click on the object they are looking at by pressing the space bar on the keyboard.

    Combine eye and key press to interact at gaze point

  4. The user presses the space bar while looking at the button he or she wants to click. The client application informs the EyeX Engine that the user has requested to ‘Activate’ the activatable button currently looked at.
  5. The EyeX Engine generates an event that informs the client application which button the user wants to activate.

    ActivationEvent

  6. The client application responds to the event and for example gives the user a visual feedback that the button on the screen is clicked, and then performs the action associated with clicking that button.

    ApplicationHandlesEvent

The Windows Presentation Foundation (WPF) and WinForms integrations included in the EyeX SDK for .Net makes a lot of the steps above seamless. The app developer specifies the desired EyeX behavior for a certain GUI element, and write the click handling code. The rest is taken care of by client side code included in the SDK.

19 thoughts on “An introduction to Tobii EyeX

    • Yes, the Tobii eye trackers work also when the head is tilted.

      Tobii’s eye tracking technology has been integrated into communication aids for people with e.g., cerebral palsy — some of which have difficulties holding their head upright — for a long time.

  1. I’ve a couple of questions to start with:

    1. What will happen if a person moves his head while using EyeX? Will the device loose its calibration?

    2. I am planning to develop an application where it will not be possible to calibrate every time someone uses it? Can it be done using EyeX?

    3. What are all the parameters that you are planning to include, other than gaze and pupil coordinates? If you want I can specify some that I might need in my application.

    Thanks

    • 1. Once a Tobii eye tracker is calibrated for a person, it allows head movements without having to recalibrate. As long as the person does not move the head too fast or outside of the tracking box, the device can track the eyes with good enough performance.

      2. If you need sub-centimeter precision, an individual calibration is needed. But if you can live with a few centimeters offset in your application, it is possible to re-use the same calibration on several persons. The best thing is to try it out and see if your users think it is ok.

      3. We have a lot of functionality that will be added in upcoming versions of the API. We will write an article during the week about some of the news. It would be good to know what needs you have also. If there are things missing in the API, please add a feature request in the community forums.

          • Hi dynamite,

            No, it is not possible to get the current frame rate programmatically. The frame rate is dynamic to the level that it can change from data frame to data frame depending on the kind of data stream. For example the Gaze Point Data stream will only stream valid gaze points on the screen, meaning that there are no data points if the user’s eyes are closed. This can happen when the user blinks. So, there is no overall “current frame rate” that would make sense to return via an API.

  2. Hi,
    Apparently I missed the hardware pre-requisite of USB3 for EyeX. Will it be extended to operate on USB2 hardware? Requiring 3 eliminates a lot of existing hardware (including mine!).
    Thanks!
    Peter

    • Hi Peter,
      I’m afraid the EyeX controller will always have the USB3 requirement, due to the high frequency of data required for this platform. Our other available development kit, Tobii REX, only requires USB2, note though that is not as small and cheap as the EyeX.

      If it is an alternative for you. there are PCI cards with USB3 connectors. Many of them are glitchy, so we have collected a list of recommended PCI cards on the Support FAQ.

    • No, it provides much more than that! Apart from a number of datastreams: gaze-point on the screen, eye position in 3D space, fixations, presence and more, there are higher level events. Using an EyeX SDK you can define regions on the screen and get events when the user is interacting with them. For example you could define a button in your application to have the Activatable Behavior, and then get “click” events when the user is clicking your button using EyeX Direct Click. To Direct Click something, the user simply looks at it and presses a pre-defined Direct Click button on their keyboard. There are more ready baked interaction modes like this available through the EyeX API, and we are continously expanding our SDKs with more samples to show how to implement them.

Leave a Reply