Tobii XR Devzone > Develop > Tobii Ocumen > Advanced Signals > Unity/C# >

Advanced Signals - C# API

To get access to the TobiiXR.Advanced API, you need a Tobii Ocumen license.

Read more about Tobii Ocumen and how to get a license here.


Table of Contents

TobiiXR.Advanced

TobiiXR.Advanced contains advanced eye tracking signals and requires a Tobii Ocumen license.


TobiiXRAdvanced.ProcessNewData()

Reads the latest data from the eye tracker. Data is already read automatically at the beginning of a frame so calling this method is only useful if you want to read the latest data later in the frame.


TobiiXRAdvanced.StartTimesyncJob()

Starts a non-blocking timesync job and returns a JobHandle. You can poll the state of the timesync job by calling JobHandle.IsCompleted.

To retrieve the result of the timesync job call FinishTimesyncJob.

If StartTimesyncJob is called again before the previous job has finished, the previous job will be cancelled.

See Time Synchronization for more details about how time synchronization is performed.

Example

This example shows how to set up a Component that continuously saves timesync data in a list without blocking the main thread.

public class TimesyncRecorder : MonoBehaviour
{
    // In a real application you might want to persist this data to disk or use it from other parts of your
    // application to create synchronization points between the eye tracker clock and host system clock.
    private List<TobiiXR_AdvancedTimesyncData> _timesyncData = new List<TobiiXR_AdvancedTimesyncData>();

    private void Start()
    {
        StartCoroutine(TimesyncCoroutine());
    }

    private IEnumerator TimesyncCoroutine()
    {
        while (true)
        {
            // Start a timesync job asynchronously and receive a handle that can be used to poll
            // for the state of the job
            var handle = TobiiXR.Advanced.StartTimesyncJob();
            while (!handle.IsCompleted)
            {
                // Keep polling the handle every frame until it signals that the job has completed.
                yield return new WaitForEndOfFrame();
            }

            // Since the handle has signalled completed, the call to FinishTimesyncJob is no longer blocking
            var data = TobiiXR.Advanced.FinishTimesyncJob();
            if (data.HasValue)
            {
                // If the timesync operation completed successfully, we store the result to use for
                // interpolating gaze data.
                _timesyncData.Add(data.Value);
            }

            // Depending on how high accuracy you need for your use case, wait a number of seconds
            // before performing another timesync operation.
            yield return new WaitForSecondsRealtime(60);
        }
    }
}

TobiiXRAdvanced.FinishTimesyncJob()

Blocks until the job started by StartTimesyncJob has finished and returns a TobiiXR_AdvancedTimesyncData struct if the timesync job was successful, null otherwise.

To prevent this method from blocking, make sure IsCompleted is true for the JobHandle received when calling StartTimesyncJob before calling this method.

Definition of TobiiXR_AdvancedTimesyncData

Name Type Description
StartSystemTimestamp long Timestamp from system clock when the sync operation started. Measured in microseconds.
EndSystemTimestamp long Timestamp from system clock when the sync operation ended. Measured in microseconds.
DeviceTimestamp long Timestamp from device clock when the sync package was received by the device. Measured in microseconds.

TobiiXRAdvanced.GetSystemTimestamp()

Gets a timestamp from the same system clock that is being used to compute TobiiXR_AdvancedEyeTrackingData.SystemTimestamp. The timestamp will be monotonic on all host systems that have a monotonic clock.

Example

This example shows how to calculate system latency.

public void PrintLatency()
{
    // Get the latest eye tracking data packet
    var data = TobiiXR.Advanced.LatestData;

    // Get a timestamp from the host system clock
    var now = TobiiXR.Advanced.GetSystemTimestamp();

    // Calculate system latency in milliseconds and print it
    var latency = (now - data.SystemTimestamp) / 1000.0f;
    Debug.Log(string.Format("System latency was {0:0.00} ms", latency));
}

TobiiXRAdvanced.LatestData

Returns the latest TobiiXR_AdvancedEyeTrackingData read from the eye tracker in local coordinate space of the camera, allowing for the most unprocessed data format. This will work even if the QueuedData queue has been cleared.

Eye Tracking Data in World Space

To translate eye tracking data to world space coordinates, the eye tracking data needs to be fused with the head pose data, which can be either predicted or historic. You can read more about this in our Coordinate Systems page.

To use historic head pose, you can use TobiiXRAdvanced.TryGetLocalToWorldMatrixFor().

To use predicted head pose, you can use the camera local to world matrix in this way:

{
    var transform = CameraHelper.GetCameraTransform(); // You could also use Camera.main.transform
    var data = TobiiXR.Advanced.LatestData;
    var origin = transform.TransformPoint(data.GazeRay.Origin);
    var direction = transform.TransformDirection(data.GazeRay.Direction);
}

Definitions

Definition of TobiiXR_AdvancedEyeTrackingData:

Name Type Description
SystemTimestamp long Timestamp, synchronized to host system clock, when the data used to calculate eye tracking signals was captured. Measured in microseconds.
DeviceTimestamp long Timestamp, from the eye tracker clock, when the data used to calculate eye tracking signals was captured. Measured in microseconds.
Left TobiiXR_AdvancedPerEyeData Stores eye tracking data specific to the left eye.
Right TobiiXR_AdvancedPerEyeData Stores eye tracking data specific to the right eye.
GazeRay TobiiXR_GazeRay Stores origin and direction of the gaze ray.
ConvergenceDistance float The distance along the combined gaze ray where the eyes converge, measured in meters. For most eye tracking systems, the accuracy of this signal is low at distances of more than 1 meter.
ConvergenceDistanceIsValid bool The validity of ConvergenceDistance. Do not use the value of ConvergenceDistance if this is false.

Definition of TobiiXR_AdvancedPerEyeData:

Name Type Description
GazeRay TobiiXR_GazeRay Stores origin and direction of the gaze ray.
IsBlinkDataValid bool True if the blink data is valid, indicates if blink signal is supported by the current device.
IsBlinking bool True if the user's eye is closed. False if open.
PupilDiameterValid bool True when pupil diameter value is valid.
PupilDiameter float Unit is mm, but it is only reliable to measure relative changes of the pupil diameter unless your eye tracker hardware specification says otherwise. Most eye trackers cannot compensate for the magnification caused by a pair of eye glasses.
PositionGuideValid bool True when position guide value is valid.
PositionGuide Vector2 Normalized (0.0-1.0) position of pupil in relation to optical axis where 0.5, 0.5 is on the optical axis. This information is useful to tell you if the user is wearing the headset correctly and if the user's interpupillary distance matches the lens separation.

Definition of TobiiXR_GazeRay:

Name Type Description
Origin Vector3 The origin point of the gaze ray.
Direction Vector3 The normalized direction vector for the direction of the gaze ray.
IsValid bool The validity of the TobiiXR_GazeRay. Do not use Origin or Direction of this data if this is false.

TobiiXRAdvanced.QueuedData

A First In First Out (FIFO) queue of TobiiXR_AdvancedEyeTrackingData. If the size of the queue reaches 30, the oldest element will be automatically dequeued and discarded before new data is enqueued.

If you want to make sure that every TobiiXR_AdvancedEyeTrackingData is read once and only once, for example when recording data to file, dequeue all the elements off the queue every frame. Beware that will leave the queue empty for other scripts in your application.

If you want to filter on multiple samples, you can use an iterator or ToArray to access the elements in the queue without changing it. Beware that they are ordered from oldest to newest.

If you only want the latest data read from the eye tracker, use LatestData instead.

Example

This example shows how to record all eye tracking data to a json file.

using System.IO;
using Tobii.XR;
using UnityEngine;

public class EyeTrackingDataRecorder : MonoBehaviour
{
    private StreamWriter _dataFile;
    private bool _receivedData;

    private void Start()
    {
        // Create a file to record eye tracking data to
        var filePath = Path.Combine(Application.persistentDataPath, "recording.json");
        _dataFile = new StreamWriter(filePath);
        _dataFile.WriteLine("[");
    }

    private void Update()
    {
        // Dequeue all new data received since last frame
        while (TobiiXR.Advanced.QueuedData.Count > 0)
        {
            var data = TobiiXR.Advanced.QueuedData.Dequeue();

            // Serialize to json and append to file
            if (_receivedData) _dataFile.WriteLine(",");
            var json = JsonUtility.ToJson(data);
            _dataFile.Write(json);
            _receivedData = true;
        }
    }

    private void OnDestroy()
    {
        _dataFile.Write("]");

        // Make sure to close the file before terminating the application
        _dataFile.Close();
    }
}

TobiiXRAdvanced.TryGetLocalToWorldMatrixFor()

Interpolates between the closest head pose data before and after the supplied timestamp. This method can be used to retrieve a local to world matrix for the time when the eye tracking data was captured. From this matrix, you can get eye tracking data in world space using the historic head pose.

{
    var et = TobiiXR.Advanced.LatestData;
    var mat = Matrix4x4.identity;
    if (!TobiiXR.Advanced.TryGetLocalToWorldMatrixFor(et.SystemTimestamp, out mat))
    {
        // TryGetLocalToWorldMatrixFor will only fail if no historic data has been recorded. Use predicted head pose in this case.
        mat = CameraHelper.GetCameraTransform().localToWorldMatrix;
    }
    var origin = mat.MultiplyPoint3x4(et.GazeRay.Origin);
    var direction = mat.MultiplyVector(et.GazeRay.Direction);
}

To be able to match captured eye tracking data to the correct camera pose, you must ensure the eye tracking data is time synchronized. See StartTimesyncJob


TobiiXRAdvanced.GetMetadata()

Queries the eye tracker for metadata. This is a blocking call and can take several milliseconds to complete.

Returns

TobiiXR_EyeTrackerMetadata contains the following values:

Name Type Description
SerialNumber string Serial number is unique to the eye tracker. However, for most eye tracker models, the first five characters are the same for all devices of the same model.
Model string The eye tracker model name.
RuntimeVersion string Version of the Tobii Runtime currently being used to connect to the eye tracker.
OutputFrequency string The expected frequency with which the eye tracker will deliver data. On eye trackers where it's not possible to get the output frequency the value will be "Unknown".