Advanced Signals - C# API
To get access to the TobiiXR.Advanced
API, you need a Tobii Ocumen license.
Read more about Tobii Ocumen and how to get a license here.
Table of Contents
TobiiXR.Advanced
TobiiXR.Advanced
contains advanced eye tracking signals and requires a Tobii Ocumen license.
TobiiXRAdvanced.ProcessNewData()
Reads the latest data from the eye tracker. Data is already read automatically at the beginning of a frame so calling this method is only useful if you want to read the latest data later in the frame.
TobiiXRAdvanced.StartTimesyncJob()
Starts a non-blocking timesync job and returns a JobHandle
. You can poll the state of the timesync job by calling JobHandle.IsCompleted
.
To retrieve the result of the timesync job call FinishTimesyncJob
.
If StartTimesyncJob
is called again before the previous job has finished, the previous job will be cancelled.
See Time Synchronization for more details about how time synchronization is performed.
Example
This example shows how to set up a Component that continuously saves timesync data in a list without blocking the main thread.
public class TimesyncRecorder : MonoBehaviour
{
// In a real application you might want to persist this data to disk or use it from other parts of your
// application to create synchronization points between the eye tracker clock and host system clock.
private List<TobiiXR_AdvancedTimesyncData> _timesyncData = new List<TobiiXR_AdvancedTimesyncData>();
private void Start()
{
StartCoroutine(TimesyncCoroutine());
}
private IEnumerator TimesyncCoroutine()
{
while (true)
{
// Start a timesync job asynchronously and receive a handle that can be used to poll
// for the state of the job
var handle = TobiiXR.Advanced.StartTimesyncJob();
while (!handle.IsCompleted)
{
// Keep polling the handle every frame until it signals that the job has completed.
yield return new WaitForEndOfFrame();
}
// Since the handle has signalled completed, the call to FinishTimesyncJob is no longer blocking
var data = TobiiXR.Advanced.FinishTimesyncJob();
if (data.HasValue)
{
// If the timesync operation completed successfully, we store the result to use for
// interpolating gaze data.
_timesyncData.Add(data.Value);
}
// Depending on how high accuracy you need for your use case, wait a number of seconds
// before performing another timesync operation.
yield return new WaitForSecondsRealtime(60);
}
}
}
TobiiXRAdvanced.FinishTimesyncJob()
Blocks until the job started by StartTimesyncJob
has finished and returns a TobiiXR_AdvancedTimesyncData
struct if the timesync job was successful, null otherwise.
To prevent this method from blocking, make sure IsCompleted
is true for the JobHandle
received when calling StartTimesyncJob
before calling this method.
Definition of TobiiXR_AdvancedTimesyncData
Name | Type | Description |
---|---|---|
StartSystemTimestamp |
long |
Timestamp from system clock when the sync operation started. Measured in microseconds. |
EndSystemTimestamp |
long |
Timestamp from system clock when the sync operation ended. Measured in microseconds. |
DeviceTimestamp |
long |
Timestamp from device clock when the sync package was received by the device. Measured in microseconds. |
TobiiXRAdvanced.GetSystemTimestamp()
Gets a timestamp from the same system clock that is being used to compute TobiiXR_AdvancedEyeTrackingData.SystemTimestamp
.
The timestamp will be monotonic on all host systems that have a monotonic clock.
Example
This example shows how to calculate system latency.
public void PrintLatency()
{
// Get the latest eye tracking data packet
var data = TobiiXR.Advanced.LatestData;
// Get a timestamp from the host system clock
var now = TobiiXR.Advanced.GetSystemTimestamp();
// Calculate system latency in milliseconds and print it
var latency = (now - data.SystemTimestamp) / 1000.0f;
Debug.Log(string.Format("System latency was {0:0.00} ms", latency));
}
TobiiXRAdvanced.LatestData
Returns the latest TobiiXR_AdvancedEyeTrackingData
read from the eye tracker in local coordinate space of the camera, allowing for the most unprocessed data format. This will work even if the QueuedData
queue has been cleared.
Eye Tracking Data in World Space
To translate eye tracking data to world space coordinates, the eye tracking data needs to be fused with the head pose data, which can be either predicted or historic. You can read more about this in our Coordinate Systems page.
To use historic head pose, you can use TobiiXRAdvanced.TryGetLocalToWorldMatrixFor().
To use predicted head pose, you can use the camera local to world matrix in this way:
{
var transform = CameraHelper.GetCameraTransform(); // You could also use Camera.main.transform
var data = TobiiXR.Advanced.LatestData;
var origin = transform.TransformPoint(data.GazeRay.Origin);
var direction = transform.TransformDirection(data.GazeRay.Direction);
}
Definitions
Definition of TobiiXR_AdvancedEyeTrackingData
:
Name | Type | Description |
---|---|---|
SystemTimestamp |
long |
Timestamp, synchronized to host system clock, when the data used to calculate eye tracking signals was captured. Measured in microseconds. |
DeviceTimestamp |
long |
Timestamp, from the eye tracker clock, when the data used to calculate eye tracking signals was captured. Measured in microseconds. |
Left |
TobiiXR_AdvancedPerEyeData |
Stores eye tracking data specific to the left eye. |
Right |
TobiiXR_AdvancedPerEyeData |
Stores eye tracking data specific to the right eye. |
GazeRay |
TobiiXR_GazeRay |
Stores origin and direction of the gaze ray. |
ConvergenceDistance |
float |
The distance along the combined gaze ray where the eyes converge, measured in meters. For most eye tracking systems, the accuracy of this signal is low at distances of more than 1 meter. |
ConvergenceDistanceIsValid |
bool |
The validity of ConvergenceDistance . Do not use the value of ConvergenceDistance if this is false. |
Definition of TobiiXR_AdvancedPerEyeData
:
Name | Type | Description |
---|---|---|
GazeRay |
TobiiXR_GazeRay |
Stores origin and direction of the gaze ray. |
IsBlinkDataValid |
bool |
True if the blink data is valid, indicates if blink signal is supported by the current device. |
IsBlinking |
bool |
True if the user's eye is closed. False if open. |
PupilDiameterValid |
bool |
True when pupil diameter value is valid. |
PupilDiameter |
float |
Unit is mm, but it is only reliable to measure relative changes of the pupil diameter unless your eye tracker hardware specification says otherwise. Most eye trackers cannot compensate for the magnification caused by a pair of eye glasses. |
PositionGuideValid |
bool |
True when position guide value is valid. |
PositionGuide |
Vector2 |
Normalized (0.0-1.0) position of pupil in relation to optical axis where 0.5, 0.5 is on the optical axis. This information is useful to tell you if the user is wearing the headset correctly and if the user's interpupillary distance matches the lens separation. |
Definition of TobiiXR_GazeRay
:
Name | Type | Description |
---|---|---|
Origin |
Vector3 |
The origin point of the gaze ray. |
Direction |
Vector3 |
The normalized direction vector for the direction of the gaze ray. |
IsValid |
bool |
The validity of the TobiiXR_GazeRay . Do not use Origin or Direction of this data if this is false. |
TobiiXRAdvanced.QueuedData
A First In First Out (FIFO) queue of TobiiXR_AdvancedEyeTrackingData
. If the size of the queue reaches 30, the oldest element will be automatically dequeued and discarded before new data is enqueued.
If you want to make sure that every TobiiXR_AdvancedEyeTrackingData
is read once and only once, for example when recording data to file, dequeue all the elements off the queue every frame. Beware that will leave the queue empty for other scripts in your application.
If you want to filter on multiple samples, you can use an iterator or ToArray to access the elements in the queue without changing it. Beware that they are ordered from oldest to newest.
If you only want the latest data read from the eye tracker, use LatestData
instead.
Example
This example shows how to record all eye tracking data to a json file.
using System.IO;
using Tobii.XR;
using UnityEngine;
public class EyeTrackingDataRecorder : MonoBehaviour
{
private StreamWriter _dataFile;
private bool _receivedData;
private void Start()
{
// Create a file to record eye tracking data to
var filePath = Path.Combine(Application.persistentDataPath, "recording.json");
_dataFile = new StreamWriter(filePath);
_dataFile.WriteLine("[");
}
private void Update()
{
// Dequeue all new data received since last frame
while (TobiiXR.Advanced.QueuedData.Count > 0)
{
var data = TobiiXR.Advanced.QueuedData.Dequeue();
// Serialize to json and append to file
if (_receivedData) _dataFile.WriteLine(",");
var json = JsonUtility.ToJson(data);
_dataFile.Write(json);
_receivedData = true;
}
}
private void OnDestroy()
{
_dataFile.Write("]");
// Make sure to close the file before terminating the application
_dataFile.Close();
}
}
TobiiXRAdvanced.TryGetLocalToWorldMatrixFor()
Interpolates between the closest head pose data before and after the supplied timestamp. This method can be used to retrieve a local to world matrix for the time when the eye tracking data was captured. From this matrix, you can get eye tracking data in world space using the historic head pose.
{
var et = TobiiXR.Advanced.LatestData;
var mat = Matrix4x4.identity;
if (!TobiiXR.Advanced.TryGetLocalToWorldMatrixFor(et.SystemTimestamp, out mat))
{
// TryGetLocalToWorldMatrixFor will only fail if no historic data has been recorded. Use predicted head pose in this case.
mat = CameraHelper.GetCameraTransform().localToWorldMatrix;
}
var origin = mat.MultiplyPoint3x4(et.GazeRay.Origin);
var direction = mat.MultiplyVector(et.GazeRay.Direction);
}
To be able to match captured eye tracking data to the correct camera pose, you must ensure the eye tracking data is time synchronized. See StartTimesyncJob
TobiiXRAdvanced.GetMetadata()
Queries the eye tracker for metadata. This is a blocking call and can take several milliseconds to complete.
Returns
TobiiXR_EyeTrackerMetadata
contains the following values:
Name | Type | Description |
---|---|---|
SerialNumber |
string |
Serial number is unique to the eye tracker. However, for most eye tracker models, the first five characters are the same for all devices of the same model. |
Model |
string |
The eye tracker model name. |
RuntimeVersion |
string |
Version of the Tobii Runtime currently being used to connect to the eye tracker. |
OutputFrequency |
string |
The expected frequency with which the eye tracker will deliver data. On eye trackers where it's not possible to get the output frequency the value will be "Unknown". |