Tobii Ocumen I/O for Unity
Record or stream live gaze and scene elements directly to Tobii Ocumen Studio from Unity.
Table of Contents
You can attach recorders to your Unity project and they will search for relevant objects in the scene and record them, such as gaze data or colliders.
Recorders are orchestrated by the
Tobii Recording Manager. The manager collects data from all enabled recorders, and either
- writes them to a
.ocumencontainer file, or
- streams them over network, or
- gives stream access for custom I/O (e.g., Bluetooth).
.ocumen container is a highly optimized self-contained streaming format that includes everything you recorded and can subsequently be used without scene access. You can think of an
.ocumen stream as a
.mkv but for interactive, analytical content.
Internally an Ocumen recording consists of multiple tracks, each stored with microsecond resolution. Some tracks are recorded for you (see below), and you can also create your own.
At the time of writing the
.ocumen format is not stabilized yet and only compatible with a matching Tobii Ocumen Studio version at the time of release.
We therefore recommend you archive your current Tobii Ocumen Studio version along with any recordings you have made.
Make sure you follow our installation instructions for Tobii Ocumen.
Step 1: Add the
TobiiXR_Initializer script to your scene
Make sure you add your license to the script.
Step 2: Add the
Tobii Recording Manager script to your scene
It can be set to record to file or stream over network. If recording to file and the path is not used defined, it will store the recordings in the application’s persistent data path.
Autostart recording will automatically start recording at the start of the app and end when closed. If you want to manually control starting and stopping recordings, you can add the
TobiiRecordingToggler to your scene or create your own script to handle this.
Step 3: Add recorders
The recorders will determine what data will be recorded. These have to be added to the same game object as the
Tobii Recording Manager.
There are several recorders, and their naming might differ slightly from the picture above. Not all recorders are available in all versions or packages.
Step 4: Run Your App!
Tobii Recording Manager and recorders have been added to your scene, you can compile or launch your app!
Step 5: Open Tobii Ocumen Studio
Either with your application running in streaming mode or after creating a recording, you can launch Tobii Ocumen Studio to analyze your data.
A list of pre-defined recorders and their specifications. Unless otherwise stated, the following properties are true for all recorders:
- Recorders are invoked once per Unity frame.
- The timestamp used will be updated in a coroutine right after
Updatehas run on all game objects.
- Timestamps are sampled from the system clock.
Headset Pose Recorder
Records the camera pose and the view and projection matrices of the left and right virtual camera.
- Many headsets do not deliver valid matrices when they are not rendering so this recorder only records data when the application is rendering.
Advanced Eye Tracking Recorder
Records native Tobii Eye Tracking data.
- This recorder records the output of
- The eye tracking clock is by default synchronized to the system clock.
- If multiple eye tracking samples have arrived since the last frame, all samples will be written with their original time stamp. In other words, eye tracking data keep its system synchronized timestamp in the recording so you won’t have to worry about if the eye tracker runs at a higher or lower frame rate than your application.
- Requires Headset Pose Recorder to be able to calculate the gaze direction in world space, in order to for example visualize the gaze rays.
Frame Data Recorder
Records the frame number given by
Records any colliders attached to game objects in your scene.
- Sphere and box colliders are recorded as is.
- Mesh and capsule colliders are recorded as boxes.
- Collider type is currently only recorded the first time the collider is encountered.
- Only records one collider per object.
- Duplicated objects that contain children colliders in a Unity scene will only allow for one recorded collider in the duplicated object.
Records vertices and world position for all visible meshes in the scene belonging to a
MeshRenderer or a
- For models you want to record, make sure
Read/Write Enabledis checked in the inspector window. Read more at Unity’s documentation.
- If you want to record meshes for static objects you may have to turn off
Static Batchingin Player Settings since
Static Batchingcreates a combined mesh that is not always readable from CPU.
Records pose and input from VR controllers
- Does not record axis values such as joystick or touchpad.
- The controller input is recorded using Unity’s XR Input.
Rendered Video (Alpha)
Records a video capture of the left eye’s camera view in the HMD.
- Supports OpenGL ES 3.0 and higher on Android platforms.
- Currently only supports NVIDIA graphics cards using DirectX 11 on Windows platforms.
- The Video Recorder is currently in Alpha state. It does not support all platforms and may exhibit a temporal offset when played back. When replaying video in Ocumen Studio, there is a slider for manually adjusting the offset to match the video with the rest of the recording.
- Video data averages about 1 MB per second and usually dominates recording size by up to 2 orders of magnitude. While enabling video can give you authentic insights in highly dynamic (e.g., shader driven) scenes, disabling video can yield minute-long recordings that are just a few kilobytes long.