☰
Navigation :
Tobii XR Devzone > Develop > Tobii Ocumen > Ocumen I/O > Unity/C# >
API Reference

Ocumen I/O - C# API

This page outlines the C# API for Tobii Ocumen I/O.

API Overview

Functions

Freestanding callables inside the module.

Classes

Methods operating on common state.

Enums

Groups of related constants.

  • BlinkStatePerEye - Describes the openness of an eye in discrete steps. What exactly constitues
  • ColliderType - The type of collider used in the game engine.
  • ConnectionReadyState - Encodes if a (remote) connection is ready.
  • CoordinateSystem - Gaze and similar data is usually recorded relative to the eye tracker. However, some
  • FrameType - Describes what information a frame contains.
  • ImageFormat - Video format of this frame.
  • LogLevel - Log level for Ocumen functions.
  • LookedAtWith - Specified with which eye(s) the hit test should be performed.
  • ShapeType - How shape data is to be interpreted.
  • TrackedData - Represents a ‘well known track’ contained in an Ocumen recording.
  • VideoFormat - The video format of this frame.
  • VideoTrack - The video format of this frame.
  • WellKnownType - Some types are ‘well known’, that is, they have a definitive bit representation
  • XRInputDevice - Which controller is being described.

Data Structs

Composite data used by functions and methods.

Data Structs

BinocularGaze

Gaze rays for the left and right eye.

Coordinates in here are local space of the eye tracker, relative to a fictional reference point between the eyes. To transform them to world space you need to use the CameraPose struct.

Fields

  • left - Gaze origin and direction of the left eye.
  • right - Gaze origin and direction of the right eye.

Definition

public partial struct BinocularGaze
{
    public Rayf32 left;
    public Rayf32 right;
}

BlinkState

Encodes blink states for each eye.

Fields

  • left - Left eye blink state.
  • right - Right eye blink state.

Definition

public partial struct BlinkState
{
    public BlinkStatePerEye left;
    public BlinkStatePerEye right;
}

CameraPose

View matrices of the combined camera sitting between both eyes.

Fields

  • cameraToWorldMatrix - The (assumed by non-existent) place where the camera that sits between both eyes would be. this is local-to-world, use case is to describe where head is; we expect a left-handed coordinate system; with z-axis facing forward (away) from user.

Definition

public partial struct CameraPose
{
    public Matrix4f32 cameraToWorldMatrix;
}

ColliderPose

Where in world space a collider resides.

Fields

  • colliderId -
  • frameTransform -

Definition

public partial struct ColliderPose
{
    public ulong colliderId;
    public EntityFrameTransform frameTransform;
}

ColliderShape

Describes the shape of a collider.

Fields

  • colliderId - The transform this belongs to.
  • objectId - The root object this collider refers to (e.g., in case this came from a collider attached to a nested transform).
  • colliderType - The type of collider, e.g. physical or trigger.
  • shape - Actual shape of the collider.

Definition

public partial struct ColliderShape
{
    public ulong colliderId;
    public ulong objectId;
    public ColliderType colliderType;
    public Shape shape;
}

ColorRGB

Encodes a single color value in 8 bit RGB.

Fields

  • r - Red value between 0 and 255.
  • g - Green value between 0 and 255.
  • b - Blue value between 0 and 255.

Definition

public partial struct ColorRGB
{
    public byte r;
    public byte g;
    public byte b;
}

CombinedGaze

Ray of a fictional, combined cyclops eye.

Some eye trackers can use internal sensor information to compute a combined gaze signal optimized for interaction purposes. Combining gaze usually acts as a spatially filtered signal with integrated error compensation, which can be better suited for medium to long range non-scientific use cases.

Coordinates in here are local space of the eye tracker, relative to a fictional reference point between the eyes. To transform them to world space you need to use the CameraPose struct.

Fields

  • gaze - The combined gaze ray.

Definition

public partial struct CombinedGaze
{
    public Rayf32 gaze;
}

ControllerButtonState

Described button states for a controller.

As of now, these buttons follow Unity’s XR Input specification

Fields

  • device - The device described by this controller state.
  • primaryButton - See Unity XR Input specification.
  • primaryTouch - See Unity XR Input specification.
  • primary2dAxisClick - See Unity XR Input specification.
  • primary2dAxisTouch - See Unity XR Input specification.
  • secondaryButton - See Unity XR Input specification.
  • secondaryTouch - See Unity XR Input specification.
  • secondary2dAxisClick - See Unity XR Input specification.
  • secondary2dAxisTouch - See Unity XR Input specification.
  • triggerButton - See Unity XR Input specification.
  • gripButton - See Unity XR Input specification.
  • menuButton - See Unity XR Input specification.

Definition

public partial struct ControllerButtonState
{
    public XRInputDevice device;
    public bool primaryButton;
    public bool primaryTouch;
    public bool primary2dAxisClick;
    public bool primary2dAxisTouch;
    public bool secondaryButton;
    public bool secondaryTouch;
    public bool secondary2dAxisClick;
    public bool secondary2dAxisTouch;
    public bool triggerButton;
    public bool gripButton;
    public bool menuButton;
}

ControllerPose

Where in world space a controller is.

Fields

  • device - The device described.
  • transform - World space position of the device.

Definition

public partial struct ControllerPose
{
    public XRInputDevice device;
    public EntityFrameTransform transform;
}

ConvergenceDistance

Closest distance of gaze convergence.

Distance here is relative to a fictional reference point between the eyes, equivalent to the world position of the CameraPose struct.

Fields

  • distanceM - Distance in meters to the reference points where closest convergence occurs.

Definition

public partial struct ConvergenceDistance
{
    public float distanceM;
}

Descriptor

Helper to request a descriptor from the recording library.

Fields

  • cls - The class of data to describe, any ASCII string up to 32 bytes, e.g., gameobject.
  • prop - The property of data to describe, any ASCII string up to 32 bytes, e.g., color.
  • entityId - The entity to describe.
  • user - The user for which this data is tracked.
  • len - Low-level sizeof byte length of a single tracked element.
  • wellKnownType - What type of data is tracked.

Definition

public partial struct Descriptor
{
    public string cls;
    public string prop;
    public ulong entityId;
    public ulong user;
    public ushort len;
    public WellKnownType wellKnownType;
}

EntityFrameTransform

Holds an update for a geometry’s transform. Usually given every frame.

Fields

  • localToWorldMatrix - This matrix is used to transform gaze data from the object’s local space to world space.

Definition

public partial struct EntityFrameTransform
{
    public Matrix4f32 localToWorldMatrix;
}

EntityLabel

Maps an entity ID to an ASCII label.

Fields

  • meta - Label meta data.
  • label - NUL-terminated ASCII pointer with label.

Definition

public partial struct EntityLabel
{
    public EntityLabelID meta;
    public string label;
}

EntityLabelID

Type-helper to store and retrieve labels for entities.

Fields

  • entity -
  • labelLen - When setting a label this field will be ignored, when retrieving a label it will contain the string length including the terminating NUL.

Definition

public partial struct EntityLabelID
{
    public ulong entity;
    public uint labelLen;
}

EntrancePupilData

Entrance pupil data.

Fields

  • left - Left eye data.
  • right - Right eye data.

Definition

public partial struct EntrancePupilData
{
    public Vector3f32 left;
    public Vector3f32 right;
}

EyeTrackerDeviceTimestamp

Timestamp reported by the eye tracker according to its own clock.

Fields

  • timestamp - Timestamp reported by the device.

Definition

public partial struct EyeTrackerDeviceTimestamp
{
    public ulong timestamp;
}

FrameDimension

Dimension of the video frame in pixels.

Fields

  • width - Width in pixels.
  • height - Height in pixels.

Definition

public partial struct FrameDimension
{
    public uint width;
    public uint height;
}

FrameMeta

Meta data about a video frame / packet when passing the packet around.

Fields

  • frameType - What the encoded packet describes.
  • frameId - (Usually sequential) number starting at 0, increased each frame.
  • dependsOn - Some frames might depend on other frames for display. If set this indicates which.
  • renderedTime - The time in micro seconds a packet was originally rendered in the game engine when a copy of this frame was taken and sent to the encoder.

Definition

public partial struct FrameMeta
{
    public FrameType frameType;
    public uint frameId;
    public Optionu32 dependsOn;
    public long renderedTime;
}

Gizmo3D

Helper that can be attached to scenes to debug or visualize certain properties.

Fields

  • gizmoId - Entity ID of this gizmo.
  • simpleGeometry - Shape in Ocumen Studio.
  • frameTransform - World position of the gizmo.
  • color - Color to show.

Definition

public partial struct Gizmo3D
{
    public ulong gizmoId;
    public Shape simpleGeometry;
    public EntityFrameTransform frameTransform;
    public ColorRGB color;
}

ImageData

Image descriptor and data pointers.

Fields

  • descriptor - Describes what is encoded in the slice.
  • data - Actual image data.

Definition

public partial struct ImageData
{
    public ImageDescriptor descriptor;
    public Sliceu8 data;
}

ImageDescriptor

Describes the data of a single video frame, as provided by encoderlib.

Fields

  • track - Track of embedded image information.
  • format - Format of the data buffer.
  • dimensions - Width and height of the image.
  • imageDataLength - Length of the encoded image.

Definition

public partial struct ImageDescriptor
{
    public byte track;
    public ImageFormat format;
    public FrameDimension dimensions;
    public uint imageDataLength;
}

InstanceLifetime

Holds information whether an entity is enabled.

An entity is enabled (i.e., visibile and participating in the scene) by default. Instances can be disabled when they disapper in the game, e.g., the underlying object is deleted.

Fields

  • instanceId - Instance described.
  • enabled - Whether instance is alive or not.

Definition

public partial struct InstanceLifetime
{
    public ulong instanceId;
    public bool enabled;
}

LogMessage

A single message recorded by our library.

Fields

  • level - The level of the message.
  • message - ASCII encoded actual message.
  • file - Our source file where this message originated from.
  • line - Line number where this message originated from.

Definition

public partial struct LogMessage
{
    public LogLevel level;
    public string message;
    public string file;
    public uint line;
}

LoggerConfiguration

Configures how logging should be handled.

Fields

  • logLevel - The minimal log level to log.
  • maxLogWaitMs - When initializing the logger, with a previous logger present, how long shall we try to wait replacing the previous logger if it is locked.

Definition

public partial struct LoggerConfiguration
{
    public LogLevel logLevel;
    public uint maxLogWaitMs;
}

Matrix4f32

A 4x4 transformation matrix.

Fields

  • m - Array member elements in column major order.

Definition

public partial struct Matrix4f32
{
    public float m0;
    public float m1;
    public float m2;
    public float m3;
    public float m4;
    public float m5;
    public float m6;
    public float m7;
    public float m8;
    public float m9;
    public float m10;
    public float m11;
    public float m12;
    public float m13;
    public float m14;
    public float m15;
}

Mesh

Identifies a series of vertices and indices describing an object.

Fields

  • meshId - Mesh ID described.
  • vertices - List of vertices making up the mesh.
  • indices - Triangle indices of the used vertices.

Definition

public partial struct Mesh
{
    public ulong meshId;
    public SliceVector3f32 vertices;
    public Slicei32 indices;
}

ObjectMesh

Links an object with a mesh.

Fields

  • objectId - Object ID linked to a mesh ID.
  • meshId - Mesh ID of this object.

Definition

public partial struct ObjectMesh
{
    public ulong objectId;
    public ulong meshId;
}

ObjectPose

Keeps pose data (translation, rotation, scale) for an object.

Fields

  • objectId - Object ID described here.
  • frameTransform - The object’s pose.

Definition

public partial struct ObjectPose
{
    public ulong objectId;
    public EntityFrameTransform frameTransform;
}

Point2f32

A point in 2D space.

Fields

  • x - x component.
  • y - y component.

Definition

public partial struct Point2f32
{
    public float x;
    public float y;
}

Point3f32

A point in 3D space.

Fields

  • x - x component.
  • y - y component.
  • z - z component.

Definition

public partial struct Point3f32
{
    public float x;
    public float y;
    public float z;
}

PositionGuide

The 2D position of the eyes in the headset.

Fields

  • left - X and Y position of left eye.
  • right - X and Y position of right eye.

Definition

public partial struct PositionGuide
{
    public Point2f32 left;
    public Point2f32 right;
}

PupilSizeAbsolute

Pupil diameter in millimeters for both eyes.

Fields

  • left - Left eye pupil diameter.
  • right - Right eye pupil diameter.

Definition

public partial struct PupilSizeAbsolute
{
    public PupilSizePerEye left;
    public PupilSizePerEye right;
}

PupilSizePerEye

Pupil diameter for the eye

Fields

  • sizeMm - Pupil diameter in millimeters for the eye.

Definition

public partial struct PupilSizePerEye
{
    public float sizeMm;
}

Quaternionf32

A quaternion we use for vector operations.

Fields

  • s -
  • v -

Definition

public partial struct Quaternionf32
{
    public float s;
    public Vector3f32 v;
}

Rayf32

A ray with an origin and direction.

Fields

  • origin - Origin of the ray.
  • direction - Direction of the ray.

Definition

public partial struct Rayf32
{
    public Point3f32 origin;
    public Vector3f32 direction;
}

RecordingInformation

Recording meta data.

Fields

  • duration - How long the recording lasted.
  • first - The first observed time stamp in the recoring. Is very likely not 0.
  • last - The last observed time stamp in the recoring.

Definition

public partial struct RecordingInformation
{
    public long duration;
    public long first;
    public long last;
}

Shape

A simple geometry expressable with just 3 points.

Fields

  • shapeType - Defines how [points] are to be interpreted.
  • points - See definition of [Shape] for interpretation

Definition

public partial struct Shape
{
    public ShapeType shapeType;
    public Point3f32 points0;
    public Point3f32 points1;
    public Point3f32 points2;
}

StereoProjections

Projection matrices used for rendering.

Fields

  • left - The applied view projection for the left eye. We expect an OpenGL project matrix for Ocumen Studio to render properly.
  • right - The applied view projection for the right eye. We expect an OpenGL project matrix for Ocumen Studio to render properly.

Definition

public partial struct StereoProjections
{
    public Matrix4f32 left;
    public Matrix4f32 right;
}

StereoViews

View matrices of each eye, i.e., where each eye / camera is in world space.

Fields

  • left - Where the left eye actually is, relative to world-space. Use case is to convert world space object coordinates to view space; so this is world-to-camera matrix).
  • right - Where the right eye actually is, relative to world-space. Use case is to convert world space object coordinates to view space; so this is world-to-camera matrix).

Definition

public partial struct StereoViews
{
    public Matrix4f32 left;
    public Matrix4f32 right;
}

TimeValueBinocularGaze

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueBinocularGaze
{
    public long time;
    public BinocularGaze value;
}

TimeValueBlinkState

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueBlinkState
{
    public long time;
    public BlinkState value;
}

TimeValueCombinedGaze

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueCombinedGaze
{
    public long time;
    public CombinedGaze value;
}

TimeValueConvergenceDistance

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueConvergenceDistance
{
    public long time;
    public ConvergenceDistance value;
}

TimeValueEntrancePupilData

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueEntrancePupilData
{
    public long time;
    public EntrancePupilData value;
}

TimeValueEyeTrackerDeviceTimestamp

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValueEyeTrackerDeviceTimestamp
{
    public long time;
    public EyeTrackerDeviceTimestamp value;
}

TimeValuePositionGuide

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValuePositionGuide
{
    public long time;
    public PositionGuide value;
}

TimeValuePupilSizeAbsolute

Contains a single value and its time.

Fields

  • time - Timestamp of the value, in microseconds.
  • value - Actual value at the given time.

Definition

public partial struct TimeValuePupilSizeAbsolute
{
    public long time;
    public PupilSizeAbsolute value;
}

TimesValuesCameraPose

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesCameraPose
{
    public Slicei64 times;
    public SliceCameraPose values;
}

TimesValuesColliderPose

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesColliderPose
{
    public Slicei64 times;
    public SliceColliderPose values;
}

TimesValuesColliderShape

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesColliderShape
{
    public Slicei64 times;
    public SliceColliderShape values;
}

TimesValuesGizmo3D

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesGizmo3D
{
    public Slicei64 times;
    public SliceGizmo3D values;
}

TimesValuesInstanceLifetime

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesInstanceLifetime
{
    public Slicei64 times;
    public SliceInstanceLifetime values;
}

TimesValuesObjectMesh

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesObjectMesh
{
    public Slicei64 times;
    public SliceObjectMesh values;
}

TimesValuesObjectPose

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesObjectPose
{
    public Slicei64 times;
    public SliceObjectPose values;
}

TimesValuesStereoProjections

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesStereoProjections
{
    public Slicei64 times;
    public SliceStereoProjections values;
}

TimesValuesStereoViews

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesStereoViews
{
    public Slicei64 times;
    public SliceStereoViews values;
}

TimesValuesu32

Maps a series of values to corresponding times.

Both slices are guaranteed to be of equal length and meant to be walked in lock step.

Fields

  • times - Timestamp in microseconds.
  • values - Values at the times above.

Definition

public partial struct TimesValuesu32
{
    public Slicei64 times;
    public Sliceu32 values;
}

TrackHandle

File-descriptor-like handle for one stream of data inside a recording.

For example user3.object3.world_pos would have its own handle.

Descriptor handles are ephemeral and are only valid during a recording or query session. For example, when a recording is reloaded, previously returned descriptors will not necessarily match the reloaded data streams.

Fields

  • sizeOf - Len in bytes of the type described.
  • unused1 - Special flags for later use.
  • unused2 - Special flags for later use.
  • id - ID of this descriptor.

Definition

public partial struct TrackHandle
{
    ushort sizeOf;
    byte unused1;
    byte unused2;
    uint id;
}

TrackedWellKnownInfo

Describes a track.

Fields

  • cls - Class name, e.g., eeg.
  • prop - Property name, e.g., ch
  • entity - Entity ID.
  • user - User of that track.

Definition

public partial struct TrackedWellKnownInfo
{
    public string cls;
    public string prop;
    public ulong entity;
    public ulong user;
}

UserAtTime

Specifies a user ID and associated time stamp in our API.

Fields

  • userId - User ID described.
  • time - Time stamp in microseconds of the event described.

Definition

public partial struct UserAtTime
{
    public ulong userId;
    public long time;
}

UserInTimeRange

Specifies a user ID and associated time range in our API.

Fields

  • userId - User ID described.
  • firstInclusive - Inclusive time range start in microseconds.
  • lastInclusive - Inclusive time range end in microseconds.

Definition

public partial struct UserInTimeRange
{
    public ulong userId;
    public long firstInclusive;
    public long lastInclusive;
}

Vector3f32

A 3-dimensional vector.

Fields

  • x - x component.
  • y - y component.
  • z - z component.

Definition

public partial struct Vector3f32
{
    public float x;
    public float y;
    public float z;
}

Version

Encodes a version number for our products, like 2.0.1.

Fields

  • major - Major component, e.g. 2.
  • minor - Minor component, e.g. 0.
  • patch - Patch component, e.g. 1.
  • build - Build ID.

Definition

public partial struct Version
{
    public byte major;
    public byte minor;
    public byte patch;
    public long build;
}

VideoFrame

Holds video meta information and actual video data.

Fields

  • meta - Video meta data.
  • data - Encoded video bytes.

Definition

public partial struct VideoFrame
{
    public VideoFrameDescriptor meta;
    public Sliceu8 data;
}

VideoFrameDescriptor

Describes the data of a single video frame, as provided by encoderlib.

Fields

  • track -
  • format -
  • dimensions -
  • commonMeta -

Definition

public partial struct VideoFrameDescriptor
{
    public VideoTrack track;
    public VideoFormat format;
    public FrameDimension dimensions;
    public FrameMeta commonMeta;
}

WellKnownTypedTrack

Maps a descriptor handle.

Fields

  • trackHandle - Handle of track described.
  • trackType - Type of the handle.

Definition

public partial struct WellKnownTypedTrack
{
    public TrackHandle trackHandle;
    public WellKnownType trackType;
}

SliceCameraPose

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceCameraPose
{
    IntPtr data;
    ulong len;
}

SliceColliderPose

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceColliderPose
{
    IntPtr data;
    ulong len;
}

SliceColliderShape

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceColliderShape
{
    IntPtr data;
    ulong len;
}

SliceControllerButtonState

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceControllerButtonState
{
    IntPtr data;
    ulong len;
}

SliceControllerPose

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceControllerPose
{
    IntPtr data;
    ulong len;
}

SliceEntityLabel

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceEntityLabel
{
    IntPtr data;
    ulong len;
}

SliceGizmo3D

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceGizmo3D
{
    IntPtr data;
    ulong len;
}

SliceImageData

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceImageData
{
    IntPtr data;
    ulong len;
}

SliceInstanceLifetime

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceInstanceLifetime
{
    IntPtr data;
    ulong len;
}

SliceMesh

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceMesh
{
    IntPtr data;
    ulong len;
}

SliceObjectMesh

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceObjectMesh
{
    IntPtr data;
    ulong len;
}

SliceObjectPose

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceObjectPose
{
    IntPtr data;
    ulong len;
}

SliceStereoProjections

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceStereoProjections
{
    IntPtr data;
    ulong len;
}

SliceStereoViews

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceStereoViews
{
    IntPtr data;
    ulong len;
}

SliceTrackedData

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceTrackedData
{
    IntPtr data;
    ulong len;
}

SliceTrackedWellKnownInfo

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceTrackedWellKnownInfo
{
    IntPtr data;
    ulong len;
}

SliceVector3f32

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceVector3f32
{
    IntPtr data;
    ulong len;
}

SliceVideoFrame

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceVideoFrame
{
    IntPtr data;
    ulong len;
}

SliceWellKnownTypedTrack

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct SliceWellKnownTypedTrack
{
    IntPtr data;
    ulong len;
}

Slicei32

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct Slicei32
{
    IntPtr data;
    ulong len;
}

Slicei64

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct Slicei64
{
    IntPtr data;
    ulong len;
}

Sliceu32

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct Sliceu32
{
    IntPtr data;
    ulong len;
}

Sliceu64

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct Sliceu64
{
    IntPtr data;
    ulong len;
}

Sliceu8

A pointer to an array of data someone else owns which may not be modified.

Fields

  • data - Pointer to start of immutable data.
  • len - Number of elements.

Definition

public partial struct Sliceu8
{
    IntPtr data;
    ulong len;
}

OptionBinocularGaze

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionBinocularGaze
{
    BinocularGaze t;
    byte isSome;
}

OptionConvergenceDistance

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionConvergenceDistance
{
    ConvergenceDistance t;
    byte isSome;
}

OptionEntrancePupilData

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionEntrancePupilData
{
    EntrancePupilData t;
    byte isSome;
}

OptionPupilSizeAbsolute

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionPupilSizeAbsolute
{
    PupilSizeAbsolute t;
    byte isSome;
}

OptionTimeValueBinocularGaze

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueBinocularGaze
{
    TimeValueBinocularGaze t;
    byte isSome;
}

OptionTimeValueBlinkState

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueBlinkState
{
    TimeValueBlinkState t;
    byte isSome;
}

OptionTimeValueCombinedGaze

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueCombinedGaze
{
    TimeValueCombinedGaze t;
    byte isSome;
}

OptionTimeValueConvergenceDistance

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueConvergenceDistance
{
    TimeValueConvergenceDistance t;
    byte isSome;
}

OptionTimeValueEntrancePupilData

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueEntrancePupilData
{
    TimeValueEntrancePupilData t;
    byte isSome;
}

OptionTimeValueEyeTrackerDeviceTimestamp

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValueEyeTrackerDeviceTimestamp
{
    TimeValueEyeTrackerDeviceTimestamp t;
    byte isSome;
}

OptionTimeValuePositionGuide

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValuePositionGuide
{
    TimeValuePositionGuide t;
    byte isSome;
}

OptionTimeValuePupilSizeAbsolute

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionTimeValuePupilSizeAbsolute
{
    TimeValuePupilSizeAbsolute t;
    byte isSome;
}

OptionVideoFrame

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionVideoFrame
{
    VideoFrame t;
    byte isSome;
}

OptionVideoFrameDescriptor

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct OptionVideoFrameDescriptor
{
    VideoFrameDescriptor t;
    byte isSome;
}

Optionu32

Option type containing boolean flag and maybe valid data.

Fields

  • t - Element that is maybe valid.
  • isSome - Byte where 1 means element t is valid.

Definition

public partial struct Optionu32
{
    uint t;
    byte isSome;
}

Enums

BlinkStatePerEye

Describes the openness of an eye in discrete steps. What exactly constitues and open or closed eye is hardware specific.

Variants

  • Invalid - Eye state was invalid, e.g., could not be measured.
  • Open - Eye is open.
  • Closed - Eye is closed.

Definition

public enum BlinkStatePerEye
{
    Invalid = 0,
    Open = 1,
    Closed = 2,
}

ColliderType

The type of collider used in the game engine.

Variants

  • Physical - A collider participating in physical hit tests.
  • Trigger - A mostly semantic collider for activating things.

Definition

public enum ColliderType
{
    Physical = 1,
    Trigger = 2,
}

ConnectionReadyState

Encodes if a (remote) connection is ready.

Variants

  • NotReady - We have received no (the most common case), or less than ~8 bytes or so, which is the minimum needed to identify whether we speak the right protocol.
  • HeadersReady - Headers have been read and are available.

Definition

public enum ConnectionReadyState
{
    NotReady = 0,
    HeadersReady = 1,
}

CoordinateSystem

Gaze and similar data is usually recorded relative to the eye tracker. However, some applications benefit from using world space coordinates instead.

Variants

  • World -
  • Head -

Definition

public enum CoordinateSystem
{
    World = 0,
    Head = 1,
}

FrameType

Describes what information a frame contains.

Variants

  • Unknown - Frame type we don’t handle just yet.
  • Init - Init frames must always be sent into the decoder as the first frame
  • KeyFrame - Keyframes are fully self-contained and can be submitted standalone (they still require an Init frame to be sent first)
  • DependsOnPrior - A frame depending on a previously submitted frame.

Definition

public enum FrameType
{
    Unknown = 0,
    Init = 1,
    KeyFrame = 2,
    DependsOnPrior = 3,
}

ImageFormat

Video format of this frame.

Variants

  • Unknown - Should not be used.
  • RawMonochromeU8 - 8 bit row major with 0 being black and 255 white.

Definition

public enum ImageFormat
{
    Unknown = 0,
    RawMonochromeU8 = 1,
}

LogLevel

Log level for Ocumen functions.

Variants

  • Off -
  • Error -
  • Warn -
  • Info -
  • Debug -
  • Trace -

Definition

public enum LogLevel
{
    Off = 0,
    Error = 1,
    Warn = 2,
    Info = 3,
    Debug = 4,
    Trace = 5,
}

LookedAtWith

Specified with which eye(s) the hit test should be performed.

Variants

  • LeftEye -
  • RightEye -
  • EitherEye -
  • BothEyes -

Definition

public enum LookedAtWith
{
    LeftEye = 0,
    RightEye = 1,
    EitherEye = 2,
    BothEyes = 3,
}

ShapeType

How shape data is to be interpreted.

Variants

  • Point - points.0 is the point.
  • Line - points.0 is the start of the line, points.1 is the end of the line.
  • Triangle - Trivial.
  • AABB - points.0 is one corner, point.1 the opposite corner, both must reside on the XY plane. points.0 is center, point.1 the diameter along the in x/y/z axis.
  • Sphere - points.0 is center, point.1 the diameter as (d,d,d)

Definition

public enum ShapeType
{
    Point = 1,
    Line = 2,
    Triangle = 3,
    AABB = 5,
    Sphere = 6,
}

TrackedData

Represents a ‘well known track’ contained in an Ocumen recording.

Variants

  • CameraPose - Pose of the world camera.
  • StereoViews - View matrices of the stereo cameras.
  • StereoProjections - Projection matrices of the stereo cameras.
  • FrameNumber - Game engine counter of drawn frame.
  • GazeDeviceTime - Timestamp when the data was captured using the eye trackers clock
  • GazeCombined - Combined gaze rays.
  • GazeBinocular - Binocular gaze rays.
  • GazeConvergenceDistance - Convergence distance for the gaze rays.
  • GazePositionGuide - Describes users pupil position in relation to the optical axis.
  • GazePupilSize - Size of the pupil.
  • GazeBlinkState - Is user blinking or not.
  • GazeEntrancePupil - Position of the entrance pupil.
  • Objects - Objects and their poses.
  • Meshes - Mesh data.
  • Colliders - Colliders and their poses.
  • Images - Images from various sources.
  • Gizmos - User defined gizmos for debugging and scientific analysis.
  • Videos - Video tracks, such as rendered GPU output.
  • ControllerPose - Poses of connected and enabled XR controllers
  • ControllerButtons - Boolean button states for XR controllers

Definition

public enum TrackedData
{
    CameraPose = 0,
    StereoViews = 1,
    StereoProjections = 2,
    FrameNumber = 3,
    GazeDeviceTime = 4,
    GazeCombined = 5,
    GazeBinocular = 6,
    GazeConvergenceDistance = 7,
    GazePositionGuide = 8,
    GazePupilSize = 9,
    GazeBlinkState = 10,
    GazeEntrancePupil = 11,
    Objects = 12,
    Meshes = 13,
    Colliders = 14,
    Images = 15,
    Gizmos = 16,
    Videos = 17,
    ControllerPose = 18,
    ControllerButtons = 19,
}

VideoFormat

The video format of this frame.

Variants

  • Unknown -
  • CommonVideoH264v1 -

Definition

public enum VideoFormat
{
    Unknown = 0,
    CommonVideoH264v1 = 1,
}

VideoTrack

The video format of this frame.

Variants

  • Unknown -
  • RenderedGame -

Definition

public enum VideoTrack
{
    Unknown = 0,
    RenderedGame = 1,
}

WellKnownType

Some types are ‘well known’, that is, they have a definitive bit representation Ocumen knowns about, which can be used for visualization purposes (e.g., graphs).

Variants

  • Unknown - The default representation.
  • F32 - This is an IEEE float (f32) value.

Definition

public enum WellKnownType
{
    Unknown = 0,
    F32 = 1,
}

XRInputDevice

Which controller is being described.

Variants

  • Unknown - An unknown controller we could not resolve. This should not be used unless you ran into errors resolving a controller.
  • Left - Left hand.
  • Right - Right hand.
  • Gamepad - Mostly pose-less game pad.
  • Other - A user-defined controller.

Definition

public enum XRInputDevice
{
    Unknown = 0,
    Left = 1,
    Right = 2,
    Gamepad = 3,
    Other = 4,
}

Functions

init_logging

Initializes logging.

If you want log messages this should be called exactly once, as early as possible. Otherwise log messages inside the library will not be recorded.

Definition

public static extern void InitLogging(LoggerConfiguration config);

process_log_messages

Process all outstanding log messages.

The provided callback will be invoked with all outstanding log messages that have occurred since the last invocation.

Definition

public static extern void ProcessLogMessages(LogCallback f);

version_ocumen_io_api

Returns the version of the used Ocumen IO API.

This version determines the library layer encoding actual VR scenes. If you have a mismatching data versions some field might be missing or wrong.

Most users should simply call connector.check_compatible() to perform compatibility checks.

Definition

public static extern Version VersionOcumenIoApi();

version_ocumen_recording

Returns the version of the Ocumen recording format.

This version determines how packets are transmitted ‘over the wire’. Mismatching low-level versions means recordings simply cannot be opened.

Most users should simply call connector.check_compatible() to perform compatibility checks.

Definition

public static extern Version VersionOcumenRecording();

matrix4_from_translation

Create a Matrix 4x4 with scale 1 translated to the given coordinates.

Definition

public static extern Matrix4f32 Matrix4FromTranslation(float x, float y, float z);

matrix4_from_translation_rotation

Create a Matrix 4x4 with scale 1 from the given translation, rotation.

Definition

public static extern Matrix4f32 Matrix4FromTranslationRotation(ref Vector3f32 translation, ref Quaternionf32 rotation);

matrix4_invert

Inverts a matrix, returning NaNs if not possible.

Definition

public static extern Matrix4f32 Matrix4Invert(ref Matrix4f32 original);

quaternion_from_angle_axis

Create a quaternion from the given angle around an axis.

Definition

public static extern Quaternionf32 QuaternionFromAngleAxis(float angle_deg, ref Vector3f32 axis);

Classes

WorldView

Provides a view of an recording like Ocumen Studio.

A world view is a ‘view’ into a recorded session (‘world’) that contains a simple API to select a point in time and query the world state for that moment.

ForUser ctor

Creates a view for the given user UUID.

The parameter user should correspond to the user you want to query.

Safety

Parameter user must not be 0.

Definition

public WorldView ForUser(ulong user);

ScrubTo

View the world as it appeared at the given time.

The reader must be an initialized WorldReader instance, and time a valid time stamp in that recording.

After this method returns all other methods return data as present at this point in time.

Definition

public class WorldView {
    public void ScrubTo(IntPtr reader, long time);
}

ColliderPoses

Returns poses for all colliders at this time.

Safety

The returned slice is only valid until your next Ocumen I/O call.

Definition

public class WorldView {
    public SliceColliderPose ColliderPoses();
}

EntityLabels

Returns labels for each known entity at this time.

Safety

The returned slice is only valid until your next Ocumen I/O call. In addition, you must NOT copy the returned slice, as each element also contains pointers to temporary memory. Instead, simply process the slice before your next call.

Definition

public class WorldView {
    public SliceEntityLabel EntityLabels();
}

GazeBinocular

Returns binocular (left / right) gaze data at this time.

Definition

public class WorldView {
    public OptionBinocularGaze GazeBinocular(CoordinateSystem system);
}

GazePupilSize

Returns pupil sizes at this time.

Definition

public class WorldView {
    public OptionPupilSizeAbsolute GazePupilSize();
}

GazeConvergenceDistance

Returns the gaze convergence distance at this time.

Definition

public class WorldView {
    public OptionConvergenceDistance GazeConvergenceDistance();
}

GazeEntrancePupil

Returns the entrance pupil position at this time.

Definition

public class WorldView {
    public OptionEntrancePupilData GazeEntrancePupil();
}

FrameNumber

Returns the game’s frame number at this time.

Definition

public class WorldView {
    public Optionu32 FrameNumber();
}

LookedAt

Perform a raycast / ‘gaze to object’ test with the user’s gaze.

The provided parameter is governed by enum LookedAtWithand can be used to specify what test to perform, e.g., whether either eye is sufficient, or if both eyes need to hit. The returned slice contains the ColliderID of each hit collider.

Safety

The returned slice is only valid until your next Ocumen I/O call.

Definition

public class WorldView {
    public Sliceu64 LookedAt(LookedAtWith with_eye);
}

WorldWriter

Creates .ocumen recordings.

Correctness

Unless otherwise stated all functions must be invoked with strictly increasing time stamps per type. For example, once you have written a frame number for time 100, you may only write another frame number for time 101 or later. Failure to follow this rule will lead to unpredictable (‘invalid’) recordings that visualize or report arbitrary query results.

This restriction holds ‘per type’, and other types are unaffected. For example, once you have written frame number for time 100, you may write a head pose for time 200 and then another frame number for time 150.

However, for most efficient recordings we recommend to minimize time jumps even between tracks.

FromUri ctor

Creates a recorder from a URI.

If the URI contains : it is treated as an ip:port to bind to, otherwise as a path in the local file system.

Network Streaming

Since it is easier to connect to a mobile headset from a PC than the other way around, the WorldWriter opens a port on the given interface, therefore acting as a server!

Definition

public WorldWriter FromUri(string uri);

FromCallback ctor

Creates a sans-io recorder.

When data is available the writer will invoke the given callback with a byte slice, which can then transmit the slice over network, bluetooth, … to another endpoint where it can be opened and read.

Definition

public WorldWriter FromCallback(DataCallback callback);

HandleIo

Writes cached data to files, network, a device …

Most data when passed to Ocumen I/O will be cached and compressed first before written to the underlying device. This function is where both a ‘check for sufficient data’ is performed and possibly executed.

Wheter data is actually written is unspecified, but depends on

  • the serialization profile (a realtime profile is selected for network targets, while a high-compression profile is selected for storage targets),
  • the size of the backing data struct,
  • the number of entries.

This function differs from flush in that it is more performant, but does not guarantee that all data is written.

Definition

public class WorldWriter {
    public void HandleIo();
}

Flush

Writes all outstanding data to the device.

Most users will not need to call this as flushing is handled transparently. A notable exception is towards the end of a session when you expect the application to be suddenly shut down.

Definition

public class WorldWriter {
    public void Flush();
}

FrameNumber

Stores the frame number for a user at a given time.

Definition

public class WorldWriter {
    public void FrameNumber(UserAtTime user, ref uint data);
}

HeadsetCamera

Stores the head set camera pose for a user at a given time.

Definition

public class WorldWriter {
    public void HeadsetCamera(UserAtTime user, ref CameraPose data);
}

HeadsetViews

Stores the head set stereo view matrices for a user at a given time.

Definition

public class WorldWriter {
    public void HeadsetViews(UserAtTime user, ref StereoViews data);
}

HeadsetProjections

Stores the head set stereo projection matrices for a user at a given time.

Definition

public class WorldWriter {
    public void HeadsetProjections(UserAtTime user, ref StereoProjections data);
}

GazeCombined

Stores ‘combined’ gaze for a user at a given time.

Definition

public class WorldWriter {
    public void GazeCombined(UserAtTime user, ref CombinedGaze data);
}

GazeBinocular

Stores binocular gaze for a user at a given time.

Definition

public class WorldWriter {
    public void GazeBinocular(UserAtTime user, ref BinocularGaze data);
}

GazeEyeTrackerDeviceTime

Stores the eye tracker’s device time stamp for a user at a given time.

Definition

public class WorldWriter {
    public void GazeEyeTrackerDeviceTime(UserAtTime user, ref EyeTrackerDeviceTimestamp data);
}

GazeConvergenceDistance

Stores the gaze convergence distance for a user at a given time.

Definition

public class WorldWriter {
    public void GazeConvergenceDistance(UserAtTime user, ref ConvergenceDistance data);
}

GazePositionGuide

Stores gaze position guidance for a user at a given time.

Definition

public class WorldWriter {
    public void GazePositionGuide(UserAtTime user, ref PositionGuide data);
}

GazePupilSizeAbsolute

Stores absolute pupil size information for a user at a given time.

Definition

public class WorldWriter {
    public void GazePupilSizeAbsolute(UserAtTime user, ref PupilSizeAbsolute data);
}

GazeEntrancePupil

Stores entrance pupil data for a user at a given time.

Definition

public class WorldWriter {
    public void GazeEntrancePupil(UserAtTime user, ref EntrancePupilData data);
}

GazeBlinkState

Stores blink state data for a user at a given time.

Definition

public class WorldWriter {
    public void GazeBlinkState(UserAtTime user, ref BlinkState data);
}

InstanceLifetime

Stores data whether objects / colliders are alive or not for a user at a given time.

Definition

public class WorldWriter {
    public void InstanceLifetime(UserAtTime user, SliceInstanceLifetime slice);
}

ObjectPoses

Stores object poses for a user at a given time.

Definition

public class WorldWriter {
    public void ObjectPoses(UserAtTime user, SliceObjectPose slice);
}

ObjectMeshes

Stores object meshes (geometry) for a user at a given time.

Definition

public class WorldWriter {
    public void ObjectMeshes(UserAtTime user, SliceObjectMesh slice);
}

MeshData

Stores object mesh data for a user at a given time.

At the time of writing, by convention, only the first stored mesh per object is evaluated. If you store more than one mesh per object the behavior of Ocumen Studio is unspecified.

Definition

public class WorldWriter {
    public void MeshData(UserAtTime user, ulong mesh, SliceVector3f32 vertices, Slicei32 indices);
}

EntityLabel

Stores an entity label for a user at a given time.

Definition

public class WorldWriter {
    public void EntityLabel(UserAtTime user, EntityLabelID entity, string label);
}

ColliderGeometries

Stores simplified collider geometries for a user at a given time.

Definition

public class WorldWriter {
    public void ColliderGeometries(UserAtTime user, SliceColliderShape slice);
}

ColliderPoses

Stores collider poses for a user at a given time.

Definition

public class WorldWriter {
    public void ColliderPoses(UserAtTime user, SliceColliderPose slice);
}

Gizmos3d

Stores debug gizmos for a user at a given time.

Definition

public class WorldWriter {
    public void Gizmos3d(UserAtTime user, SliceGizmo3D slice);
}

Image

Stores image data for a user at a given time.

Definition

public class WorldWriter {
    public void Image(UserAtTime user, ref ImageDescriptor image, Sliceu8 data);
}

ControllerButtonState

Stores controller button state for a user at a given time.

Definition

public class WorldWriter {
    public void ControllerButtonState(UserAtTime user, SliceControllerButtonState slice);
}

ControllerPoses

Stores controller poses for a user at a given time.

Definition

public class WorldWriter {
    public void ControllerPoses(UserAtTime user, SliceControllerPose slice);
}

VideoFrame

Stores a (rendered and compressed) video frame for a user at a given time.

Definition

public class WorldWriter {
    public void VideoFrame(ulong user_id, ref VideoFrameDescriptor frame_data, Sliceu8 data);
}

GenericDescriptor

Obtains a low-level track handle to store custom data.

Definition

public class WorldWriter {
    public void GenericDescriptor(Descriptor request, out TrackHandle handle);
}

GenericData

Store custom data for a track handle.

Definition

public class WorldWriter {
    public void GenericData(TrackHandle handle, Slicei64 times, IntPtr data);
}

WorldReader

Wraps a WorldConnector and provides methods to read XR events.

This is the main service to read Ocumen recordings. It knows about all concepts encoded in a recording and provides low-to-midlevel helper functions to extract event data.

Depending on your use case the methods in here might be enough, or you have to pass this world reader to various utils that can help you extract specific ‘views of the world’ more efficiently.

FromConnector ctor

Creates a new reader from a connector.

After calling this method the provided connector is invalidated and must not be reused.

Definition

public WorldReader FromConnector(IntPtr connector);

ReadIncremental

Read all (newly) available data.

Ocumen streams can be read from any I/O, including offline recordings from disks and live streaming from networks. This function tries to read and parse all data that is availble so far. When reading from disk it usually needs to be called once, when streaming live data from network it makes sense to call this function a few times every second.

Ultimately, this function is what makes ’new data appear’. If you open any data source and never call read_incremental the recording will look empty.

Definition

public class WorldReader {
    public void ReadIncremental();
}

SetErrorCallback

Sets a callback invoked on read errors.

As errors are rare and primariliy related to corrupt recordings, and to simplify regular calls, most methods return data instead of error codes. When such an error occurs they will return empty options or slices, and the callback provided here is invoked instead.

Definition

public class WorldReader {
    public void SetErrorCallback(ErrorCallback error_callback, IntPtr error_ctx);
}

TrackedData

Lists all track types known to exist in this recording.

Definition

public class WorldReader {
    public SliceTrackedData TrackedData(ulong user);
}

TrackedWellKnowns

List generic handles and their type information.

Definition

public class WorldReader {
    public SliceWellKnownTypedTrack TrackedWellKnowns(ulong user);
}

TrackedWellKnownsInfo

Returns meta data for track handles.

Definition

public class WorldReader {
    public SliceTrackedWellKnownInfo TrackedWellKnownsInfo(TrackHandle handle, ulong user);
}

ListUsers

Lists all users present in this recording.

Definition

public class WorldReader {
    public Sliceu64 ListUsers();
}

LatestMeshesFor

Lists the latest available mesh data for a user in a time range.

Definition

public class WorldReader {
    public SliceMesh LatestMeshesFor(UserInTimeRange meta);
}

LatestEntityLabelsFor

Lists the latest entity labels for a user in a time range.

Definition

public class WorldReader {
    public SliceEntityLabel LatestEntityLabelsFor(UserInTimeRange meta);
}

LatestInstanceLifetimeFor

Lists the latest instance lifetime information for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesInstanceLifetime LatestInstanceLifetimeFor(UserInTimeRange meta);
}

LatestObjectPosesFor

Lists the latest object poses for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesObjectPose LatestObjectPosesFor(UserInTimeRange meta);
}

LatestObjectMeshesFor

Lists the latest object meshes for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesObjectMesh LatestObjectMeshesFor(UserInTimeRange meta);
}

LatestFrameNumbersFor

Lists the latest frame number for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesu32 LatestFrameNumbersFor(UserInTimeRange meta);
}

LatestHeadsetCameraFor

Lists the latest headset camera pose for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesCameraPose LatestHeadsetCameraFor(UserInTimeRange meta);
}

LatestHeadsetProjectionsFor

Lists the latest headset stereo projections for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesStereoProjections LatestHeadsetProjectionsFor(UserInTimeRange meta);
}

LatestHeadsetViewsFor

Lists the latest headset stereo view matrices for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesStereoViews LatestHeadsetViewsFor(UserInTimeRange meta);
}

LatestGazeCombinedFor

Returns the latest combined gaze for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueCombinedGaze LatestGazeCombinedFor(UserInTimeRange meta);
}

LatestGazeBinocularFor

Returns the latest binocular gaze for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueBinocularGaze LatestGazeBinocularFor(UserInTimeRange meta);
}

LatestGazeDeviceTimeFor

Returns the latest eye tracking device data for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueEyeTrackerDeviceTimestamp LatestGazeDeviceTimeFor(UserInTimeRange meta);
}

LatestGazeConvergenceDistanceFor

Returns the latest convergence distance for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueConvergenceDistance LatestGazeConvergenceDistanceFor(UserInTimeRange meta);
}

LatestGazePositionGuideFor

Returns the latest position guide information for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValuePositionGuide LatestGazePositionGuideFor(UserInTimeRange meta);
}

LatestGazePupilSizeAbsFor

Returns the latest pupil size absolute for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValuePupilSizeAbsolute LatestGazePupilSizeAbsFor(UserInTimeRange meta);
}

LatestGazeBlinkStateFor

Returns the latest blink state for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueBlinkState LatestGazeBlinkStateFor(UserInTimeRange meta);
}

LatestGazeEntrancePupilFor

Returns the latest entrance pupil for a user in a time range.

Definition

public class WorldReader {
    public OptionTimeValueEntrancePupilData LatestGazeEntrancePupilFor(UserInTimeRange meta);
}

LatestColliderGeometriesFor

Lists the latest collider shapes for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesColliderShape LatestColliderGeometriesFor(UserInTimeRange meta);
}

LatestColliderPosesFor

Lists the latest collider poses for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesColliderPose LatestColliderPosesFor(UserInTimeRange meta);
}

LatestGizmos3dFor

Lists the latest debug gizmos for a user in a time range.

Definition

public class WorldReader {
    public TimesValuesGizmo3D LatestGizmos3dFor(UserInTimeRange meta);
}

Image

Lists the latest image data for a user in a time range.

Definition

public class WorldReader {
    public SliceImageData Image(UserInTimeRange meta, byte track);
}

LatestControllerButtonStatesFor

Lists the latest controller button states for a user in a time range.

Definition

public class WorldReader {
    public SliceControllerButtonState LatestControllerButtonStatesFor(UserInTimeRange meta);
}

LatestControllerPosesFor

Lists the latest controller poses for a user in a time range.

Definition

public class WorldReader {
    public SliceControllerPose LatestControllerPosesFor(UserInTimeRange meta);
}

VideoFramesMeta

Returns the latest video frame descriptors for a user in a time range.

Definition

public class WorldReader {
    public OptionVideoFrameDescriptor VideoFramesMeta(UserAtTime meta, VideoTrack track);
}

VideoFrames

Lists the latest video frames for a user in a time range.

Definition

public class WorldReader {
    public SliceVideoFrame VideoFrames(UserAtTime meta, VideoTrack track);
}

VideoFrameSingleOnly

Returns the latest video frame for a user in a time range.

Definition

public class WorldReader {
    public OptionVideoFrame VideoFrameSingleOnly(UserAtTime meta, VideoTrack track);
}

VideoFrameInitPacket

Returns the latest video init packet for a user in a time range.

Definition

public class WorldReader {
    public OptionVideoFrame VideoFrameInitPacket(ulong user_id, VideoTrack track);
}

GenericDescriptor

Returns a specific generic descriptor.

Definition

public class WorldReader {
    public void GenericDescriptor(string cls, string prop, ulong id, ulong user, out TrackHandle handle);
}

GenericCount

Counts how many elements are available for the given descriptor.

Definition

public class WorldReader {
    public void GenericCount(TrackHandle handle, long first_inclusive, long last_inclusive, out uint count);
}

GenericData

Loads data for a generic descriptor.

Definition

public class WorldReader {
    public void GenericData(TrackHandle handle, long first_inclusive, long last_inclusive, SliceMuti64 times, IntPtr data);
}

RecordingInformation

Return common recording information.

Definition

public class WorldReader {
    public RecordingInformation RecordingInformation();
}

TrackTimes

Lists all times for which data is available given a certain entry.

Definition

public class WorldReader {
    public Slicei64 TrackTimes(ulong user_id, TrackedData times);
}

WorldConnector

Handles stream endpoint connections to something speaking “Ocumen”.

Use this class to open a file or network connection and verify whether the endpoint has the right version. Once the connection is established pass it to a WorldReader.

FromFile ctor

Opens a file endpoint at the given path, e.g., /tmp/file.ocumen or C:\file.ocumen.

Definition

public WorldConnector FromFile(string uri);

FromMultipleFiles ctor

Loads multiple recordings as one.

Ocumen support overlaying multiple recordings into one, e.g., when dealing with multiple users, or when loading a recording and additional scientific annotations generated by Python.

Definition

public WorldConnector FromMultipleFiles(IntPtr sources);

FromNetwork ctor

Opens a network endpoint at the given TCP address, e.g., 127.0.0.1:6677.

If the additional parameter mirror_copy is given the network stream will also be stored in the specified file.

Definition

public WorldConnector FromNetwork(string uri, string mirror_copy);

ConnectionState

Returns if all WorldConnector have been loaded from the network stream.

Definition

public class WorldConnector {
    public void ConnectionState(out ConnectionReadyState ready_state);
}

AppVersion

Returns the application version embedded in this endpoint.

In most cases this should be Piglet with a version matching this library.

Definition

public class WorldConnector {
    public void AppVersion(out Version version);
}

StreamVersion

Returns the stream version of this endpoint.

Definition

public class WorldConnector {
    public void StreamVersion(out Version version);
}

CheckCompatible

Check if the endpoint is compatible with this version.

When they are compatible the function silently passes, if they are incompatible an error code is returned, translated to an exception in Python or C#.

Definition

public class WorldConnector {
    public void CheckCompatible();
}

MultipleFiles

Helper for loading combined (stacked) recordings.

New ctor

Create a new stacked recording handler.

Definition

public MultipleFiles New();

AddFile

Add a file to a stack.

Definition

public class MultipleFiles {
    public void AddFile(string source);
}