EyeX gaze data stream in Matlab

[Elias]

Hi,

I would like to import a live gaze data stream of the EyeX sensor into Matlab 2014a. I have tried two different approaches:

1. I have tried compiling a MEX-file from the sample “MinimalGazeDataStream.c”, but when I call mex, using Visual Studio 2013 as compiler, I get an error telling me that the include file EyeX.h cannot be found. While this could also be a problem with my Matlab-installation, I’m wondering if anybody tried this approach successfully?

2. In the second approach I tried to import the C library into Matlab using the “loadlibrary” command. I have searched for the header of the “Tobii.EyeX.Client.dll” or “Tobii.EyeX.Client.lib”, but have not been able to find it.

Is there any guide-line to incorporate the data stream into Matlab?

Thanks!

[Roland [Tobii]]

For 1), it seems your include paths are not setup correctly. EyeX.h is supplied by Tobii and the compiler should be able to find it.

[Anders]

Hi Elias,
the loadlibrary function is part of Matlab (which in turn uses a Windows API function with the same name), so you won’t find it in the EyeX headers. Matlab does have some good documentation and there are probably plenty of examples of people using C APIs from Matlab out there.

Both the MEX approach and the loadlibrary approach should be possible. I would probably go for the loadlibrary approach myself.

good luck!

[Pete]

I’ve made a start with the loadlibrary method. Unfortunately my ignorance of C is a bit of a limiting factor, and I’m still not 100% sure whether it is even possible to pass handlers from Matlab to C. Nonetheless, below is my first attempt, which at least gets as far as initializing the library (before I lost the plot and gave up). Perhaps somebody with more technical expertise may be able to take it further.

if libisloaded('eyex')
    unloadlibrary('eyex')
end
    
% specify paths to .dll, and .h files
d1 = 'D:\Dropbox\MatlabToolkits\Tobii\4. EyeX\TobiiEyeXSdk-Cpp-0.23.325\lib\x86'; % path to .dll
d2 = 'D:\Dropbox\MatlabToolkits\Tobii\4. EyeX\TobiiEyeXSdk-Cpp-0.23.325\include\eyex'; % path to .h files

loadlibrary(fullfile(d1,'Tobii.EyeX.Client.dll'),               ...
            fullfile(d2,'EyeX.h'),                              ...
            'addheader',fullfile(d2,'EyeXActions.h'),           ...
            'addheader',fullfile(d2,'EyeXAsyncData.h'),         ...
            'addheader',fullfile(d2,'EyeXBehavior.h'),          ...
            'addheader',fullfile(d2,'EyeXBounds.h'),            ...
            'addheader',fullfile(d2,'EyeXClientTypes.h'),       ...
            'addheader',fullfile(d2,'EyeXCommand.h'),           ...
            'addheader',fullfile(d2,'EyeXContext.h'),           ...
            'addheader',fullfile(d2,'EyeXEvent.h'),             ...
            'addheader',fullfile(d2,'EyeXFrameworkTypes.h'),	...
            'addheader',fullfile(d2,'EyeXInteractor.h'),        ...
            'addheader',fullfile(d2,'EyeXInternalLiterals.h'),	...
            'addheader',fullfile(d2,'EyeXInternalTypes.h'),     ...
            'addheader',fullfile(d2,'EyeXLiterals.h'),          ...
            'addheader',fullfile(d2,'EyeXMacros.h'),            ...
            'addheader',fullfile(d2,'EyeXNotification.h'),      ...
            'addheader',fullfile(d2,'EyeXObject.h'),            ...
            'addheader',fullfile(d2,'EyeXProperty.h'),          ...
            'addheader',fullfile(d2,'EyeXQuery.h'),             ...
            'addheader',fullfile(d2,'EyeXSnapshot.h'),          ...
            'addheader',fullfile(d2,'EyeXStates.h'),            ...
            'addheader',fullfile(d2,'EyeXSystem.h'),            ...
            'addheader',fullfile(d2,'EyeXUtils.h'),             ...
             'mfilename', 'eyexM', ...
            'alias','eyex')
% N.B. multiple warnings cannot be fixed because function pointers are not supported by loadlibrary

%% display
libfunctions eyex        
libfunctionsview eyex   

%% extract enumerated values
[methodinfo, structs, enuminfo, ThunkLibName]  = eyexM; % courtesy of http://uk.mathworks.com/matlabcentral/answers/109924-accessing-enum-value-in-dlls-header-file

%% ------------------------------------------------------------------------
% Test 1, just attempt to initialize the library

%% 1
[TX_RESULT, int32Ptr] = calllib('eyex','txIsSystemInitialized',0)

%% 2
[TX_RESULT, TX_LOGGINGMODELptr, TX_THREADINGMODELPtr, TX_SCHEDULINGMODELPtr] = calllib('eyex','txInitializeSystem',enuminfo.TX_SYSTEMCOMPONENTOVERRIDEFLAGS.TX_SYSTEMCOMPONENTOVERRIDEFLAG_NONE, [], [], [])

%% 3
[TX_RESULT, int32Ptr] = calllib('eyex','txIsSystemInitialized',0)

%% 4
calllib('eyex','txUninitializeSystem')

%% 5
[TX_RESULT, int32Ptr] = calllib('eyex','txIsSystemInitialized',0)

%% ------------------------------------------------------------------------
% try to initialize a full minimal working example, as per http://developer.tobii.com/c-sample-walk-minimalgazedatastream/
% useful links:
% http://developer.tobii.com/c-sample-walk-minimalgazedatastream/
% http://uk.mathworks.com/help/matlab/matlab_external/working-with-pointers.html
% http://stackoverflow.com/questions/2094666/pointers-in-c-when-to-use-the-ampersand-and-the-asterisk

[TX_RESULT, TX_LOGGINGMODELptr, TX_THREADINGMODELPtr, TX_SCHEDULINGMODELPtr] = calllib('eyex','txInitializeSystem',enuminfo.TX_SYSTEMCOMPONENTOVERRIDEFLAGS.TX_SYSTEMCOMPONENTOVERRIDEFLAG_NONE, [], [], [])

%%
% see EyeXClientTypes.h
% TX_EMPTY_HANDLE = 0;
TX_EMPTY_HANDLE = libstruct('txInteractionContext') % ????
TX_INVALID_TICKET = 0
TX_TRUE = 1
TX_FALSE = 0

hContext = libpointer('txInteractionContext',TX_EMPTY_HANDLE)
hConnectionStateChangedTicket = libpointer('int',TX_INVALID_TICKET)
hEventHandlerTicket = libpointer('int',TX_INVALID_TICKET)
% remember to clear these objects before rerunning: clear('TX_EMPTY_HANDLE','hContext','hConnectionStateChangedTicket','hEventHandlerTicket')

%%
[TX_RESULT, txInteractionContextPtrPtr] = calllib('eyex','txCreateContext',hContext,0) % let matlab convert to a pointer pointer (?)

%%
OnEngineConnectionStateChanged = [] % ???????
TX_RESULT = calllib('eyex','txRegisterConnectionStateChangedHandler', hContext, hConnectionStateChangedTicket, OnEngineConnectionStateChanged, [])

%....
% 
% TX_RESULT = calllib('eyex','txRegisterEventHandler(hContext, &hEventHandlerTicket, HandleEvent, NULL)')
% TX_RESULT = calllib('eyex','txEnableConnection(hContext)')

[Anders]

Pete,
thanks for sharing! I hope there is someone willing to take on the challenge…

One thing that strikes me when I see your code is that will all those callbacks, maybe a mex wrapper would be more feasible after all?

[Pete]

Thank you for your reply.

Just to confirm, it is not possible to pass function pointers from Matlab to a dll (http://uk.mathworks.com/matlabcentral/newsreader/view_thread/255164). A fully Matlab-based solution is therefore not possible.

An intermediate solution would be to wrap any calls to commands requiring function pointers in mex functions (e.g., http://www.mathworks.com/matlabcentral/answers/100602-how-do-i-create-a-function-pointer-that-points-to-an-matlab-file-using-libpointer-in-the-matlab-gene) – i.e., the mex file provides its own internal callback, which then calls in turn a Matlab callback function.

This would mean that 95% of the code is still written in Matlab, but it means compiling a mex file, which many users would find offputting.

One alternative would be to write a standalone c/c++(/mex) application that is solely responsible for interfacing with the device, and which buffers the incoming data in its own internal buffer. Whenever the user wants to, they could then manually query/poll this wrapper from Matlab, and have it return any new data.

The logic of this would be simpler from an end-user perspective, and it might also be slightly more efficient (although speed should not really be a primary concern given the relatively low sampling rate of the eye tracker)

If anybody gets round to attempting either of these things, I’d love to hear.

[Pete]

Ok, I really didn’t intend to do this, but I had a quick go at writing a mex wrapper. Turns out to be surprisingly straightforward! Below is rough proof-of-concept code, that allows gaze data and eye position data to be read into Matlab in near-real-time. All untested, and no doubt it can/should be improved and expanded, but may serve as a useful starting point for new users. If anybody uses/adapts this code then please let me know how you get on – good luck!

Matlab code (myex.m):

%% 1. compile
% - Only needs to be run on first usage.
% - Must be run in a directory containing:
%       ./eyex (subdirectory containing EyeX.h, EyeXActions.h, etc.)
%       myex.c
%       Tobii.EyeX.Client.dll
%       Tobii.EyeX.Client.lib
% - Note:
%       This *did* work with: Microsoft Software Development Kit (SDK) 7.1 in C:\Program Files (x86)\Microsoft Visual Studio 10.0
%       This did *not* work with: Lcc-win32 C 2.4.1 in C:\PROGRA~2\MATLAB\R2012b\sys\lcc 
%       (i.e., since lcc does not permit variable definition/initialisation on same line)
%       - You can change compiler using mex -setup
%       - You can download the visual studio compiler as part of the
%         Microsoft .Net dev kit (if my memory serves)
mex myex.c % compile to generate myex.mexw32, myex.mexw64, or whatever
WaitSecs(.1);

%% 2. run
% connect to EyeX Engine
myex('connect')
% allow to track for N seconds
WaitSecs(1);
% poll myex.c for any data in buffer (N.B. will return empty if eyes were
% not tracked)
x = myex('getdata');
% disconnect from EyeX Engine
WaitSecs(.1);
myex('disconnect')

%% 3. show results
close all
plot(x(:,1:2))
% print data to console
fprintf('%6.2f  %6.2f  %7.2f     %i %i  %6.2f %6.2f %6.2f  %6.2f %6.2f %6.2f  %7.2f\n',x')

% Example console output:
%     ...
%     833.48  476.04  2795213.47     1 1  -35.46 -69.89 606.63   25.79 -61.06 618.22  2795206.13
%     833.38  481.44  2795230.65     1 1  -35.86 -69.89 606.43   25.32 -60.86 618.23  2795223.11
%     827.62  486.93  2795265.50     1 1  -36.31 -69.77 605.84   24.87 -60.45 618.16  2795258.28
%     824.11  490.15  2795281.87     1 1  -36.46 -69.66 605.70   24.67 -60.23 618.15  2795274.20
%     823.07  494.03  2795296.70     1 1  -36.50 -69.40 606.18   24.67 -59.98 618.15  2795289.09
%     821.80  494.31  2795315.55     1 1  -36.48 -69.01 607.07   23.92 -62.15 605.15  2795308.19
%     ...

C code (myex.c):

/*
 * myex v0.0.1 [13/01/2015] 
 * A simple proto-toolbox to act as a middleman between Matlab and the EyeX Engine. Adapted from MinimalGazeDataStream.c.
 *
 * Incoming data is stored asynchronously (via callbacks) in a stack (implemented as a linked list).
 * The buffer can be manually polled by Matlab, at which point the buffer is cleared.
 *
 * Data is returned in a matrix. Each row contains TX_GAZEPOINTDATAEVENTPARAMS and TX_EYEPOSITIONDATAEVENTPARAMS data, thus:
 *		X (px)
 *		Y (px)
 *		EyeGazeTimestamp (microseconds)
 *		HasLeftEyePosition (0 or 1)
 *		HasRightEyePosition (0 or 1)
 *		LeftEyeX (mm)
 *		LeftEyeY (mm)
 *		LeftEyeZ (mm)
 *		RightEyeX (mm)
 *		RightEyeY (mm)
 *		RightEyeZ (mm)
 *		EyePosTimestamp (microseconds)
 *
 * This code works, but has not been extensively debugged or tested, and could no doubt be improved. It is intended
 * as a proof-of-principle only.
 *
 * 	Commands are:
 *		eyex('connect')
 *		eyex('get')
 *		eyex('disconnect')
 *
 * Copyright 2015 Pete Jones <[email protected]>
 */

#include <Windows.h>
#include <stdio.h>
#include <conio.h>
#include <assert.h>
#include "eyex\EyeX.h"
#include "mex.h"

#pragma comment (lib, "Tobii.EyeX.Client.lib")

// ID of the global interactor that provides our data stream; must be unique within the application.
static const TX_STRING InteractorId = "Twilight Sparkle";

// global variables
static TX_HANDLE g_hGlobalInteractorSnapshot = TX_EMPTY_HANDLE;
static TX_CONTEXTHANDLE hContext = TX_EMPTY_HANDLE;

/* 
 * INTERNAL DATA STORAGE -------------------------------------------------------------------------------
 */
 
// temporarily store last known eye position
TX_EYEPOSITIONDATAEVENTPARAMS lastKnownEyePositionData;

// stack data type
struct node
{
    TX_REAL X_px;
	TX_REAL Y_px;
	TX_REAL Timestamp; // For TX_GAZEPOINTDATAMODE_LIGHTLYFILTERED this is the point in time when the filter was applied. For TX_GAZEPOINTDATAMODE_UNFILTERED this is the point in time time when gazepoint was captured.
    TX_BOOL HasLeftEyePosition;
    TX_BOOL HasRightEyePosition;
    TX_REAL LeftEyeX_mm;
    TX_REAL LeftEyeY_mm;
    TX_REAL LeftEyeZ_mm;
    TX_REAL RightEyeX_mm;
    TX_REAL RightEyeY_mm;
    TX_REAL RightEyeZ_mm;
	TX_REAL EyePosTimestamp; //The point in time when the eye position was captured.
    struct node *ptr;
}*top,*top1,*temp;
int count = 0;

/* Create empty stack */
void q_create()
{
    top = NULL;
}
 
/* Push data into stack */
void q_push(TX_GAZEPOINTDATAEVENTPARAMS GazeData, TX_EYEPOSITIONDATAEVENTPARAMS lastKnownEyePositionData)
{
    if (top == NULL)
    {
        top =(struct node *)malloc(1*sizeof(struct node));
        top->ptr = NULL;
        top->X_px = GazeData.X;
		top->Y_px = GazeData.Y;
		top->Timestamp = GazeData.Timestamp;
		top->HasLeftEyePosition = lastKnownEyePositionData.HasLeftEyePosition;
		top->HasRightEyePosition = lastKnownEyePositionData.HasRightEyePosition;
		top->LeftEyeX_mm = lastKnownEyePositionData.LeftEyeX;
		top->LeftEyeY_mm = lastKnownEyePositionData.LeftEyeY;
		top->LeftEyeZ_mm = lastKnownEyePositionData.LeftEyeZ;
		top->RightEyeX_mm = lastKnownEyePositionData.RightEyeX;
		top->RightEyeY_mm = lastKnownEyePositionData.RightEyeY;
		top->RightEyeZ_mm = lastKnownEyePositionData.RightEyeZ;
		top->EyePosTimestamp = lastKnownEyePositionData.Timestamp;
    }
    else
    {
        temp =(struct node *)malloc(1*sizeof(struct node));
        temp->ptr = top;
        temp->X_px = GazeData.X;
		temp->Y_px = GazeData.Y;
		temp->Timestamp = GazeData.Timestamp;
		temp->HasLeftEyePosition = lastKnownEyePositionData.HasLeftEyePosition;
		temp->HasRightEyePosition = lastKnownEyePositionData.HasRightEyePosition;
		temp->LeftEyeX_mm = lastKnownEyePositionData.LeftEyeX;
		temp->LeftEyeY_mm = lastKnownEyePositionData.LeftEyeY;
		temp->LeftEyeZ_mm = lastKnownEyePositionData.LeftEyeZ;
		temp->RightEyeX_mm = lastKnownEyePositionData.RightEyeX;
		temp->RightEyeY_mm = lastKnownEyePositionData.RightEyeY;
		temp->RightEyeZ_mm = lastKnownEyePositionData.RightEyeZ;
		temp->EyePosTimestamp = lastKnownEyePositionData.Timestamp;		
        top = temp;
    }
    count++;
}
 
/* Pop Operation on stack */
void q_pop()
{
    top1 = top;
 
    if (top1 == NULL)
    {
		mexErrMsgIdAndTxt("eyex:buffer:indexError", "Error : Trying to pop from empty stack.");
        return;
    }
    else
	{
        top1 = top1->ptr;
		free(top);
		top = top1;
		count--;
	}
}
 
/* Return top element */
struct node *q_topelement()
{
	return(top);
}
 
/* Check if stack is empty or not */
int q_isempty()
{
	return(top == NULL);
}
 
/* Destroy entire stack */
void q_destroy()
{
    top1 = top;
 
    while (top1 != NULL)
    {
        top1 = top->ptr;
        free(top);
        top = top1;
        top1 = top1->ptr;
    }
    free(top1);
    top = NULL;
 
    count = 0;
    //printf("\n All stack elements destroyed\n");
}

/* Create empty stack */
int q_nelements()
{
    return(count);
}

/* 
 * EYEX ENGINE INTERFACE -------------------------------------------------------------------------------
 */

/*
 * Initializes g_hGlobalInteractorSnapshot with an interactor that has the Gaze Point behavior.
 */
BOOL InitializeGlobalInteractorSnapshot(TX_CONTEXTHANDLE hContext)
{
	TX_HANDLE hInteractor = TX_EMPTY_HANDLE;
	TX_HANDLE hBehavior   = TX_EMPTY_HANDLE;
	TX_HANDLE hBehaviorWithoutParameters = TX_EMPTY_HANDLE;
	TX_GAZEPOINTDATAPARAMS params = { TX_GAZEPOINTDATAMODE_LIGHTLYFILTERED };
	BOOL success;

	success = txCreateGlobalInteractorSnapshot(
		hContext,
		InteractorId,
		&g_hGlobalInteractorSnapshot,
		&hInteractor) == TX_RESULT_OK;
	success &= txCreateInteractorBehavior(hInteractor, &hBehavior, TX_INTERACTIONBEHAVIORTYPE_GAZEPOINTDATA) == TX_RESULT_OK;
	success &= txSetGazePointDataBehaviorParams(hBehavior, &params) == TX_RESULT_OK;

    // add a second behavior to the same interactor: eye position data.
    // this one is a bit different because it doesn't take any parameters.
    // therefore we use the generic txCreateInteractorBehavior function (and remember to release the handle!)
    success &= txCreateInteractorBehavior(hInteractor, &hBehaviorWithoutParameters, TX_INTERACTIONBEHAVIORTYPE_EYEPOSITIONDATA) == TX_RESULT_OK;
	
	// release the handles
	txReleaseObject(&hBehavior);
	txReleaseObject(&hBehaviorWithoutParameters);
	txReleaseObject(&hInteractor);

	return success;
}

/*
 * Callback function invoked when a snapshot has been committed.
 */
void TX_CALLCONVENTION OnSnapshotCommitted(TX_CONSTHANDLE hAsyncData, TX_USERPARAM param)
{
	// check the result code using an assertion.
	// this will catch validation errors and runtime errors in debug builds. in release builds it won't do anything.

	TX_RESULT result = TX_RESULT_UNKNOWN;
	txGetAsyncDataResultCode(hAsyncData, &result);
	assert(result == TX_RESULT_OK || result == TX_RESULT_CANCELLED);
}

/*
 * Callback function invoked when the status of the connection to the EyeX Engine has changed.
 */
void TX_CALLCONVENTION OnEngineConnectionStateChanged(TX_CONNECTIONSTATE connectionState, TX_USERPARAM userParam)
{
	switch (connectionState) {
	case TX_CONNECTIONSTATE_CONNECTED: {
			BOOL success;
			mexPrintf("The connection state is now CONNECTED (We are connected to the EyeX Engine)\n");
			// commit the snapshot with the global interactor as soon as the connection to the engine is established.
			// (it cannot be done earlier because committing means "send to the engine".)
			success = txCommitSnapshotAsync(g_hGlobalInteractorSnapshot, OnSnapshotCommitted, NULL) == TX_RESULT_OK;
			if (!success) {
				mexPrintf("Failed to initialize the data stream.\n");
			}
			else
			{
				mexPrintf("Waiting for gaze data to start streaming...\n");
			}
		}
		break;

	case TX_CONNECTIONSTATE_DISCONNECTED:
		mexPrintf("The connection state is now DISCONNECTED (We are disconnected from the EyeX Engine)\n");
		break;

	case TX_CONNECTIONSTATE_TRYINGTOCONNECT:
		mexPrintf("The connection state is now TRYINGTOCONNECT (We are trying to connect to the EyeX Engine)\n");
		break;

	case TX_CONNECTIONSTATE_SERVERVERSIONTOOLOW:
		mexPrintf("The connection state is now SERVER_VERSION_TOO_LOW: this application requires a more recent version of the EyeX Engine to run.\n");
		break;

	case TX_CONNECTIONSTATE_SERVERVERSIONTOOHIGH:
		mexPrintf("The connection state is now SERVER_VERSION_TOO_HIGH: this application requires an older version of the EyeX Engine to run.\n");
		break;
	}
}

/*
 * Handles an event from the Gaze Point data stream.
 */
void OnGazeDataEvent(TX_HANDLE hGazeDataBehavior)
{
	TX_GAZEPOINTDATAEVENTPARAMS eventParams;
	if (txGetGazePointDataEventParams(hGazeDataBehavior, &eventParams) == TX_RESULT_OK) {
		//mexPrintf("Pushing in Gaze Data: (%.1f, %.1f) Timestamp %.0f ms\n", eventParams.X, eventParams.Y, eventParams.Timestamp);
		q_push (eventParams, lastKnownEyePositionData);	
	} else {
		mexPrintf("Failed to interpret gaze data event packet.\n");
	}
}

/*
 * Handles an event from the EyePosition data stream.
 */
void OnEyepositionDataEvent(TX_HANDLE hGazeDataBehavior)
{
    TX_EYEPOSITIONDATAEVENTPARAMS eventParams;
    if (txGetEyePositionDataEventParams(hGazeDataBehavior, &eventParams) == TX_RESULT_OK) {
        //printf("Has Left Eye Position: %d\n", eventParams.HasLeftEyePosition);
		lastKnownEyePositionData = eventParams;
    } else {
        printf("Failed to interpret gaze data event packet.\n");
        printf ("Error code: %d.\n", txGetEyePositionDataEventParams(hGazeDataBehavior, &eventParams));
    }
}

/*
 * Callback function invoked when an event has been received from the EyeX Engine.
 */
void TX_CALLCONVENTION HandleEvent(TX_CONSTHANDLE hAsyncData, TX_USERPARAM userParam)
{
	TX_HANDLE hEvent = TX_EMPTY_HANDLE;
	TX_HANDLE hBehavior = TX_EMPTY_HANDLE;

    txGetAsyncDataContent(hAsyncData, &hEvent);

	// NOTE. Uncomment the following line of code to view the event object. The same function can be used with any interaction object.
	OutputDebugStringA(txDebugObject(hEvent));

	if (txGetEventBehavior(hEvent, &hBehavior, TX_INTERACTIONBEHAVIORTYPE_GAZEPOINTDATA) == TX_RESULT_OK) {
		OnGazeDataEvent(hBehavior);
		txReleaseObject(&hBehavior);
	} else if (txGetEventBehavior(hEvent, &hBehavior, TX_INTERACTIONBEHAVIORTYPE_EYEPOSITIONDATA) == TX_RESULT_OK) {
		OnEyepositionDataEvent(hBehavior);
		txReleaseObject(&hBehavior);
	}

	// NOTE since this is a very simple application with a single interactor and a single data stream, 
	// our event handling code can be very simple too. A more complex application would typically have to 
	// check for multiple behaviors and route events based on interactor IDs.

	txReleaseObject(&hEvent);
}

/* 
 * MATLAB ENGINE INTERFACE -------------------------------------------------------------------------------
 */

void Connect(void)
{

	// initialize variables
	TX_TICKET hConnectionStateChangedTicket = TX_INVALID_TICKET;
	TX_TICKET hEventHandlerTicket = TX_INVALID_TICKET;
	BOOL success;

	// create data stack
	q_create();

	// initialize and enable the context that is our link to the EyeX Engine.
	success = txInitializeSystem(TX_SYSTEMCOMPONENTOVERRIDEFLAG_NONE, NULL, NULL, NULL) == TX_RESULT_OK;
	success &= txCreateContext(&hContext, TX_FALSE) == TX_RESULT_OK;
	success &= InitializeGlobalInteractorSnapshot(hContext);
	success &= txRegisterConnectionStateChangedHandler(hContext, &hConnectionStateChangedTicket, OnEngineConnectionStateChanged, NULL) == TX_RESULT_OK;
	success &= txRegisterEventHandler(hContext, &hEventHandlerTicket, HandleEvent, NULL) == TX_RESULT_OK;
	success &= txEnableConnection(hContext) == TX_RESULT_OK;
	
	// short pause to try and ensure that the callbacks are fired and the printf messages sent to Matlab before this function terminates
	Sleep(100);
}

void Disconnect(void)
{
	// disable and delete the context.
	txDisableConnection(hContext);
	txReleaseObject(&g_hGlobalInteractorSnapshot);
	txShutdownContext(hContext, TX_CLEANUPTIMEOUT_DEFAULT, TX_FALSE);
	txReleaseContext(&hContext);
	
	q_destroy();
}

/*
 * Application entry point.
 */
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) 
{
	char *string_command;
	double *outArray;
	mwSize M; //mwSize is a platform independent alternative to int
	mwSize i;
	
	if(nrhs!=1) {
		mexErrMsgIdAndTxt("myex:nrhs:invalidN", "Invalid input: One command string required.\n\nValid commands are:\n   myex('connect')\n   myex('getdata')\n   myex('disconnect')");
	}

	string_command = mxArrayToString(prhs[0]);
    //mexPrintf(">> %s\n", string_command);
	switch (string_command[0]) {
	case 'c':
		Connect();
		break;
	case 'g':
		// retrieve (any) data from the internal buffer - return as plhs[0]
		M = q_nelements();
		i = M; // start at the end of the output array, because we're actually using a FILO data structure for the internal buffer (stack)
		plhs[0] = mxCreateDoubleMatrix(q_nelements(),12,mxREAL);
		outArray = mxGetPr(plhs[0]);
		// iterate through internal buffer
		while (!q_isempty())
		{
			// get next item from the internal buffer
			struct node *topelement = q_topelement();
			i--;
			// add gaze data to output row
			outArray[i] = topelement->X_px;
			outArray[i+M] = topelement->Y_px;  // N.B. C indexing is like the tranpose of MATLAB variable: http://uk.mathworks.com/matlabcentral/newsreader/view_thread/249954
			outArray[i+2*M] = topelement->Timestamp;
			// add eye position data to output row
			outArray[i+3*M] = topelement->HasLeftEyePosition;
			outArray[i+4*M] = topelement->HasRightEyePosition;
			outArray[i+5*M] = topelement->LeftEyeX_mm;
			outArray[i+6*M] = topelement->LeftEyeY_mm;
			outArray[i+7*M] = topelement->LeftEyeZ_mm;
			outArray[i+8*M] = topelement->RightEyeX_mm;
			outArray[i+9*M] = topelement->RightEyeY_mm;
			outArray[i+10*M] = topelement->RightEyeZ_mm;
			outArray[i+11*M] = topelement->EyePosTimestamp;
			// remove this item from the internal buffer
			q_pop();
		}
		break;
	case 'd':
		Disconnect();		
		break;		
	default:
		mexErrMsgIdAndTxt("myex:nrhs:unrecognised", "Invalid input: Unrecognised command.\n\nValid commands are:\n   myex('connect')\n   myex('getdata')\n   myex('disconnect')");
		break;
	}

    mxFree(string_command);	
	return;
}

[Anders]

Pete, you’re The Man! Again, thanks for sharing 🙂

[Gregg]

Hi all,

First of all, I’d like to thank Pete for the code posted, I’ve been working on it in order to get Matlab to stream a live eye gaze data, however, I have encountered one main problem: the code compiles fine, but the connection with the eye tracker remains in the state “TRYINGTOCONNECT”.
I’ve tried to modify the functions in the code by comparing it with the default minimalgazedatastream.c file, but have remained unsuccessful.
I think that somehow the problem lies in the MEX function and how it relates to Matlab.
I am running the code on a Lenovo laptop and Matlab is using Visual Studio 2013 as a C compiler.
Furthermore, I am using the x64 Tobii.EyeX.Client.dll and Tobii.EyeX.Client.lib files to run the code.

Thank you for your help,
Gregg

[Pete]

Unfortunately I don’t have my EyeX to hand to start digging into things. Though it might be worth noting that even when I run the code on a computer with no EyeX device physically connected, I get the following output:

The connection state is now TRYINGTOCONNECT (We are trying to connect to the EyeX Engine)
The connection state is now CONNECTED (We are connected to the EyeX Engine)
Waiting for gaze data to start streaming...
The connection state is now DISCONNECTED (We are disconnected from the EyeX Engine)

Possible differences that spring to mind are:

1. the engine version number. I’m running EyeX Version: 0.8.17.1196

2. it seems that I only tried it in 32bit Matlab (though running on a 64 bit machine). My mex file can be found (for now) here: https://www.dropbox.com/s/y3ncaefy8nykgtu/myex.mexw32?dl=0

[Pete]

A number of people have pointed out that the code I posted previous only works with older versions of the EyeX software [for the record, it was written using: Tobii EyeX Engine (0.8.17.1196), Tobii EyeX Cpp SDK (0.23.325)].

I have updated the code, which now works with the latest Tobii software [Tobii EyeX Engine (0.8.17.1196), Tobii EyeX Cpp SDK (0.23.325)]. This code, including precompiled versions of the .mex binding for both 32-bit and 64-bit Matlab can be found here (see section marked: “Tobii Eye-X Binding”).

Happy tracking!

[Agostino Gibaldi]

There is a nice Matlab implementation you can download here:
[EDIT]——–[EDIT]

Feel free of providing feedbacks and suggetions.

[Jonathan Cannon]

I tried to get this implementation up and running with my EyeX. The EyeX demo ran fine, and it was tracking my eyes. When the ‘init’ command was run, it was an unsuccessful read:

Warning: Unsuccessful read: The specified amount of data was not returned within the Timeout period.
Attempted to access DATA(1); index out of bounds because numel(DATA)=0.

Any idea how I should try to address this?

Also, I really need data in real time for what I want to do. Is Matlab not the way to go?

[Konstantin [Tobii]]

Hi!
As our license agreements for the EyeX SDK as well as the Gaze SDK do not permit storage of media (except for calibration files and data used for accuracy-enhancing filters), we cannot share this plugin on our developer-forum.
I unfortunately had to remove the link.

Please always make sure to that you have read our license agreements here:

EyeXSDK: http://developer.tobii.com/license-agreement/
GazeSDK: http://developer.tobii.com/?wpdmdl=120

/Konstantin

[Agostino Gibaldi]

There is a nice toolbox for connecting the EyeX with Matlab, which recreate the functionalities of the SDK.

You can find it on surceforge, as Matlab Toolbox EyeX, just go and search it.

Cheers.

[Author]

Posts