Reply To: Problem with receiving interaction events

[Jenny [Tobii]]

Hi Veronika,

This is going to be another long reply… 🙂

Since you are basing your EyeX implementation on one of the WinForms samples, I guessing there are two main reasons why it doesn’t work as desired.

One reason is that your application probably is hooked up with a FormsEyeXHost that expects all interactors to be GUI components of the application itself. Another reason is something you have to add that I missed in my reply above: in order to receive queries from another application, a query handler has to be registered explicitly for that application’s process id.

Let’s start with the first reason. The idea with the Forms code in the EyeXFramework is to provide code that can take an object that inherits from Window.Forms.Control and create an EyeX interactor that correspond to the location, width, height and z ordering of that control, and also add the behaviors to the interactor that have been mapped to that control using a BehaviorMap.

In your case, as you explained, you already have code that can find and describe a GUI component or region of another application that you want to be an EyeX interactor with certain behaviors. So, what you actually need is not a FormsEyeXHost with its associated code, but rather your own implementation of an EyeX host that derives from the EyeXHost class found in the EyeXFramework namespace.

There you need to override the HandleQuery(…) and HandleEvent(…) methods. In the HandleQuery method you should create a snapshot and create and add interactors that are within the bounds of the query:

// using the method implemented in EyeXHost:
var snapshot = CreateSnapshot(query);

Then, for each region in the other application that you want to be an interactor:

var interactor = snapshot.CreateInteractor(interactorId, parentInteractorId, windowId); // interactor id has to be unique
var bounds = interactor.CreateBounds(InteractionBoundsType.Rectangular);
bounds.SetRectangularData(x, y, width, height);
interactor.Z = z; // only needed if interactor regions overlap

Then you add the behavior to the interactor. This is how you would add a “Foo” behavior to an interactor:

var behaviorParameters = new FooParams { SomeFooSpecificParam = someValue };
interactor.SetFooBehavior(ref behaviorParameters);

Finally, you should commit the snapshot:

// using the method implemented in EyeXHost:
CommitSnapshot(snapshot);

In the HandleEvent method you have to find out which interactor it concerns:
var interactorId = event_.InteractorId;

Then you loop over the behaviors and try and extract the event type and event parameters and invoke an appropriate response in the application. How to extract depends on the event type. Please, see ActivatableBehavior.HandleEvent(…) and GazeAwareBehavior.HandleEvent(…) how to do that.

Remember that all callbacks to HandleQuery and HandleEvent are called on a worker thread, so take care to dispatch to a UI thread if needed.

Now, for the other part of the problem: that you don’t receive queries for the other application. The EyeX Engine creates queries around the current gaze point of the user. It finds out which windows fall within the query bounds and looks up the process id of the process each window belongs to. If there is a registered query handler for that process, it will call the specified callback function for that handler.

In your case, you only get queries for your own application since you have hooked it up with the EyeXHost base class that registers a query handler for the current process, in other words your application’s process.

To make your application receive queries also for the other application, you have to register a query handler for the other application’s process: expose the private InteractionContext _context in EyeXHost to your derived class (by for example making it protected), and override the EyeXHost.Init() method in your derived class, where you add the registration:

public override void Init()
{
    base.Init();
    // given a string processId representing the process id of the other application:
    _context.RegisterQueryHandler(processId, HandleQuery);
}

When you get the queries from the engine, make sure you answer correctly for the window ids in the query. Remember that you could gaze-enable both your own application and the other application at the same time, by using the FormsEyeXHost and the Forms EyeXFramework code for your own application, and the EyeXHost derivative described above to enable the other application.