[Solved] Help using TobiiEyex to click on controls in a Windows form

[Sarah]

Hi
I have the tobii eyex and I want to use it to fill a simple questionnaire form by clicking on buttons/ textboxes on a windows Form using eyes only

my intention was to try it on a simple power point form with text boxes and clickable buttons, then develop a simple VC++ program for that afterwards

I am using the windows fall creator update, please help me with these problems

1) somehow when I try the tobii eyex with the form, I can not click any button or place the cursor in the textbox.
I move the mouse fine, and when I fix my gaze on the button/textbox the windows shows me the “mouse” box where I can adjust exactly where I want to click and then when I click it doesn’t click on where I was pointing at so I can’t place the cursor in the textbox and can’t click the button.

2) I have 2 samples that can help me in my project: the Interaction_Wpf_101
which does not contain any buttons, and simply shows where I am looking on one of two rectangles

the other one is a sample from the old SDK: TobiiEyeXSdk-Cpp-1.8.498
ActivatableButtons which doesn’t work because when I try to click on the button using windows fall creator update “Mouse” feature it doesn’t click on the button, it performs a click in another position (exactly like what happens in my first problem)

So in summary: will I be able to use TobiiEyex in order to click on a button or click on a textbox in a windows form with my eye? please help

[Grant [Tobii]]

Hi @sarahhosni, certainly you can use the Tobii EyeX to click a button or textbox within a Windows Form using your eyes.. that indeed is the intention with the Interaction API within the Core SDK!

I am not sure if you are aware but we have created a series of samples to show developers how to use our Interaction API to build interactive eye tracking enabled games and applications. You can find these @ https://github.com/Tobii/interaction-samples

Here is included several methods of using eye gaze to click buttons (and other WPF components) which hopefully should guide
you in the right direction.

I can’t say I understand the issue you were describing with reference to the clicking in other position, if you can perhaps show a screenshot or video of this problem, that would be ideal.

In any event, please try these new samples and let us know if we can be of any further assistance.

[Sarah]

Hi Thank you for the reply
I downloaded Interaction samples
and I am running the “Activatable Windows form Sample”
on the form it is written: “Click the button with your eyes! Look at the button and use the Shift Key to trigger a direct click”
This is not what I am asking about, so please help me
I am asking about triggering the click using some dwelling time for example, or any other “only eye” critertia that does not involve the Keyboard

Can I achieve this using the SDK???

one solution would be to use the Windows Fall creator update “mouse” feature but unfortunately it is not working with me and I am going to send you a video showing exactly what happens but unill I do can you please let me know if I can click on the buttons or textboxes without using a key from the keyboard or not??

Thanks

[Alex [Tobii]]

Hi!

Please check this sample:
https://tobii.github.io/CoreSDK/articles/wpf.html#activatable-behavior

Specifically something like

Host.Commands.Input.SendActivationModeOn();
Host.Commands.Input.SendActivation();
Host.Commands.Input.SendActivationModeOff();

You would need to code the timer code yourself to enable dwell activation.

[Sarah]

Thank you very much
I will take a look at it
Can I ask if there is similar samples/examples for visual C++??

and for the not working feature in Windows 10 Fall creator update here is a link for a video I recorded:
https://drive.google.com/file/d/1q6PBnItbiCbalNuvdWtx69qLEJXv5_nP/view?usp=sharing

basically, I am trying to use Windows feature to click on a textbox, I choose the mouse from the Windows eyetracking toolbar on top, then perform the click, but actually the cursor after the click doesn’t appear on the textbox, it apprears on the label beside it. I try to use the keyboard to type and it doesn’t because the cursor is simply not in the textbox

I then click on a clear area to bring the next slide, which now has 2 textboxes, I try to click on the one on the right but but the click is actually performed on the left one and the cursor appears on the left one. I try to type the text qppears on the left one although i clicked on the right!

I then try to click on the “confirm button” but actually it clicks outside of it and brings the next slide…

Can somebody tell me why this feature is not working as it supposed to be?

Thanks

Sarah

[Grant [Tobii]]

Hi @sarahhosni, okay thank you for the video showing the behaviour, however as know this new eye tracking software
is developed by Microsoft not Tobii, so there is unfortunately little we can do to resolve the issue from our end assuming
your eye tracking is functioning well otherwise. I would certainly recommend posting the issue on Microsoft’s own form.

That being said, there is other third party software which offer to achieve the same results such as ‘xcessity’ which may
prove more reliable and so you can check that out.

With respect to the Interaction Samples, I am afraid the Core SDK Interaction API samples are C# only for the time being but we hope to have C++ bindings and samples available soon.

[Magnus]

I had the same issue and found a solution for my case. It seems it was the dual monitor setup I’m using. When I changed the windows display setting to only one screen it worked.

Hope that will help some and that MS will fix this issue.

Edit: Just saw that somebody already opend up a bug report for it and that Microsoft points out that tobii only supports single monitor setups. So the issue is indeed with tobii and the post here fitting.

Link:
https://microsoftaccessibility.uservoice.com/forums/307429-microsoft-accessibility-feedback/suggestions/36997702-eye-control-issue-on-two-monitor-setup

[Author]

Posts