- This topic has 21 replies, 8 voices, and was last updated 9 years, 3 months ago by
.
- Posts
I assume the SDK is more for custom behavior or for interaction with full-screen apps, or developing a custom gaze-based mouse controller. How usable is the EyeX out-of-the-box for regular desktop development–say, if I want to be able to move the cursor around, select autocomplete options (e.g., Intellisense), and move windows around?
Hi intelligen,
the EyeX for Windows software developed by Tobii as part of the EyeX experience adds eye-gaze interaction to the Windows environment. It lets you accelerate cursor movements and it provides eye-gaze interaction such as panning and zooming in most common Windows 8 applications. EyeX for Windows is currently not included with the EyeX Engine, but it’s likely that it will be, at least in some product offerings.
Hi, I just wanted to followup on this.
As intelligen mentioned, I would be very interested in this product for simply moving my mouse around while coding.
I could eliminate the mouse completely if I was able to eye track, and then just press a keyboard button to emulate a mouse click.
Would something simple like this be trivial to code with your SDK?
Thanks.
Hi Steve,
the API gives you the eye-gaze point on the screen in pixels, and there are functions in the Windows API that lets you set the cursor position. So yes, this should be an easy thing to do.
EDIT: While it’s certainly possible to control the cursor using the eyes, it has to be done with finesse in order to provide an enjoyable, natural user experience.
We see gaze tracking as a complement to the mouse, not a replacement. The eye is more direct and natural for pointing, but less accurate since it is never still.
Gaze clicking the way you suggest can be a good use case for eye tracking as long as the accuracy is sufficient. (The EyeX Engine improves the accuracy by letting the gaze click snap to the nearest activatable interactor within range.)
Directly controlling the cursor isn’t recommended though. Imagine trying to read on the screen when the cursor follows your gaze point all the time. Not only would it obscure the text, you’d soon want to shake off that thing that is stuck on your eyes. Go away, you cursed cursor!
Thanks for the answers. So, to clarify, if I buy the EyeX Dev Kit (the EyeX Controller featured on your front page), I can download the EyeX Engine but the EyeX for Windows software is not included and I will have to write my own mouse controller software using the EyeX Engine API. Is that correct?
How much does the EyeX for Windows software cost if I buy it separately vs. with an EyeX Controller?
Hi intelligen,
You are partly right. Right now it is not possible to download EyeX for Windows, since we are still working on some remaining features and stabilization of the software. However, when we are shipping the EyeX Dev Kit it will include a trial version of EyeX for Windows.
The limitations of the trial is still not decided, but will be communicated when we get closer to the release date. Also, EyeX for Windows will probably not be sold separately, but always be bundled with the eye tracking device.
The EyeX Engine improves the accuracy by letting the gaze click snap to the nearest activatable interactor within range
In regards to activatable interactors, can the SDK make interface controls of most existing desktop applications like Chrome and Microsoft Word, and current web applications like Google Docs and Google Maps respond to eye gaze, or is the SDK for building programs that are compatible with eye tracking from the ground up?
Thanks.
Hi @JeffKang. You are correct, the EyeX SDK is best suited to build from the ground up. You can of course use it to add EyeX interaction capabilities into existing programs, but when we release EyeX for Windows it will handle most common desktop applications, web browsers and web apps.
That’s great news. Adding interaction capabilities, and building add-ons and enhancements could be a good way to experiment before making large applications.
I saw a YouTube video that shows the interface for controlling Windows for Tobii’s PCEye. I’m guessing that “EyeX for Windows” will function similarly to that. I would totally buy the software to test it out with the EyeX Controller when it arrives.
Thanks for the info.
EyeX for Windows and Windows Control software for PCEye is quite similar. The big difference though is that in the PCEye software uses mono-modal interaction where the eyes are used both for pointing and “clicking”. In EyeX for Windows, only the “pointing” part (without moving any mouse cursor or such) is done by the eyes, while the “clicking” is done by keyboard, mouse or other explicit input method.
I’ve had some pretty fantastic success pairing the eye cursor and an xbox controller as hybrid pointing devices. My current setup (which I’m now using at work, and at home for gaming) is the Rex, with a fast damped filter and a 30 pixel dead zone moving my mouse cursor, and the xbox controller for fine adjustments once my gaze has moved the cursor close to my target. I’m not using my homebrewed foot-pedal mouse buttons yet, but that’s only because I ran out of USB ports : D
I rarely have to adjust the cursor more than an inch on-screen with the gamepad, and with this hybrid setup, my eyestrain is gone. Total programming time to add these features to the MinimalGazeDataStream sample was maybe a few hours. Short example of my playing Peggle here: http://youtu.be/ErgHfJABRGk
The Peggle video is awesome! And I really like your concept of using the eye for large movements and the joystick for small adjustments.
Check out the one I just uploaded, where I’m applying the same control scheme to play Spore. The joystick control override makes the menus effortless.
After using this scheme all day at work, I hate my laptop trackpad. The Rex + xbox controller is SO MUCH faster and less work. : D
Great video! Good that you are evaluating different ways of filtering the data and combining it with the joystick. The unfiltered data seems to work fine, but I guess it might be annoying for you to have the cursor following your eyes all the time. Would it be possible to hide the mouse cursor inside the game?
Anyways, we already look forward to your next video. It would be great if you could keep posting your progress in the Community Projects forum
Thanks! I’m just really happy to be able to do use a computer all day without my wrists screaming at me.
In these videos, when you see the cursor, it’s an in-game cursor. So in that case, it’s not possible at all to hide it.
On that note: I’ve noticed a few things about the cursor following my gaze in and out of game. It’s distracting to a different degree depending on the filtering and damping that I’m doing. If it’s bouncy and laggy, I’m tempted to follow it with my eyes, but if it’s just following where I’m looking (so, fast damping and a big deadzone) it’s not distracting at all, and in fact makes the whole pointing business feel more like looking. I had a weird adjustment phase at the end of the day yesterday when I was using a different machine and I was looking at controls and expecting the cursor to be near them already, without having to drag something around with my hands.
I’ll definitely post my experiements and result in the community forum. Thanks for the interest.
- You must be logged in to reply to this topic.