Tobii EyeX / Facetracknoir

[miket]

Hi there, I have an MSI laptop with the EyeX built in. I was wondering if I can use the EyeX for eyetracking and Facetracknoir for headtracking using the laptop camera at the same time? Has anyone done this before? Is it possible?
thx

[Grant [Tobii]]

Hi @miket, I am afraid I don’t currently have Facetracknoir to experiment with (and it’s no longer free software) but as the devices used by this software and Tobii Interaction Software are different, I do not anticipate any issue.

Are you actually trying to combine the data streams into a singular SDK solution or is the intention for gaming?

[miket]

Yes, I wanted to use it for gaming, specifically with Elite Dangerous. I was hoping it would more or less function as the tobii 4C.

[Grant [Tobii]]

Hi @miket, it may work indeed.. did you already try it out and are experiencing problems? From what I gather there should be no issue so long as the game in question can support multiple input streams. In which case, your query seems more game specific.. accordingly it might be worth asking in a dedicated Elite Dangerous Forum to see if anyone else has managed to configure your desired setup.

[miket]

I have tried it, but it seems like there is some kind of doubling effect on the video output, like two images rendered on top of one another. I’m not sure what’s causing it, but it only stopped by turning one of them off.

[Alex [Tobii]]

Unfortunately, you can’t run them in parallel using existing software. But you can code your own software using the SDKs.

[miket]

Hmm. I guess I’m out of luck as I’m not a coder by any stretch of the imagination. What would something like that involve?

[Grant [Tobii]]

Hi @miket, a cursory glance on google revealed quite a large base of users currently experimenting with FaceTrackNoir in Conjunction with Elite Dangerous. Certainly, I would recommend posting on other sites in case other users may already have some usable code that works in conjunction with Tobii.

For those users getting started in using our SDK, we have a great deal of documentation and tutorials online for free which you can read @ https://tobii.github.io/CoreSDK/

with ready to run samples downloadable @ https://github.com/Tobii/CoreSDK/tree/master/samples

[Sean Voynaud Brisson]

I’m using Tobii and FacetrackNoIR… In fact i use Tobii and TrackIR”

[Grant [Tobii]]

Hi @superseani2, thanks for stepping in.. would be useful to know if your setup in fact no allows you to control Elite Dangerous with eye tracking from Tobii and headtracking via the Webcam.

If so, I am sure @miket would appreciate the setup instructions. Thanks!

[Sean Voynaud Brisson]

This Completly Avoid the SDK BTW, you can emulate ANY, Input, with FreePie and FacetrackNoIR, WHich is GREAT! you can Basicly play RACING Gamings, in First persson View with only your Eyes and Pedals.
But This can also be VERY Game Breaking in COMPETITIVE Games and i dont want to give Details… But here it is i dont believe ANYBODY execpt Great Tobii Community will read this.
(Please Alex Make the dangerous info Confidential?, the whole post)

It sure can do anything i believe, simply using FacetrackNOir set the output to facetracknoir_UDP, set the Adress to localhost and assign an NONUSED port.(doesnt really matter). THen you can use (well i use) OpenTrack, Input as UDP Receiver. so Opentrack Can Get the Information from Tobii EyeX.
Alternatively you dont NEED OpenTrack, but Center Hotkeys i found were not working for me on FaceTrackNoIR, also Functions i needed are inside OpenTracks such as, Set Center, when looking at 50% X/Y(the center) of the screen. Doing so makes it so EVEN if the Gaze is not AS ACCURATE AS it SHOULD… It will be for OpenTrack.

Now you must set the Curves at the SAME . else you will get False Data going on, Depending on your Setup always. Then you you want HeadTracking,
— i perssonaly found it EASIER to SETUP, and Calibrate, BECAUSE alot of calibration will be needed to get Correct in-game Funcs.
— Using Python, FreePie. Which can Manipulate From the Tobii .dll themself, to TRACKIR… and even Translate one another Easily.

I can give you an Exemple of Gaze – to Virtual DirectInput. to emulate what i called Byakugan, Basicly looking to your right would turn your camera to the right, since you would be looking at Something specific , exemple : a monster, when your looking to your right at the monster, your character will mostly center at him since your eyes cant lie, if you focus on what your looking FOR. it will just center to it correctly. Now i’ve been Trying and Trying and Trying to get this working WITH the HeadTracking… But i Find it Really NON working =/. The Tobii Headtracking is Really good althoo for COCKPITS views, i gotta admit…

But To get some Extra Accuracy and an Rather finished Smoothing, I PERSSONALY, stick an Android Phone under a hat
Run Facetracknoir UDP Android app. and It just Works… Better then TrackIr. BUT i think an Synergy Between The 2 is The BEst of ALL.

Be advised tho :::

NOTICE ::::(PLEASE Apply some HOT protection under the phone as it might get VERY hot you dont want to burn your head)
:::::::::::(with an Samsung S6(underclocked a bit to get WAY LESS HOT 😀 ), i Remote control it to active the headTracking Function at will with Python scripts again)
And make a lil hole in the hat Butt, Plug that phone so it doesnt Discharge(still can play up to 3Hours Wireless)

[Sean Voynaud Brisson]

I also dont believe that ANYBODY whitout Python skills can operate this… But You know These Days (especialy Python) you might just be able to Copy Paste it from somewhere else…

Considering Python is WAY too easy to learn, Yea it might just be dangerous for alot of Games that WORK hard against Cheating, to let this Post Open to public.

[Sean Voynaud Brisson]

The problem with Cheating holes is not FaceTrackNoIR, nor the Tobii themself… the problem is python.

Simply Picture NOT Having a EyE Tracker, Simple Python could STILL use Face Recognition with In-game character Faces Assest, Draw a Square around Faces, and press a key to Move Mouse to Center of Square….

Anyway just to Picture how you can REALLY DO Anything with Python.

[Author]

Posts