Home Forums Feature Requests Full precise mouse cursor control without a mouse.

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #20330
    Aleksey
    Participant

    Perhaps the offer was already on the forum, but my knowledge of English is not enough to check it. I propose to implement full control of the mouse cursor with the help of eye tracking, head tracking and macros.
    In fact, I have already made a similar system using free software. To do this, I needed
    Soft
    1. eViacam (head tracking)
    2. key manager (setup macros)
    3. Tobii Experience
    Hardware
    1. webcam, 30 fps.
    2. your device.
    It works as follows.
    1. Warp the cursor by the button assigned in your application (right alt)

    2 In the key manager program, we set 2 scenarios for the Caps Lock key.
    2.1 If the position was not accurate. Your device does not have enough accuracy for 100 percent hits for example, in the text I need to move the cursor to a certain position… according to my feelings, the accuracy of positioning the mouse cursor when using only eye tracking is at best 1/4 and only in the middle of the screen, and not on its edges, in games such accuracy is usually enough, but I do not write about games, but about text editing (programming, journalism, writing, etc.) and then such accuracy becomes insufficient:
    -I hold down the Caps Lock key for 1 second and the key manager program through the macro enables head tracking in the eViacam application for 1.5 seconds.
    – With the help of the head tracking, during this time, I bring the cursor to the desired location.
    – And after 1,5 second, the same macro emulates a left mouse click and disables tracking eViacam.
    2.2 If the position was accurate:
    If the cursor is in the right place, then just use the second script for the capslock key. Short press capslock = left mouse click, two short presses =2 mouse clicks, and so on.

    This implementation is quite a working one, but it has its own significant drawbacks.
    1. I need an additional webcam.
    2. I need to use additional software, the terms of use of which may change significantly or it will no longer be supported on the new OS.
    3. You have to configure several different applications and few people will do this, and then if something stops working, then sit and figure it out.
    4. The most important thing. To use such a scenario, I have to use at least 2 keyboard keys. And the button assigned to the mouse warp becomes unavailable in the key manager. That is, you can assign a command to it, but it will not be executed. There is also a disadvantage that 1.5 seconds may not be enough to navigate the cursor and you will get a mouse click in an unnecessary place for you, or this time will be too much and you will have to sit still until the macro works.

    So I suggest the following implementation in your application using the example of the same capslock button.
    1. single click = single click of the left mouse button.
    2. if you hold down the button, then the cursor makes a warp to the place you are looking at and while you hold the button down, you can place the cursor in the desired position with your head, then release the button and the system disables head tracking and makes a left click.
    Thus, if you work with a large volume of text, you do not need to take your hands off the keyboard to move the print carriage in the text editor to the desired position without errors.

    In fact, in the current implementation of your software, “warp cursor by button” without precise cursor positioning and left mouse click is a useless thing. In any case, you will have to take your hand off the keyboard and take the mouse or reach for the touchpad. In my already finished and described above implementation, I clearly showed that you clearly did not realize even half of the potential of your device, even taking the shortcomings into my implementation.
    Also, my implementation allows you to solve dead zones at the edges of the display and especially in the corners, and head tracking is not tied to 1 monitor, and with the help of my implementation, I can also freely enable cursor control using the head and on the second monitor.
    Thanks for attention.

    #20334
    Hongbin Zou
    Participant

    Try Precision Gaze Mouse.
    https://precisiongazemouse.org/

    #20335
    Aleksey
    Participant

    Perhaps my English is so bad that you do not understand what I wrote. This program does not allow me to do this logic.
    if (CapsLock – pressed 500 ms){
    int count = 0
    while (CapsLock – pressed){
    if (count == 0){
    press&release F11
    count=1
    }
    }
    press&release leftButtonMouse
    press&release F11
    }
    else if (CapsLock – press&released){press&release leftButtonMouse}

    i use logic used in syntax Java
    And your program threw non-interceptable exceptions 3 times in a row, with a complete hang of all control, solved by restarting the OS. So no, that alone is enough not to use it

    #20336
    Hongbin Zou
    Participant

    I don’t think there are any problems with your English.
    The “Precision Gaze Mouse” program is not mine. I mentioned it just because it claims to do “precise mouse cursor control” in a similar way with what you described. If you are not satisfied, or it is error prone, then ignore it and stick with your own way.
    I believe tobii knows everything about the potential of their devices. The reason they don’t add those features is not on the tech side.

    #20337
    Aleksey
    Participant

    And in General… a few hours of dancing and learning the script language called AutoHotkey… they let me write this script.

    CapsLock::
    Send {F11 down}{F11 up}
    KeyWait CapsLock
    Send {LButton down}{LButton up}
    Send {F11 down}{F11 up}
    return

    CapsLock:: setup capslock hotkey and push it key
    Send {F11}{…} send command to eViacam to enable tracking via webcam
    KeyWait CapsLock – wait release CapsLock. while you hold CapsLock headtracking work and script wait
    Send {LButton}{…} – send emulated click left button mouse, after release CapsLock
    Send {F11}{…} – send command to eViacam to disable tracking
    return – If there is no caller to which to return, Return will do an Exit instead.

    Create 1 script file, copy, paste, save, close, run, and enjoy. The script works until the PC restarts, there seems to be an opportunity to do autorun, but I’m too lazy to understand, and I use sleep mode, so this problem does not worry me. If you fast make double click CapsLock and don’t move your head (deadzone setup in eViacam), then it is like double click mouse.

    The only thing that gets in the way, the warp button and the enable headtracking button are still different buttons, since Toby Experience intercepts the control of the warp button and does not allow it to be used together. And i not tested it in game, when control may be intercept

    #20341
    Grant [Tobii]
    Keymaster

    Hi @fallgeratoor, thanks you so much for sharing your project and describing it for us, if you have any links that might be useful for other users to try it out that would be great.

    I was not sure if you were looking for assistance, however we can say that mouse emulation and computer control overall is handled by our sister company Tobii Dynavox which specialised in this area, which is why at Tobii Tech we have not prioritised these features but instead have placed our focus on gaming.

    Should you have any specific questions for us, please do let us know. Best Wishes.

    #20346
    Aleksey
    Participant

    Rather than questions, they are error messages and suggestions.
    1. The warp key stops working if you change the input language (I checked it on the Russian layout, the right alt button).
    2. I also noticed the dead zones at the edges of the screen, and at this point I thought, why not make the screen physical 1920*1080, and the actual/virtual 2000*1200 (random numbers only indicate a larger size), when you bring your eyes to the edge of the display, the image on it shifts and a hidden area opens (in fact, the same thing that you do in elite dangerous, when the player looks at the edge of the screen and his character sees the side control panels). And in this hidden area, you can place the same browser bookmarks, music management, program list, start panel, and so on. Moreover, everything that was in the dead zone will be shifted to the center of the screen and will be in the right place, you only need to correctly configure the timings and trigger zones.

    Tobii Dynavox costs approximately $ 3,000, and eye tracking costs $ 250… maybe in normal countries, 3 thousand dollars is not very much, but alas, Russia is not one of the normal countries and the saddest thing is that it does not strive to become one. You will never see such a number of disabled people on the streets, because they simply can not leave the house, and not because we do not have them. Therefore, the question is how a disabled person can earn 3 thousand dollars in the country, when the environment is not calculated for him, despite the fact that the average salary in the country is about 250 dollars after taxes and without taking into account mandatory payments (electricity, Internet, heating, etc.).
    I think the reason is somewhere in this area

    #20373
    Aleksey
    Participant

    On the first fault. I figured out what was going on, some “overmind” (sarcasm) thought that in the Russian layout, when you click on the right alt (I had a warp on this key, because I don’t use it anywhere else), the right alt is not pressed, but 2 keys are pressed: LCtrl+RAlt. In English layout only RAlt pressed… my face twisted when i understand it. Sorry, I need to calm down, because I don’t have any censored words. Palm-face

    #20379
    Grant [Tobii]
    Keymaster

    Hi Alexsey, yes I understand your frustration with scenario on pricing naturally however there are a few user built solutions for mouse control, etc that should hopefully close this gap for you. Unfortunately, from an official Tobii perspective, we must allow dynavox to specialise in this area to avoid competing directly within this sector.
    Hopefully, you can find a solution to your needs at any rate.

    #20382
    Aleksey
    Participant

    Okay, since this has nothing to do with you, let’s focus on the games.

    Add the ability to throw the cursor warp in 3d shooters. For example, this is done in Dying Light, where when shooting through the sight, it automatically points to the place where the player is looking. In fact, you do not need such difficulties as are made in the same Division, where the interface is highlighted, hiding places, and so on. For example, in the same Battlefield, eye-pointing will be a sufficient degree of integration for this type of game when you click the right mouse button. The main thing is that this is not perceived as a dishonest game, because playing for the same sniper at close range, quickscopes will be much easier to perform, on the other hand, ping in 4 ms and 44 ms is also a real and legal cheat. You have this feature disabled in game, moreover, it is disabled even when the mouse cursor disappears in full-screen mode when watching a video, which is annoying. And it is bug IMHO.

    You have 2 functions in the settings now – warp by mouse movement and warp by pressing a button in the OS. I suggest adding 3 variant – warp mouse settings in the game when you click on the right mouse button, as in the dying light.
    By doing so, you will remove the restriction of integration into games that your device supports, and limited integration is a strong stopping factor when buying.

    #20397
    Grant [Tobii]
    Keymaster

    Thanks for your feedback Aleksey, yes we are aware that mouse warp is disabled for full screen application which is designed to avoid interference with a games own eye tracking configuration, however, I will pass along your comments to the development team for consideration.

Viewing 11 posts - 1 through 11 (of 11 total)
  • You must be logged in to reply to this topic.