Hi,
Great to hear that you are working on voice control integration. I think it a really interesting use case.
However, in the new EyeX 1.0.0 release, published yesterday, we have done some changes in responsibilities between the EyeX Engine and EyeX Interaction (formerly known as EyeX for Windows) that you need to know about:
- From now on, the EyeX Engine does not hook any keyboard keys at all. The EyeX Engine API just exposes the Activatable behavior with action commands to trigger activation or to toggle activation mode/activation focus on or off.
- The EyeX Interaction applications hooks one or two buttons, depending on the Direct Interaction settings. In one button mode, a quick tap on the activation key sends an activation command immediately. In two button mode, you can hold down the activation key to get activation focus and not activate until you release the key
Since the pointer teleport functionality also resides in the EyeX Interaction software and since this software does not have any API, we no not recommend to build any voice integration that is dependent on neither direct click or pointer teleport.
As I said before, I would recommend to build a little proxy program in between that:
- Registers as an EyeX Engine Client so it can receive gaze data and send action commands.
- Registers low level hooks for 3 keys: move cursor, toggle activation focus and activate.
- If a “move cursor key” is received: SetCursorPos(lastGazeX, lastGazeY).
- If a “toggle activation key” is received, turn on the activation focus via an action command.
- If an “activate key” is received, send an activation action command.
Then you can user Dragon or VAC or whatever to listen for voice commands and send key presses that your proxy program can handle.
If you take inspiration from the C code sample MinimalGazeDataStream, you can probably build such a program in a couple of hours.