Conflict WPF + Virtual Window

[Magnus]

Hi,

I’m developing a WPF app that shows one main window. The window has several gaze aware buttons. The buttons work when I’m using them them as the sole gaze aware elements.

At the same time I want to recognize gaze events in a virtual windows that covers the whole screen. This by itself works just fine as I need it too.

Now the moment I bring both together only the virtual window is working, but the WPF buttons are not recognizing any gaze events.

The important code parts are as follows:

#Simplified MainWindow.xaml.cs

public partial class MainWindow : Window
{
private Host mEyeTrackingHost = null;
private WpfInteractorAgent mWpfEyeInteractorAgent = null;
EyeAlarm mMyClass = new MyClass();

public MainWindow()
{
mEyeTrackingHost = new Host();
mWpfEyeInteractorAgent = mEyeTrackingHost.InitializeWpfAgent(“WpfAgent”);
mMyClass.StartTracking(mEyeTrackingHost);

InitializeComponent();
}

#Simplified MyClass.cs

class MyClass
{
private Host mHost = null;
private VirtualWindowsAgent mVirtualWindowsAgent = null;
private VirtualWindowBase mTrackingWindow = null;
private VirtualInteractorAgent mTrackingInteractorAgent = null;

public void StartTracking(Host _host)
{

mHost = _host;

int height = SystemInformation.VirtualScreen.Height;
int width = SystemInformation.VirtualScreen.Width;

mVirtualWindowsAgent = mHost.InitializeVirtualWindowsAgent();

mTrackingWindow = mVirtualWindowsAgent.CreateFreeFloatingVirtualWindowAsync(
“VirtualTrackingWindow”,
new Rectangle
{
X = 0,
Y = 0,
Height = height,
Width = width
}).Result;

mTrackingInteractorAgent = mHost.InitializeVirtualInteractorAgent(mTrackingWindow.Id,”VirtualAgent”);

mTrackingInteractorAgent
.AddInteractorFor(new Rectangle(0, 0, height, width))
.WithGazeAware()
.HasGaze(() => {Trace.WriteLine(“Has Gaze”); TrackingWindowHasGaze();})
.LostGaze(() => Trace.WriteLine(“Lost Gaze”));

}

#WPF buttons in MainWindow.xaml#

<Style TargetType=”Button”>
<Setter Property=”FontSize” Value=”24″ />
<Setter Property=”FontWeight” Value=”Bold” />
<Setter Property=”tobii:Behaviors.IsGazeAware” Value=”True” />
<Style.Triggers>
<Trigger Property=”tobii:Behaviors.HasGaze” Value=”True”>
<Setter Property=”Background” Value=”Magenta” />
</Trigger>
</Style.Triggers>
</Style>

<Button x:Name=”btnMyButton” Content=”<” HorizontalAlignment=”Left” Height=”62″ Margin=”253,46,0,0″ VerticalAlignment=”Top” Width=”93″/>

Help would be much appreciated.
Kind regards
Magnus

[Grant [Tobii]]

Hi @magnuslp, thanks for your query and apologies for the delay. In the WPF app, gaze aware buttons are created within a window context which makes them react to gaze.

When the virtual window is created, this resembles a window on its own which logically covers the other gaze aware elements if its Z-order is in the foreground of the associated WPF window.

Hence the buttons will stop working since they are “occluded” by the virtual window. Could you kindly explain what is the purpose of the virtual window, just to capture gaze regardless whether gazing on a button or not? “global interactors” may be a more useful strategy in this regard.

[Magnus]

Hi Grant,

thanks for the reply. I was suspection that an occlusion issue might be the cause, that’s why I tried as well to set the nonOcclusion parameter in “CreateFreeFloatingVirtualWindowAsync” to “true”, but that did not help. How would I have to set the Z value of the virtual window to be behind the the WPF window? While this is not the perfect solution for me, it would work for my use case I assume.

Global interactors sound very interesting and indeed might be what I’m looking for. I wasn’t aware of them sofar, since I did not see any documentation about them. Can you tell me a source where I can read more about them?

The reason for the virtual window is, that I want a permanent full screen eye tracking while the user operates other applications. I’m not caring though at which coordinate he is looking, but solely if he is looking at the screen or not (I’m tracking those events). The WPF window I’m solely using to allow the user to change settings of my program if he wants to, but is otherwise supposed to be minimized in the background.

Cheers
Magnus

[Grant [Tobii]]

Hi @magnuslp, apologies for the delay in reply. So it sounds by the description of your project, that a global interactor is what you need.

Essentially, this is calling a gaze aware component to the entire windows form. Whilst did not produce documentation for this for the Core SDK, you can read the concept (which applies equally) for the EyeX SDK @ https://developer.tobii.com/c-sample-walk-minimalgazedatastream/

In addition, if you have not already, please download the Core SDK samples @ https://github.com/Tobii/CoreSDK
which show a similar methodology applied for console applications which you should be able to translate over to your WPF window.

Hopefully this points you in the right direction! Please do let us know if we can provide any further assistance.

[Samuel Engelman]

I’m facing the same issue. In my case, I don’t think it’s the Global Interactor. To keep it simple, I have an application with many gaze aware buttons and controls. I want to allow the user to look at a section of the screen and have it pop up a settings panel. I also need a virtual window over the main part of the screen to help manage this functionality.
I am able to it all except for the problem that once I create the virtual window that covers the main part of the application, it blocks any other control from registering gaze. I tried setting the nonOccluding property to true and it doesn’t seem to make any difference.

[Grant [Tobii]]

Hi @magnuslp, I wonder with your recent experience in this application, you could help out Samuel? Many thanks

[Author]

Posts