2D homemade GazeAware component

[Gregoire]

As Alex from Tobii said : “GazeAware component only works for 3d objects at the moment but we are planning to add support for 2d objects as well. For now you can just compare gaze point coordinates with object bounds and add a delay after switching so that it doesn’t flicker between objects that are close to each other.”
For now I am working on the solution he suggested. The code is not working and I hope someone can help me fixed that.
It seems that sprite.bounds.Contains() need a Vector3 in World Space and not Screen Space.
That’s why it’s not working for the moment. But I am sure could be worth for the community to have this code, so I am posting it hoping someone will correct it.

‘
private SpriteRenderer sprite;

void Start () {
sprite = GetComponent<SpriteRenderer>();
}

void Update () {
Vector2 _gazePoint = TobiiAPI.GetGazePoint().Screen;
Debug.Log(_gazePoint);
if (sprite.bounds.Contains(_gazePoint))
{
Debug.Log(“Bounds contain the point : ” + _gazePoint);
}

}
‘

[Alex [Tobii]]

Hi @gregorm, you can try something like this:

			Vector3[] corners = new Vector3[4];
			Button.GetComponent<RectTransform>().GetWorldCorners(corners);
			var x1 = float.MaxValue;
			var x2 = float.MinValue;
			var y1 = float.MaxValue;
			var y2 = float.MinValue;
			foreach (Vector3 corner in corners)
			{
				var screenPoint = RectTransformUtility.WorldToScreenPoint(Camera.current, corner);
				if (screenPoint.x < x1)
				{
					x1 = screenPoint.x;
				}

				if (screenPoint.x > x2)
				{
					x2 = screenPoint.x;
				}

				if (screenPoint.y < y1)
				{
					y1 = screenPoint.y;
				}

				if (screenPoint.y > y2)
				{
					y2 = screenPoint.y;
				}
			}

			var rect = new Rect(x1, Screen.height - y2, x2 - x1, y2 - y1);

[Gregoire]

Hey thank you for your reply, this is probably a clever solution, I will check that, but I was looking to something that I can use with more complexe collider as those made with polygon collider. That’s why I used bounds.Contains, I have you got something with that ? I am still not able to get around the probleme of the World space.
Ps: for those who don’t know a quick solution is simply to add a boxCollider on the sprite which then works as a 3D plane.

[Alex [Tobii]]

You can try something like bounds.IntersectRay(Camera.current.ScreenPointToRay(TobiiAPI.GetGazePoint().Screen))

Don’t forget to put some time delay before you remove/switch the focus from the object to avoid flickering because of gaze point jitter.

Something like “Don’t switch focus for next 0.5 sec if object got gaze point current frame”.

[Gregoire]

Ok I finally made it, thanks a lot for your help, you lead me on the way to find the answer.
So I tried your solution which sounded great but actually what unity made with a PollygonCollider2D.bounds.IntersectRay is that they draw a Rect around your sprite which is in the end the exact same thing as putting a BoxCollider on your sprite or use the RectCollider solution above.
So I look online to find a solution which is more a Unity problem than a Tobii, and they propose to use Collider.OverLapPoint(Vector2) // inWorldSpace.
So I needed to transform ScreenPoint given by the GetGazePoint().Screen to WorldPoint.
Camera.main.ScreenToWorldPoint() does that but you need a Vector3 so there is the solution. (I used a tricky opacity animation on gazeFocus to see the different states).

Hope it will help some of you :

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Tobii.Gaming;

public class IsColliding : MonoBehaviour {

// Use this for initialization
PolygonCollider2D myCollider;
Camera m_MainCamera;
private Color tmp;
private bool IsCollide;
private bool checkFocus = true;

void Start () {
myCollider = GetComponent<PolygonCollider2D>();
m_MainCamera = Camera.main;
tmp = GetComponent<SpriteRenderer>().color;
tmp.a = 0.49f;
GetComponent<SpriteRenderer>().color = tmp;

}

void Update () {
if(checkFocus){
float zPos = transform.position.z;
Vector3 _gazePoint = new Vector3 (TobiiAPI.GetGazePoint().Screen.x, TobiiAPI.GetGazePoint().Screen.y, zPos);
IsCollide = myCollider.OverlapPoint(m_MainCamera.ScreenToWorldPoint(_gazePoint));
checkFocus = false;
Invoke(“enableCheckFocus”, 0.5f);
}
if(IsCollide){
if(tmp.a <= 1f){
tmp.a += 0.01f;
GetComponent<SpriteRenderer>().color = tmp;
}
} else{
if(tmp.a > 0.5f){
tmp.a -= 0.01f;
GetComponent<SpriteRenderer>().color = tmp;
}
}
}

void enableCheckFocus(){
checkFocus = true;
}
}

[Gregoire]

By the way just want to thank one more time the Tobii dev team that does an amazing job responding to the community on this forum!

[Alex [Tobii]]

Great that we managed to help. I’m not sure about float zPos = transform.position.z;
z coordinate you pass into ScreenToWorldPoint is a distance from the camera.
So you may want to put correct z so that points end up being on the plane where you draw the sprites.

[Gregoire]

Oh yes you right, in my case camera position on z is always 0 so that was working but you are right it’s a good improvement!

[Author]

Posts