[Unity5 EyeX] Raycast from gazepoint on perspective camera to vector3 worldspace

[Peter Olthof]

Hi there,

I have been fiddling with this for some time now, but I can’t get it to work for perspective camera. What I want is where your gazepoint in a 3D perspective game is, there is a raycast from that point directly to the 3D world, and where it hits a mesh, it becomes a vector3 position in worldspace.

I was able to get this to work following the Spotlight example, but that is in orthographics view. The problem I think has to do with the near clip plane of the camera in perspective view.

I would love to get some help on this. This is much much more efficient than to give every object it’s own GazeAware component!

Thanks in advance!

[Peter Olthof]

Just for clarification. Here is my scene, in which a grid of GazeAware components are build as a terrain. As you can see they start overlapping eachother and not very optimal.
I would like to be able to get through one single raycast from the camera angle/pos to the exact pixel on which you gaze, a position, and I have no idea which component of EyeX I can speak to for this.

[Jenny [Tobii]]

Hi Peter,

Yes, using the GazeAware component does not scale well. They’re kind of neat as a proof-of-concept and might work well in some specific game setting. But to do something proper with 3D interactors in Unity, we (as in Tobii) would need to a kind of source integration into the Unity 3D engine that is currently not possible (for more than one reason).

Using ray casts is a way to go around this. Since eye-tracking is not pixel perfect, you are going to have to do some work to get the gaze interaction experience right. You’ll have to re-create enough of the heuristics, filtering and snapping that is built-in to the GazeAware behavior. What is a good-enough approximation of the GazeAware behavior is context-dependent from game to game. (This is one of the reasons why we do not have sample code for ray casting.).

If you are using the Spotlight sample as a basis for your experiment, I think what you might want to change to make it work in the perspective view is this:
– Change from Camera.main.ScreenToWorldPoint to Camera.main.ScreenPointToRay
– From the Ray you get from the previous method, use the ray.origin and ray.direction as the first two parameters to Physics.RayCast. The resulting hit point will be in the RaycastHit out parameter.

Here some links to sample code and further explanation of the ScreenToWorldPoint in perspective view problem:
http://docs.unity3d.com/ScriptReference/Camera.ScreenPointToRay.html
http://docs.unity3d.com/ScriptReference/Physics.Raycast.html
http://answers.unity3d.com/questions/566519/camerascreentoworldpoint-in-perspective.html (The first part of the reply is applicable here).

[Author]

Posts