Hi! You shouldn’t use color key and Window capture for Gaze Overlay in OBS. Instead you should use Game Capture with “Allow transparency” enabled (see instructions). Otherwise alpha values for each pixel are ignored and overlay looks bad with sharp edges.
There always will be some trail for smooth experience.
But thanks for feedback!