Hi Emek (@mithgroth), Calibration issues can be caused by a number of environmental factors such as proximity to high amount of infrared light (eg a window closeby), high ambient lighting, etc.
It might be worth sending us a couple of pictures of setup including the surrounding space.
Also, try in another room to see if indeed light interference is the cause.