In the previous blog post of this short series, I described the findings of my study Benefits and Parameters of Shadow in Augmented Reality-Environments on how shadow influences the perception of height (distance to the floor) in Augmented Reality environments. Before we get to a possible solution for Mixed Reality Scenarios with Microsoft HoloLens, let’s take a look at more interesting results of the study.
Estimation of Depth
To correctly percieve a mixed reality scene, the user also has to judge the depth (distance to the camera) of a virtual 3D object in a real scene. If the perception is off, it can destroy the mixed reality effect. In some applications where accuracy is vital, wrong perception of depth can have even more severe implications.
During this experiment, participants of the study were presented with two spheres that varied in size and depth. In random order, every scene was presented twice – once with shadows present, once without shadows.
Results of two scenes (1 & 5) showed a significant improvement of the correctness of answers when shadows were present. In these cases, the shadow clearly highlighted the virtual position of the objects, even if the size of the objects indicated different spatial relationships.
Two other scenarios (3 & 4) did not show a significant difference: two equally sized spheres were placed at different heights. Their relationship was difficult to judge no matter if shadows were present or not.
In the most tricky scenario (2), the spheres were placed in a way that is more confusing regarding both the placement, as well as the sphere size. Most answers did not correctly judge the scene, with a (non-significant) disadvantage comparing the scene with shadows to the one without. Sometimes, shadows can also be deceptive: both the small and the big sphere were positioned at the same distance from the floor and the camera; however, the bigger sphere obviously has a larger shadow; causing a few people to think it was closer to the camera.
Light Position
The previous two scenarios proved that shadows of virtual objects are important for Augmented Reality scenarios. But how accurate does the shadow need to be?
When using mixed reality glasses like HoloLens in an un-controlled environment, it is tremendously difficult to correctly estimate the actual lightning situation. The position of the real-life lightning is required to create a virtual equivalent so that the holograms can throw the correct shadows.
Therefore, the next scenario tested how accurate the shadows need to be: how precisely does the position of the virtual light have to match the real-world light?
Subjects were asked if they thought the virtual shadow was correct. The position of the light was varied both in height and distance from the camera, while the real-world light source always stayed in the same position.
Obviously, the results were best for the correct scenario where the virtual light position was at the same position as the real light.
The farther down the light source is, the longer the shadow gets. The effect is visible very quickly – while it did not make much of a difference if the virtual light was 3 cm closer to the floor than the real light, at 6 cm pretty much everyone noticed that something didn’t match in the scene.
When moving the virtual light source backwards (= away from the camera), the direction of the shadow changes. There is a steady decline in the results the farther away the virtual light is placed from the real light. The results show a certain degree of allowed inaccurracy – when the virtual light is moved by 6 cm, still half of the subjects found the shadow to be correct.
In general, the test series shows that a wrong direction of the shadow is less visible than a wrong length of a shadow. When the light source is moved 6 cm backwards (difference to original position: 6.74°), the result is highly significantly better (p < 0.001) than when it is moved 6 cm
down (5.26° difference).
Of course, it is imporant to consider that for these experiments the subjects had to take a close look at the shadows and had real-world comparison objects; in most applications, the shadow will just be a supporting visual factor that will not be the center of attention. It can be assumed that in this case, everyone will be much more error-tolerant and higher inaccuracies can be allowed.
The next part of the blog series will focus on the required realism of shadows.
Shadow in AR & HoloLens blog series: