andreasjakl.com

Usually, holograms placed in real environments through Microsoft HoloLens do not have a shadow. This is mainly due to technical reasons of the HoloLens display, as I’ll discuss in a later blog post.

However, for the scene perception and to correctly determine the holographic object position in the real-world 3D space, as well as for the “feeling” that the scene looks real, shadow is of tremendous importance.

Study on Shadow in Augmented Reality

To get exact numbers and to better understand the situation, I’ve written my Bachelor Thesis “Benefits and Parameters of Shadow in Augmented Reality-Environments” in 2004. Even though back then the study was done with marker-based Augmented Reality, the general setting is comparable to scenes created with HoloLens.

Using an experimental approach, 23 participants were tasked with conducting tests and filling out a questionnaire, judging different AR situations. The tool to create the AR scenes was ShadowAReality by Stephan Drab et. al.

Why Shadow is Important for AR

Literature research showed that shadows are of significant importance for realistic perception. They allow a correct estimation of the placement of objects in the virtual direction, as well as the distance to the camera. Furthermore, shadow defines the volume of the object. Overall, shadows contain a big amount of additional information, which obviously lacks if shadows are missing. They play an important role in how realistic a scene looks.

Screenshots taken with HoloLens are often difficult to comprehend, as they only show the scene as seen by the user, but not the user itself. How is he interacting with the scene, where is he standing in relation to the view?

Therefore, photos of HoloLens apps usually show a 3rd person view. Getting a real live transmission of the 3D objects seen by a HoloLens user works with the new Spectator View by Microsoft. However, setting that up is tricky: you need 2 HoloLenses, some special hardware and in the ideal case a 3D printed mount.

A simpler approach for a still screen is to take a photo with a good quality camera, and then to render the scene with matching perspective.

Unity itself is not directly suited to rendering a scene to an image, and it also lacks some of the advanced tools to align the 3D scene with a photo.

Here, I’ll describe an approach to use 3ds Max to simulate the view of a HoloLens user in 3rd person view. I used this for our PalKranHoloPlan AR8000 project, which we created during the Palfinger Hackathon.