To make Spectator View work in our own HoloLens project, we actually have to understand how it’s working, what it is doing and how it is related to the HoloLens Sharing Experience. Turns out that there is a lot to do in our app (including transmitting custom messages) to prepare it for the full Spectator View experience!
Synchronizing Objects: Anchor & SceneManager
The Spectator View is based on the Sharing experience of the HoloToolkit, but it contains its own “fork” of the code. Instead of us having to manually decide & code which objects and interactions to send via network messages (as in the normal Sharing use case), the fork of the Spectator View transmits some data by default to our DSLR-mounted HoloLens.
However, it’s important to understand that if you use Spectator View, you also need to integrate most of the Sharing code. Essentially, a lot of code is duplicated between the two frameworks, with only slight differences in order to give the DSLR-mounted HoloLens a special role, plus the PC running Unity also needs to connect in order to correctly render the scene.