Oculus has revealed a device kit, Passthrough API Experimental, which will make it somewhat easy to “seamlessly” merge VR with the real-world view from the Quest 2’s cameras.
Gamers can project images on flat surfaces, create combined layers that float in space, and even apply visual styles (akin to social media filters) to real scenes. Users could appreciate a virtual monitor to use with their real-world keyboard, as an example, or turn their home into a psychedelic dreamscape by flicking a virtual switch.
Oculus notified that privacy shouldn’t be a problem since the API only improves raw camera footage on-device, and apps can’t access, store or view images of the planet around you. To put it differently, a rogue app shouldn’t transmit video of your home.
Oculus anticipates submitting the framework to Unity engine creators with its next software development kit release. It will take time for finished apps to surface, but do not be shocked if mixed reality games and productivity tools become quite familiar as a result of Oculus’ new tools.