Next week I’ll be launching my science fiction novel, Presence. It’s a book that speculates on the future of virtual reality. The title comes from a term that has been (re)popularized in recent years by VR technology. As Palmer Luckey, the inventor of the Oculus Rift, said in 2014: “Presence is the magic of virtual reality. It’s the feeling that you are truly someplace else.” I used that very quote to set the scene for my novel. Needless to say, I took some liberties in imagining what VR presence will be like in 2051. But what about the current state of VR presence, in 2016? Let’s look and see…
Presence in VR means that when you’re in a virtual environment, your brain believes it’s in a real world. Perhaps Palmer Luckey called it magic because in the current era, presence is extremely difficult to attain. Even he will acknowledge that VR headsets are early in their evolution, and there’s a lot of work to be done before presence is a persistent, reliable feature in VR.
Nuts & Bolts
For a start, presence is a tremendously hard technical challenge. Partly that’s why I didn’t go the ‘hard science fiction’ route and attempt to work out what the physics of presence will be in 2051. It’s impossible to predict, and anyway I was more interested in exploring the implications of full presence on society. In particular, what a Facebook-like corporation (the fictional Doppel) would do with it.
Zvi Greenstein, from the microprocessor firm Nvidia, gave a presentation earlier this year outlining the technical hurdles of presence. Greenstein noted that you have to fool three human senses: vision, hearing and touch. He also mentioned a fourth sensation, “behavior.” Nvidia is working on solutions, but the demo showed it’s a work in progress. There’s even more work to do by the VR hardware companies. Currently Oculus doesn’t cover touch (although that will change shortly, when its Touch controller is finally released). So at best, the Oculus Rift only fools the brain at the vision and hearing levels. Two out of five senses is hardly full presence.
There’s also a lot more to VR presence than simply replicating, for example, the sounds in a physical environment. In a May post on the Nvidia blog, Jason Paul noted that “audio can have a huge impact on presence in VR.” It’s worth reading his full explanation, to appreciate all the subtle, small things that can prevent presence:
Traditional VR audio provides an accurate 3D position of the sound source within a virtual environment. However, sound in the real world reflects more than just the location of the source. It’s changed by the physical environment as the waves move through walls and bounce off objects, creating echoes, reverberations or muffled sound. We expect these subtle changes in real life, so their absence in virtual environments subtracts from the realism.
As Paul noted in the same post, “true presence in VR must be convincing to all your senses.” It’s a huge technical challenge to account for all the subtleties of our five human senses in virtual reality.
Use Your Illusion
Even if all five senses are covered, that still may not be enough to achieve full presence. Leading presence researcher Mel Slater came up with two new terms to try and describe what’s required: “Place Illusion” and “Plausibility Illusion.” The paper in which he explained these concepts is too complex to get into here. But in a recent blog post, Kent Bye (from the excellent Voices of VR podcast) quoted this summary from researcher Richard Skarbez:
Place Illusion represents the degree of immersion that you feel by being transported to another place, and the Plausibility Illusion is the degree to which you feel that the overall scene matches your expectations for coherence.
If I may highlight one thing from that quote: presence isn’t just immersion. You often hear the word “immersion” used in VR circles, as if it’s an indicator of the quality of the experience. But presence is more than that. There’s also the “plausibility” of the scene you’re immersed in – has your brain been convinced that the scene is real, or is there nagging doubt?
Suit Up!
Let’s go back to 2051 for a moment. In my novel, I described a new kind of VR helmet that makes a direct connection with your brain in order to transport you to Doppel’s virtual world. Once again, I make no claims to this being hard science fiction. But it seemed like a viable extrapolation, that one day there’ll be a way to connect your brain to virtual reality – and achieve presence. My fictional helmet is kind of like the headjack in The Matrix, except I opted for a wireless connection to the brain.
In 2016, obviously the current crop of headsets cannot connect directly with our brains. So what is the state of the art in virtual reality hardware? Perhaps it’s a full body suit. I recently came across a VR project sporting a name that any science fiction author would be proud of: Skinterface. It’s a prototype “skinsuit” that allows you to literally step from the physical world into the virtual world. The designers, a group of students from London’s Royal College of Art, claim the suit is “capable of facilitating two way interactions with virtual objects or people.” One example could be having a virtual meeting with your work colleague across the other side of the world. Although it’s likely the porn industry will find other uses for this type of product first!
Conclusion
As this post hopefully showed, we’re a long way from achieving full presence in virtual reality. Perhaps thirty-five years away, if my novel is any indication. But I’m confident we’ll get there some day. Regardless, it’s fascinating to watch bright minds like the Skinterface team, along with companies like Oculus, Nividia, Sony and others, pursue the dream of full presence. I’m curious to see who gets there first.
Image credit: Skinterface
Presence, my science fiction novel about the future of VR, is now available on Amazon.
Each December going back to 2004, I’ve done a year in review blog post about technology. This year I’m focusing on technology trends rather than specific products. But I’ll mention many of my favorite tech products as part of the review.
In time, we may look back on 2016 as the beginning of the big shift away from mobile phones. Why? Because much of the innovation in 2016 happened in product categories like Virtual Reality, Artificial Intelligence, Automation, Wearables, and Internet of Things (IoT). In those categories, the mobile phone typically isn’t the primary device (although it’s often a supporting or connecting device – at least for now). In VR, headsets such as Oculus Rift or HTC Vive are the primary devices. With wearables and IoT, the primary device is either attached to our body or integrated into our environment. And while consumer AI is sometimes phone-based (for example, Siri), usually it’s either device-less (like IBM Watson) or a bold new type of device (like the Amazon Echo). As these trends continue to evolve, eventually we won’t need smartphones at all. But I’m getting ahead of myself. Let’s look at the trends of 2016 that started this gradual shift…
1. Virtual Reality Becomes Reality
After decades of hype, 2016 was the year that VR arrived as a consumer product. Three major VR headsets were released this year: Facebook’s Oculus Rift in March, the HTC Vive in April, and Sony’s Playstation VR in October. All helped make VR a reality this year. On the down side, we learned over 2016 that compelling VR content hasn’t arrived yet. Especially if you weren’t already an avid gamer. Also the level of presence – that feeling of truly believing you’re in an alternate reality – has a long way to go for the current crop of devices.
But at least VR is a real, commercially viable technology now. Not to mention it has inspired certain science fiction authors to speculate about where it might take us in the future.
2. Conversing With Artificial Intelligence
AI has been evolving at a steady clip for many years now, but until now there hasn’t been a breakthrough consumer AI product. In 2016 it became clear that Amazon Echo was that product; it was released outside of the US for the first time this year. The idea is that you talk to the Echo device via a voice service called Alexa (Amazon calls Alexa “the brain behind Echo”). This may prompt comparisons to the infamous movie AI, HAL 9000, in Stanley Kubrick’s movie 2001: A Space Odyssey. Luckily, so far there haven’t been any reports of a rebellious Alexa, although it did make a couple of guest appearances this year in the dark (and brilliant) tv show Mr Robot.
Alexa is a form of intelligent assistant, a product type that Google, Facebook, Microsoft and others are all exploring. A related trend over 2016 was the rise of ‘chatbots,’ which are integrated into Instant Messaging apps. What they have in common with Alexa is their conversational interface. Facebook began experimenting with chatbots this year in the Messenger app, in an attempt to catch up to its more sophisticated Asian competitors.
3. Social Media Jumps The Shark
2016 was the year “selfish media” finally went too far. Whether it was fake news, filter bubbles that prevent people from seeing (let alone understanding) other viewpoints, over-sharing people dominating our feeds, the outrage culture that permeates the media, or simply the overwhelming flow of blinkered opinions we get every day on social media… I’ve had enough. I’ve already begun to dial down my social media consumption.
It’s not all bad, of course. I still enjoy keeping in touch with family and friends on Facebook, and Twitter is useful for tracking narrowly defined interests. But social media proved in 2016 that it is not a viable news platform – at least if we want truthful and open-minded discussions. Bring back blogs in 2017?
4. Society Begins To Tackle Automation
We’ve only just started the conversation about how to transition to an economy which is heavily automated. In 2016, there were multiple warning signs of the potential impact. Take Uber, for example. The popular ride-sharing app began actively trialling driverless Uber cars in 2016. It’s likely that Uber’s driverless car fleet will eventually take the jobs of tens of thousands of human drivers – and that could easily happen within a decade. What will all those drivers do next?
This conversation is less about the technology itself, than it is about finding solutions to what automation will do to our working culture. That could mean implementing a Universal Basic Income, or people becoming more creative in how they earn an income. We don’t yet know how to deal with increasing automation. But it’s an important topic and in 2016 we, as a global society, at least started talking about it.
5. Pokémon GO & The Dawn of Augmented Reality
I couldn’t do a review of 2016 without mentioning Pokémon GO, which had an extraordinary burst of popularity over July and August. Without a doubt the killer app of this year, Pokémon GO brought Augmented Reality (AR) into the mainstream. At one point it seemed like every kid in my city was chasing cartoon characters down the street. And the irony, at least for me? Pokémon GO was a smartphone app. So mobile phones are not dead yet!
Conclusion
Ultimately, I think Pokémon GO was an outlier this year. I look back on 2016 as the year in which Internet technology went well beyond our mobile phones. Whether it was VR headsets in our lounges, Alexa in our living rooms, or driverless cars being tested on our roads, 2016 expanded the scope of what it means to be ‘online’ (or ‘invirt’ as I put it in my VR novel). I expect to see more of this expansion in coming years.
So when will the smartphone lose its status as our primary Internet device? Probably not for many years. However, perhaps this generation of teenagers will be the last to walk around with their necks craned downwards, staring at a small rectangular screen.
Lead image: Wired