How Apple, Google, Microsoft & Facebook Are Trying To Augment Our Reality Like Never Before

During the recent iPhone X unveiling, there was a live demo involving a new game that went beyond just tapping the iPhone's screen.

AR game demo at iPhone X launch event

The guy on the right was pointing his iPhone X at the table's surface top in front of him. Sure, he was playing the game on his iPhone, but the game's environment was actually anchored on the table top. He was interacting with a virtual 3D model of the game in real-time, using the iPhone as a window into this augmented reality. Sounds incredible, right?

That's the potential of AR or augmented reality. While Apple hasn't been the first one to showcase the mad possibilities of AR, the Cupertino giant seems heavily invested in AR's future, according to recent developments.

What is augmented reality?

Unlike VR or virtual reality, which requires you to fully immerse into a virtual environment (where one or many senses are completely cut off from the real world), AR or augmented reality merely adds a few virtual layers on top of real world objects and structures, allowing you to do and experience more. Unlike VR, AR doesn't completely suspend visually-experienced reality -- things that we can see. And between VR and AR, the latter's implementation is seemingly less cumbersome from a consumer's perspective.

The marker (QR code) throws up an AR model when viewed through smartphone camera

Augmented reality applications on the smartphone generally involve two aspects so far -- the camera and a marker. The smartphone already has a built-in camera, and through it you have to scan a marker which triggers the augmented reality action.

Apple's AR initiative

Earlier this year, Apple CEO Tim Cook gave a pretty good insight into how Apple was looking at AR, when he made the following comments: "I view AR like I view the silicon here in my iPhone. It's not a product per se, it's a core technology."

That's a pretty big signal of intent directly from the horse's mouth, so to speak. And subsequent slew of announcements from Apple further underscored the importance of AR in the company's scheme of things.

Apple in June this year showcased ARKit, the company¡¯s first foray into the field of AR. The tool lets developers build AR applications for iPhones and iPads. Later in August, Ikea, AMC, Giphy, among others, gathered at Apple¡¯s Cupertino campus to show off their AR-enabled apps -- apps that they claimed were able to easily create in six to eight weeks using Apple¡¯s ARKit platform.

AR chair that you can only see through the iPhone, thanks to ARKit

For instance, Ikea¡¯s new iOS app 'Ikea Place' takes a page from developers in the past who¡¯ve tried furniture trial apps. You can open up Ikea Place on your phone, use the iPhone's camera to measure the space you¡¯re in, and then place virtual furniture to take a closer look. This allows you to walk around the piece and get a sense of its dimensions, colour scheme, etc before you finally decide to buy it.

Google's AR take, rivalry with Apple

Unlike Apple, Google's no newcomer to AR, having revealed Tango, its AR software system, for the first time back in 2014. Unlike ARKit, Google Tango requires infrared depth sensing capability on phones for AR to work, and only two current mobile phones have the technology: Lenovo¡¯s Phab 2 Pro and ASUS' ZenFone AR.

Google Tango AR on Lenovo Phab 2 Pro

Apple¡¯s ARKit uses the iPhone¡¯s existing hardware (camera and gyroscope) to achieve the same AR effect. So even though Apple has arrived a couple of years late to the AR party, compared to Google, its implementation of AR is more convenient, and therefore more advantageous for mass deployment.

But hang on, Google has also announced a brand new AR platform, called ARCore, which is separate from Tango. Like Apple's ARKit, Google's ARCore is similar in some of the underlying tech and programming. Unlike Tango, which requires sophisticated hardware, Google's ARCore -- much like Apple's ARKit -- doesn't require much else than your Android phone and its camera to deliver AR experiences.

Here's where it all boils down to, in terms of numbers: Apple has over 1 billion iPhones and iPads in use globally on a monthly basis, but there are over 2 billion monthly active Android devices. So you'd think Google already has Apple beat, that ARCore has a vast numerical advantage over ARKit already. And that would be somewhat true, if it weren't for another revealing statistic. That over 85% of Apple devices are running on its latest iOS software, whereas less than 12% currently run the latest Android OS. Fragmentation of Google's device ecosystem is its #1 bottleneck. Apple doesn't suffer from that problem. This may well decide the AR fortunes of both these giant companies.

and AR too, no doubt!

When thousands of third-party app developers are trying to develop AR experiences for several different smartphone types, building an AR ecosystem could prove challenging for Google. If fewer people will use AR apps, because of software or hardware restrictions, why would app developers make them in the first place? Apple doesn't have this problem on ARKit platform baked inside iOS 11. And because Apple makes its own hardware (iPhones, iPads) and software (iOS), and iOS' awesome adoption rate, by the time Apple's new iPhones hit the market over half a billion iOS devices are expected to be AR-ready.

In contrast, Google's ARCore can only be used on the Pixel phone and Samsung's Galaxy S8 for now, that too only if they're running Android 7.0 Nougat or above. Eventually, of course, Google will have ARCore run on millions of more Android devices from manufacturers like Samsung, Huawei, LG, Oppo, Vivo, OnePlus, ASUS and more.

But this will take months. And it's hard to not imagine Apple already seizing on the AR apps market at first.

Thoughts on Microsoft & Facebook's AR plans

There's a simple reason why AR dominance is a two-horse race between Apple and Google for now. It's because Microsoft and Facebook don't have something as ubiquitous as a smartphone.

Microsoft's only AR (or Mixed Reality) experiment is a heads-up display called the HoloLens. It's not a mass-market device right now. There's no clear cut plan defined by Microsoft for its AR or Mixed Reality initiative in the short term.

Microsoft is creating a Mixed Reality -- that we don't know much about, sadly

Facebook's Mark Zuckerberg has vaguely spoken about his plans for AR, never revealing any specifics. We know Facebook's working on thought-reading technology inside its secretive Building 8 facility at Menlo Park, we don't know if any AR products are in the works. Apart from camera AR effects on Facebook's apps -- which is too restrictive for Zuckerberg to do anything groundbreaking -- there seems very little on offer when it comes to AR. For now at least.

AR is shaping up to be a key technology trend of the near future, something that will be felt more immediately than AI or something more fancier. Therefore, a platform to more easily build them makes it that much more likely we¡¯re going to see more and more of these kinds of apps in the near future, as soon as iOS 11 and Android Oreo effectively drop. And whenever Microsoft's HoloLens and Facebook's hardware debut and become affordable.

For now, one thing is for certain. Our sense of reality will never be the same again in the near future. Fun time to be alive.