Audio is everyhere, but audio augmented reality is relatively nascent. In February 2017, Tim Cook, the CEO of Apple, gave an interview to the Independent in which he described Augmented Reality (AR) as the next big idea, he likened it to the smartphone. And since then AR became a passion project for Apple.

“I think AR is that big, it’s huge. I get excited because of the things that could be done that could improve a lot of lives. And be entertaining” said Tim Cook.

Investors seem to agree. According to a recent report by CB insights, over 3,600 startups in 70+ countries have raised $66B since 2013 while the second quarter of 2019 set the record in AR funding activity with $7.4B invested in one single quarter. That’s the side of the start-ups. 

In addition, investments and acquisitions by tech giants such as Google, Apple, Amazon, Facebook, related to AR intensified the already competitive landscape. While many are trying to bring products and innovations to the market in record speed, the consumer appetite appears relatively weak thus far. Unless we count Pokemon and Snapchat, of course, both of which are successes of AR.

Audio Augmented Reality: Is the future in hearables?

But where is “the next big thing” in AR in the consumer space?

One vertical of AR which deserves attention is audio augmented reality. In the summer of 2018, Fast Company writer Peter Burrows wrote a very enticing article about hearables titled “The future is ear: Why hearables are finally tech’s next big thing”. Which begs a question: what needs to be in place so that audio augmented reality becomes ubiquitous?

“For a technology to become ubiquitous it needs to be so pervasive that we already stopped paying attention to it.” 

Only when a product joins the background, becomes invisible and part of our daily habits, we can claim it became ubiquitous. Who notices TVs anymore? Or smartphones? Or toothpaste, for that matter? But take them away for a day and see how we react.

Many people dream of turning off their smartphones and going away to vacation for a week, but who really dares to stop using it for even a day let alone a week. Do the benefits of being off the grid outweigh the benefits of connectedness?

Although it’s tempting to say yes, for many, the answer would be no. We love our smartphones, most of us truly need them. It’s hard to imagine a life without them.

Most don’t think of toothpaste as a technology product, although it surely is. Consider not having access to toothpaste for a few days. Wouldn’t it create discomfort, many uncomfortable moments, let alone health issues? It stays in the background and most of us only think about it when we’re in the grocery store or pack for trips.

“Toothpaste has become so pervasive, we don’t even think much about it although we use it 3 times a day.” 

How To Make Audio Augmented Reality Ubiquitous

There appears to be three criteria for a product to reach massive adoption: delivering on affordability, finding the “killer app”, and creating an ecosystem focused on the end-user experience. For a product to become ubiquitous, it has to meet all three criteria. Products that fail to meet one of the three, will find it difficult to be pervasive.

Let’s take a look at a recent example: the Google Glass. When it was first introduced, a pair of glasses cost $1,500 — not a number most folks could part with easily. 

A product priced at above $499 price range will either have to be truly spectacular or fall into the “can’t live without” category right away. 

Otherwise, it will have a hard time finding massive adoption. This is true for most technology products today.

By way of example, the iPhone was spectacular when it was first introduced and could claim that spot despite its high price. Many products aren’t spectacular, therefore would have a hard time selling to millions above an introductory price of $499. We know that Google Glass failed the affordability criteria. But was it a spectacular product?

Not quite, because the Google Glass couldn’t prove that it was necessary for the segment it was positioned for.

Harvard professor Theodore Levitt once said, “people do not want a quarter-inch drill, they want a quarter-inch hole.” People buy products to get something specific done in a specific circumstance. This can be to “entertain oneself while riding the subway in New York” for which alternatives could range from listening to music, reading a book, or playing a game on a smart phone. If the need is going from point A to B, the alternatives could well be walking, biking, driving, or taking a cab or Uber. Most bicycle makers don’t consider themselves competing against Uber, but in essence they do, as long as they make city bikes.

Jobs To Be Done: Start with A True Customer Need

Strategists and marketers call this “the job to be done” and design products thinking about the jobs consumers are trying to get done and the circumstances they are in.

Yet, companies are still advertising features vs. benefits or experience. Although some people do, most of us don’t buy a product simply for its features or because it’s cool. 

Cool is fine but is not sufficient to make a product ubiquitous. 

Therefore, spectacular should be what innovators should aim for. It turns out, companies that internalize and apply “the job to be done” theory to design and launch spectacular products are on average more successful than their competitors.

Going back to Google glass…

What jobs were consumers struggling with for which the Google Glass could truly help them get done and be unique about it?

The product’s basic functions were quick capture of footage and immediate display of real-time information on a glass in front of one’s eyes. It turns out that these weren’t things that most of us were struggling with. Very few need immediate display of real-time information, as a matter of fact, most of us are perfectly fine with looking at our smartphones on and off. Besides, a built-in camera on the glass was in many instances, not a desirable feature due to privacy issues. Therefore, one could argue, that the original idea to position this product as a consumer product was destined to be a failure. 

But is this the end of the story? Actually, it isn’t.

Google recently shifted its strategy to meet the needs of the enterprise market, because that’s where the need is obvious and defensible. The enterprise version of the AR glasses is now in the market at a price tag of $999, a price out of reach for the average consumer but affordable for the enterprise customer. 

The product competes for use cases where having a camera and real-time information augments the user and helps the user perform certain tasks and jobs faster, cheaper and better. These are tasks that happen to be in places like a warehouse facility, think about picking, packing and shipping boxes. Also, of interest to the enterprise version are remote locations such as maintaining expensive equipment that requires expertise, e.g. oil rigs, factory equipment, aerospace that require specialized people to go travel to these locations which can be extremely costly. 

The business case for each of these use cases is quite clear with all the productivity and cost savings one might imagine delivering. Increasing pick rates and accuracy by 10% can immediately hit the bottom line and end up delivering multiples of the $999 spent for a connected Google Glass.

Strategically, it’s easy to see that AR glasses have a huge opportunity in enterprise applications. Of course, Google is not the sole contender. If Google can pull it off, that is selling equipment to the enterprise buyer which is not necessarily Google’s core competency, this strategy might work. This will require creating an ecosystem which is the third criteria of pervasiveness. Time will tell.

Building Ecosystems is Critical

“Building the ecosystem that supports turning a product into an experience is arguably the most arduous task for a company launching an innovation.”

It takes time, resources, craftiness, meticulous execution and sometimes creation of a new organization. Apple has been the master in this, Netflix demonstrated it can do it well, and Peloton is on its way to turn their spectacular product into what I consider one that we “can’t live without”, a truly immersive experience using ecosystems.

Think about the App Store concept. The App Store didn’t simply happen because Steve Jobs was a genius and came up with this idea one day. It happened through several strategic choices that Jobs made, and meticulous execution that allowed Apple to focus on the right things and build an ecosystem that could help them turn a product into an experience. 

In 1997, Jobs decided to put a “bullet in the head of” many innovations that Apple engineers were working on that were taking the company into many different directions “doing arguably interesting things”. 10 years later, although first opposed to the idea of developing an SDK for 3rd party app developers, Jobs found his killer app in the App Store concept and built it starting in 2008 with smart partnerships, investments, and execution turning it into an ecosystem with tens of thousands of partners and creators.

Today, it’s not the Apple smartphone which customers want, it’s the App Store experience that the ecosystem delivers. And that’s what makes the iPhone ubiquitous. With the App Store, Apple was able to outsource the creative innovation to the crowd while maintaining the role of the curator and quality controller and building a two-sided platform and evolving the software and the business models that powered it.

“The most difficult part for making audio AR, or hearables, ubiquitous is going to be building the experience ecosystem focused on a real customer pain point.”

Most companies in audio AR have solved the price equation. Hearables in the market today can be in-ear computers like those envisioned and introduced by Noah Kraft and his company Doppler Labs in 2016. Although Doppler Labs stopped operations, there are many players in the market today offering similar devices such as Bose, Apple, Google, Skullcandy, Sennheiser and others.

It turns out, hearables don’t have to be in your ear for you to hear it either. They can also be shades. With a brilliant move, that they call an audio-first approach to AR, Bose introduced a new product in 2018, called Bose Shades. Bose Shades are AR sunglasses one can wear. Moreover, they are available for everyone to enjoy at a reasonable price tag of $199. I should confess that I’ve been testing my Bose Shade for over a month, and love wearing it almost every day. 

Bose Shades taps to the fundamental idea behind hearables, which is that one can use sound, or audio, as a layer of additional information to augment what the consumer may be hearing and experiencing real-time in a location. Besides, these devices are also connected to GPS and have sensors, hence track location, movement and therefore can provide location-specific information delivered in audio and change it based on one’s movements. Doing so, they can significantly enhance one’s experience and capabilities. The beauty of having the AR interface leveraging the ear vs. eyes is that it allows the user to experience full immersion. And they don’t have a camera which not only reduces the cost but also the issues that Google faced with their product.

If price is not an issue, how about the customer “job-to-be-done”? 

For which unique circumstances can location-aware audio can become a necessity? What can be the killer-app?

In 2012, I launched a business that built an innovative mobile app called SoUL. SoUL was short for Sounds of Urban Life. The core idea behind SoUL was that it would deliver audio augmented reality narratives to explorers that wanted to visit a neighborhood in a city listening to and be guided by a location-based sound narrative. What we ended up creating was a much better product than an audio walk, or a guided walk, so we called these sound narratives cinematic walks. We learned, that audio doesn’t only have the power to provide a truly immersive experience, but it also allows one to imagine and fill the gaps, as well as having one’s hands free.

Despite its coolness factor, and the interest we got from users, we learned that monetization of an innovative product like that was quite difficult. It may have been too early for the market, but it wasn’t delivering on expectations so I shelved the idea. In the following years, others entered the market with similar products focusing on different parts of the world, the biggest being Detour which Bose acquired last year.

Travel is certainly an interesting use case, but is it the killer app? 

I would argue it’s not, because walking with an audio narrative, although it’s amazing, is not a necessity unless the user belongs to a special segment, one that is visually impaired.

At the same time, that doesn’t mean that it should not be part of an audio-first strategy and ecosystem. It absolutely should, because there is a role for entertainment in an ecosystem and a product like this would be unique. Yet, there needs to be recognition that it would take time to change user habits, and this type of product may just be considered cool, until it becomes the way to travel which might take years and changing habits.

A potential killer app in the audio-first market? 

One obvious vertical to think about is inthe hearing and visually impaired segment which offers an immediate pain point that is real and is growing with an aging population, certainly in the Western world.

According to Alango, an Israeli company formed in 2002, there are 600 million people with hearing impairment. The company claims that 90% don’t use a device to enhance it. Those that do, have to go through a lot of friction and cost to get a hearing device that needs to be tuned up or work with an OTC alternative. That leaves 60 million people who are aware and are doing something about it, and the majority with an opportunity for action.

Alango’s product, BeHear® NOW, sold at $249, is a personalized hearing enhancement that works over Bluetooth® and delivers stereo in-ear audio. It helps people with tinnitus, which is the perception of noise or ringing in the ears, hear audio in noisy environments that would otherwise be difficult for them to hear. Bose also has a product called Hearphones in this vertical that retails at $499.95. In 2019, Apple introduced Live Listen turning the smartphone or tablet into a listening device and connecting it to AirPod or made-for-iPhone device to help amplify sound.

Yet, what appears not fully available in the market is a navigation system and platform that can help people with impairment find their ways in a crowded environment such as driving a car, riding a bike or walking in heavy traffic and finding direction. And do that in a way that is experiential.

Advancing this thought experiment to include visual impairment might potentially lead one to “the killer app”. Considering that there are 220 to 280 million visually impaired people in the world, the total market opportunity including audio impaired is a little shy of a billion. This might be interesting enough for a newcomer or an incumbent to dedicate resources and focus on solving a big problem.

Making the Unseen Accessible Through Audio Augmented Reality

Aira, a company with a mission “to make the unseen accessible to all”, appears to be getting close to have found the “killer app”. Aira helps visually impaired navigate by connecting them to a live agent that sees what they see using Google Glass or Bose Shade with a smartphone and guide them close to real-time. The company blends assistive wearables technology, Artificial Intelligence and Augmented Reality to help people to perform tasks that they can hardly do on their own and be independent. Now, that sounds like a killer app idea, doesn’t it? At the same time, Aira needs live agents, and a monthly subscription that starts at ~$100 per month, which is reasonable for the US market but expensive for many other markets. Clearly, that’s friction that one should consider taking out.

For Aira, or a service like Aira, to become “self-sustaining”, they will need to build true AI that doesn’t need live agents, and an ecosystem of partners and content creators that can be part of a platform. This includes companies like Bose or Google Glass to provide the platforms as well as syndicate data, information and content which they have. But it also includes companies like Detour, that has unique and entertaining content. The ecosystem should also extend to retail companies that can join the platform to provide beacons, and hot spots to expand the network, as well as to cities that are investing in 5G to improve reliability.

The big players are certainly there, so winning won’t be easy. Microsoft launched its AI for Accessibility program last year with $25M funds committed to put Microsoft technology in the hands of start-ups, developers, and others to drive innovation. And the company has already made Soundscape, an app that helps visually impaired to better navigate a city. Having said that the applications we have today are mostly discrete, un-integrated point in time solutions that don’t succeed at delivering full independence let alone the experience one might imagine. There appears to be a market gap there.

Last week, the Economist Intelligence Unit, announced the most livable cities in the world. Vienna, Melbourne, and Sydney topped the list. London and New York ranked 48th and 58th while otherwise advanced cities, such as Singapore and Dubai ranked 40th and 70th, respectively. Perhaps, there is an opportunity for an audio first AR firm to work with these laggards to improve the livability index by offering an innovative solution to the hearing or visually impaired.

“The next big thing in audio AR will require a true pain point, a killer app that solves it and an ecosystem delivering an experience.”

Conclusion

It seems to me that there are segments for whom audio is more than just fun, and entertainment but true necessity. Imagine that there was a product for the visually impaired segment that would allow them to leave their house, do shopping, take a stroll, perhaps even drive to a new city, all on their own, independent of anybody else. That would make such a product ubiquitous, wouldn’t it? At least for that segment. 

The potential for impact is huge and the gains are likely proportional. The company that will win Audio Augmented Reality in the impaired segment will increase its chances to win in other adjacencies given the enormous challenges and difficulties to win in this market. 

Time will show who will dare to play.