AR + SOUND

XR in all its shapes has started to become more part of our daily life. We are overlaying navigation, capturing pokemon and testing new sunglasses without thinking too much about the underlying technology.

A couple of years ago one of the large silicon valley tech companies asked us at IDEO to explore what an AR technology anchored in sound, not the traditional visual first approach, would be like. What would the UX be? What was the core experience? What form factors would be desired? 

How might we design a desirable wearable device that augments the physical world with the social web?

Below I'm sharing a couple of snippets of that exploration. 

2017 

MY ROLE

Prototyping

UX design

What does an audio-first AR experience sound like? 

Augmented hearing isn't new

We have over many centuries explored different ways to augment our ears. From simple cones to focus better to more refined hearing aids and today's cochlear implants. So trying to give ourselves superpowers for the ears isn't as new as it might be for our eyes or other senses. 

That said, one of the big differences with today is that we can decouple the creation of sound from time and place. We do that also through our phones but with always-on, and always-listening devices dedicated to augmenting our lives with a hidden layer of sound we can create new experiences that is more intimate and designed first hand for our ears. 

ear
37c8383
ear_trumpet_John_Ryenders_illustration_c.1880
0bf3170e61fdf2ebab9155f28bdf337f

Exploring through making

Even though sound is a very integral part of our lives we quickly found that most people lack the vocabulary to talk about it and had a hard time imagining different experiences. So to overcome that hurdle we took a prototyping approach where we built a large set of small experiments of what new experiences for a sound first AR experience would feel like.

IMG_2048
IMG_0399
unnamed-1-2
unnamed-9
unnamed-1-1

Research to evolve

The most promising of our prototypes we brought to a set of individuals that all had different forms of "extreme" relationship with sound. We had discussions with people that were visually impaired, music producers and "golden ears". Through those discussions, we learnt what worked, what was creepy and what would be exciting to augment their daily life. 

We also shared early form factors of what a device could look like to better understand what is socially acceptable to carry with you on the body, what signals it communicates and how important the overall form was. 

DSCF4440
DSCF4390
IMG_9439
IMG_9445
IMG_9483b

Exploring the form

Throughout the whole research, one feedback was very clear which was the ambiguity of the status of the audio experience for the outside viewer. The form of the device needed to be able to clearly articulate this status to allow people to understand if someone was "connected" to this hidden audio first world or open for connection with the people around them. 

For this exploration, we paired with an industrial design firm called Instrument industries which worked alongside the IDEO team to explore different potential form factors based on the research findings and the hardware requirements to deliver the experiences.  Together we landed on a design language inspired by jewellery that merged both the ability to communicate its status and the desire to be worn. 

NARNIA-8
MAGA-5
NARNIA-6
MAGA-1
Hoku-035
Hoku-142
Hoku-132

Honing the experience

As we learned more we started to hone in on a set of form factors and UX that was both desirable for our users and had enough exploration regarding feasibility to be worth taking seriously. We landed in two main UX directions which we called Narnia and Maga. 

Narnia was all about transporting and connecting you with another location while Maga was about exploring a hidden world of sound around you. From a confidentiality perspective I can't share more details on the process we took bringing these concepts forward. 

Hoku-086

Where we landed

The final designs and recommendations were used as input into a larger initiative on positioning the company better for an AR-driven future. The hypotheses and insights are being used across their teams to guide decision-making and to inform the next generation of AR experiences. 

David Sjunnesson


 || 2023 ||