Using Shazam song recognition capacity to create a live AR multi-sensorial immersive experience for people from the hard of hearing/deaf community.
Product Design
UX
Interaction Design
Product Strategy
Team Project (2)
2-week sprint
Research indicates that the most common use case of Shazam lasts less than half a minute, suggesting that users utilize the app to identify a song and then quickly switch to a different app to collect or listen to the song.
How might we increase user engagement in our app by providing services that help connect users and their music?
Business Model: Since Shazam was purchased by Apple in 2018 for $400M, primarily due to its extensive data on users' music interests, most in-app actions guide users towards Apple Music.
Assumptions: Apple aims to keep users engaged on Shazam to gather more data on their music preferences and general social behaviors. Gamification or social media features could satisfy both user and business needs.
After conducting the first round of limited interviews (6 users) and completing our affinity mapping, we concluded that users appear to be unwilling to invest additional time in the app beyond its existing offerings. Therefore, our project boundaries now include the social media model, gamification, and streaming.
Brainstorming: To facilitate unconstrained ideation, we recognized the need to stop viewing Shazam solely as a music streaming or social media app and instead focus on what truly made it unique and popular. In addition to its audio fingerprint technology for song recognition, Shazam demonstrated an advanced ability to recognize sound on a highly sophisticated level.
Assumption: The deaf and hard of hearing community currently consists of approximately 466 million people. Given that deafness often leads to depression, Shazam has the potential for a positive social impact within this demographic.
How might we leverage Shazam's sound recognition technology to help the deaf community connect with the auditory world?
Objectives: After redefining the problem, our new research objectives were to understand the community's relationship to music and the tools and technology they currently use to overcome their impairment.
Limitations: Constrained by the time allocated for this sprint, we had limited access to deaf and hard-of-hearing users. We interviewed a total of two hard-of-hearing users and one user who self-identified as deaf to gain insights into their relationship with music and how they experience it.
Online: To complement our interview findings, we collected videos and articles focusing on the sensory experience of music for our target users.
"We got refused to enter the club because we were using sign language. That was one of the most discriminating moment of my life"
The deaf/HOH community feels marginalized, especially when they use technology that was not designed for accessibility.
"Feeling sound, instead of hearing it, is a form of sensory substitution—using information from one sense as a proxy for another."
Music is a multisensory experience, and the physical sensation of bass vibration is a common substitute.
"it’s not always the beat that Deaf people can use, the visuals are great too! I’ve experienced coordinating light shows and immersive subtitles at concerts"
Visuals and lyrics will provide external cues to follow along with the song and enhance the experience.
Connect with the audible world and other people around them through music.
Enjoy the emotional experience of connecting through music.
Understanding the full spectrum of music: melody, lyrics, and rhythm.
Feeling isolated and depressed from deafness
Can feel music through the bass, but sometimes it is not enough to understand the song's intention
Unable to experience the full range of emotions
... a way to access the full spectrum of sensory experience delivered by music in order to feel less marginalized from the hearing world.
I have designed an interactive mid-fi prototype to continue testing with users and collect early feedback on the product. A summary of user feedback will be included in the case study at a later time.
Our user is sitting at a lively coffee shop when they receive a notification on their phone from Shazam indicating that a song is playing. They are curious and want to feel included, so they decide to go on Shazam Sphere to understand what everybody is experiencing. They enjoy the experience so much that they share their Shazam Sphere experience with their friends.
