Creating an Immersive AR Music Experience at Coachella
For Coachella 2023, Tool worked with Virtue Worldwide to activate an immersive experience for Coke Studio on the festival grounds. At a moment when artists travel from all over the world to perform in person for their fans, we partnered with the only artist who defied space and time in order to show up. Enter Yameii Online, a metaverse music artist and social influencer. Across both festival weekends, this immersive experience ran every 5 minutes or approximately 600 times over the course of the festival.
The Activation Space
In Coke Studio’s 2,000 sq. ft. activation footprint, we created a content experience that would take over 13 ft. tall LED walls, surrounding guests with a 360° immersive array. Reflective floors and a mirrored ceiling completed the space, transporting guests into a virtual world of light, animation and sound.
The Experience
As guests were led into the experience by a brand ambassador, they would tap their phone on a NFC enabled plinth, launching a second screen AR experience that provided a deeper layer of engagement during the virtual performance, directly on their own devices.
Once inside the space, a short intro on the surrounding screens, allowed users to familiarize themselves with AR markers from the world of Coke x Yameii & Friends and how to scan and activate them during the main experience.
The Music Video
Tool was presented with the Yameii character and a few hero visuals from the Yameiiverse. Leveraging this creative foundation, we designed and directed the immersive music video. Our vision was to transport the audience into a vibrant cloud world, interwoven with Coke metaverse-inspired architecture, and brought to life by a delightful cast of Yameii’s friends.
Tool was the lead in the design and animation process, ensuring every detail reflected Yameii’s and Coke's vision. A standout moment in the music video was the hero immersive Coke ASMR scene, meticulously created to provide viewers with a sensation of floating in space. This immersive experience left a lasting impact, captivating audiences and leaving them mesmerized.
Exploring Yameii’s World
We wanted to give guests a tangible way to explore Yameii’s world, using creative tech to set it apart from the performances on the main stages. We leveraged WebAR, to create a magic window on a user’s phone, and placed specific visuals within the LED content that triggered AR animations as they viewed the performance.
Four special characters (including a chrome purple Coke bottle), triggered animation that popped off the screen in augmented reality when scanned. After exploring Yameii’s world, at the end of the performance, the surrounding screens as well as the mobile app UI, instructed guests to take a selfie with a larger than life Yameii and share it on social media.
Bringing Yameii To The Main Stage
Utilizing Sky Segmentation in Augmented Reality, Tool catapulted Yameii onto the main Coachella stage. Employing cutting-edge computer vision to accurately extract the sky in real-time, this technology facilitated the dynamic enhancement of AR elements against the backdrop. Tool seamlessly projected Yameii’s performance onto the sky canvas, creating a visual spectacle that captivated the audience.
Creative Tech Magic
The LED content was run through Resolume, while the WebAR experience was a browser-based instance running on each user’s own device. In order to ensure that all users had a time synced experience, and that users who came in at different times during the intro would be time synced with the LED content, we developed a simple Node.js and Socket.io architecture. We created a controller interface that a brand ambassador would use to fire an event to start the experience and monitor that everything was running smoothly during the show.
This BA controller, communicated with the cloud-based node server, which in turn would speak to Resolume, as well as to each client instance, communicating the time code of the experience.
Time syncing the screen content and the mobile content, resulted in an elegantly choreographed sequence of visuals, interactions, and messaging, that created a simple and intuitive experience for a diverse set of participants.
The Results
Over the course of both weekends, the activation was able to have more than 9,700 participants come through the space. In total, that represents more than 300 hours of consumer engagement within this immersive, branded experience.
Additionally, we provided screen capture for sharing, which when used, was used 3.8 times per user. Guests also were able to have a sampling experience at the end, on their way out of the activation, with some lucky guests receiving incredible Yameii inspired merch. And, to the kind gentleman who offered to trade us a pepperoni pizza for a Yameii hoodie, we salute you!
