Kristin Glushon, Head of Partnerships & Marketing – Trigger, The XR Agency
Zoe Soon, Vice President Experience Center – IAB
Why Mixed Reality for Entertainment
Engaging entertainment—from movies and TV to music and sports—has the ability to transport consumers to alternate worlds and states of mind. It invites us to escape to a different reality or connect with our inner emotional world. Mixed reality (MR, an umbrella term that covers augmented reality and virtual reality) allows us to merge physical and virtual worlds to varying degrees and therefore lends itself perfectly to all types of entertainment. Its technical capabilities can range from a simple AR overlay like a Snap filter, to fully immersive virtual reality using head-mounted displays (HMDs). Mixed reality is multisensory and immersive, allowing storytellers, entertainers and brands to bring entertainment experiences to life like never before.
MR experiences are increasingly accessible for consumers as cloud computing and lens technology advances; headsets become lighter and go wireless; and as a result of Apple and Google phones including AR-ready software, which eagles the use of AR on a daily basis. In fact, this year the number of AR users in the U.S. alone is expected to reach 85 million and by 2022 there will be an estimated 3.4 billion AR capable devices (AR Insider). Consumers, particularly Gen Z who represent 40% of US consumers and hold a significant amount of buying power, also expect additional layers of content via their mobile devices. In addition, 93% of Gen Z also want to see brands do something new and unique to capture their attention (Mediapost). XR offers brands a way to do all this and more and, according to Forbes, we’re seeing the use of AR in entertainment growing at a rate of 24.4% per year with the projections it will reach $4.5B by 2030.
Growing expectations and access to these types of experiences demonstrates that brands need to capitalize on the benefits of MR for their branded entertainment initiatives. There are a number of ways that brands can successfully utilize this technology and the following sections will outline some of the key trends, as well as best practices and future outlook. The shift to MR is happening before our eyes, transitioning from passive 2D experiences into interactive, immersive 3D experiences. Brands that capitalize on this momentum have the opportunity to achieve greater brand recognition, deeper engagement with their consumers and ultimately a stronger marketing ROI.
Immersive Storytelling
Brands are using MR to enhance interactions with their products and services by creating compelling stories that allow consumers to feel as if they are a part of the narrative and setting. Using AR, VR, MR, 360-degree video and immersive audio technologies, brands can bring different worlds to life, such as the perilous lands of Jumanji, which users explored in the WebAR experience created by Sony Pictures to promote the sequel, Jumanji: The Next Level. In this experience, fans visited three stunning locations from the movie: the Desert Dunes, the Moat, and the Mountain Fortress. They could explore these locations with detailed animations and enjoy exclusive content, such as videos featuring Dwayne Johnson, Karen Gillan, Nick Jonas and Jack Black and others from the star-studded cast. The XR experience played an important role in garnering excitement for the film and to promote ticket sales.
Coors Light also used MR for immersive storytelling to engage with specific consumer segments, such as their Pacific Northwest region. Through a WebAR River Tour, the AR experience employed a mix of 3D environments and authentic 360º footage, so users feel as if they are base jumping over the Snake River or riding on a dogsled across Alaska. This beautiful, fun and immersive tour provided an experience worth talking about as it earned over 3 minutes in dwell time, as noted in the recent IAB AR ROI Creative Showcase.
Character Interactions
Character interactions in MR are also an effective way for brands to enhance sports and entertainment experiences because of their ability to bring real and fictional characters into a user’s environment. XR can drive consumer engagement by giving fans access to their favorite character, athlete, celebrity or musician using technology such as volumetric capture (volcap), which uses 360º still or video footage of people, objects or places to create lifelike 3D representations. Volcap can facilitate interactive moments and social sharing. For example, Verizon developed the AR Pro Interactive app that allows fans to interact with famous athletes in 3D who then share a breakdown of their signature move. Likewise, Disney, Verizon, Google and Snap Inc collaborated to create AR lenses that showcase the powers of several iconic Avengers who their fans know and love: utilizing reactive tracking, the social filters allow users to hold Thor’s hammer while wearing his armor or utilize Captain Marvel’s binary mode to shoot energy beams from their hands.
Gamification
Gamification can further promote branded entertainment by adding a layer of gaming elements to an XR experience. This type of gamification provides the user with something of value in a fun and novel way when interacting with the brand. One such creative AR gamified experience was the Ghostbusters: AR Haunted Candy Hunt that candy-maker Ferrara developed in partnership with Sony as an accessible and scalable alternative to Trick-or-Treating due to COVID-19 restrictions. The experience featured an AR game with characters from the Ghostbusters franchise, along with a new candy ghost composed of iconic Ferrara candies. The gameplay allowed users to stun and capture AR ghosts before time runs out and then be entered into a sweepstakes to win prizes. The experience also included retail signage and product packaging integration, where users were able to text a code displayed on store stands and candy packaging to receive a link to access the WebAR experience on their smartphones. This approach created a cohesive brand experience across their various marketing channels.
Brand Sponsorship/Promotional Partners
An effective approach to expand the reach of an XR-based sports or entertainment experience is through brand sponsorship and promotional partners. By working collaboratively with a brand to create a product that features elements of an entertainment experience, you capture fans of both the brand as well as the IP partner. For instance, General Mills partnered with the Star Wars IP to create a “Do Good for the Galaxy” charitable campaign, where users were prompted to scan cereal boxes to unlock AR activities with 3D models of the droids BB-8 and D-O, ultimately attracting both Cinnamon Toast Crunch and Star Wars fans. Burger King also expanded their reach by partnering with UK rap artist Tiny Tinie to create a private AR concert on a burger. Once the QR code on top of the burger was scanned, Tiny Tinie performed his song Whoppa on top of a Whopper. The experience garnered 10.8 million social impressions, 468k TikTok views and 220 pieces of media coverage (8th Wall).
OOH and Location-Based
Out-of-home and location-based XR entertainment experiences allow brands to engage with consumers at places they visit and can complement entertainment venue experiences at stadiums, theaters and concert venues. These types of experiences often leverage GPS technologies to identify a device’s location and position, triggering XR experiences that can create a direct link to a venue or event. For example, Riot Games used location-based AR to bring to life a 3D dragon that flew above the stadium during the League of Legends World finals. It enhanced the onsite fan experience and gave users an opportunity to engage with the game world and characters. To connect with fans in the theaters and celebrate the release of Disney’s Toy Story 4, Pixar created an AR experience that was fully integrated into the Regal Cinemas app. During the theatrical release, users were prompted to scan select Toy Story 4 movie posters to see the characters, like Woody, come to life, adding an additional entertainment value to the moviegoer experience.
Live Events
XR activations for live events provide a unique way for brands to elevate the event experience. Brands such as Verizon, the NBA and the PGA TOUR have used AR portals to provide exclusive access to events, including behind-the-scenes experiences. For instance, Verizon’s portal experience gave fans 360º access to the stage where the Grammy Award-winning band, Black Pumas, was performing. Other brands look to elevate the in-person experience by adding XR layers, such as the Xbox promotion of Halo Infinite during halftime show at the Oregon vs. Oregon State football game that projected an AR Halo aircraft in the Autzen Stadium. It not only promoted the stadium’s brand sponsors but also added a unique experience to the event.
Fully immersive VR experiences for live events are also a growing trend. For instance, Fortnite received significant fanfare for a virtual 15 min concert featuring Ariana Grande that attracted an audience from all over the world, capturing 1 million viewers at its peak (Stream Charts). Wave, the virtual music platform, hosted a live concert featuring Grammy award-winning singer-songwriter John Legend by utilizing Xsens MVN Animate Motion Capture System, a suit used for capturing highly dynamic movements. The 17-minute concert featured a variety of his hits and captured his physical movements digitally with incredible detail. In addition to virtual concert platforms, social media has also drawn attention to immersive live events, including one hosted by TikTok for Digital Fashion Month. The experience involved a virtual fashion show, performances from musical artists, collaborations with TikTok Creators and WNBA athletes modeling clothing for three featured designers. The hour-long program included seven 3D worlds uniquely created for each designer and performers showcased during the event.
Best Practices
There are important best practices for brands to consider when implementing XR in entertainment. First and foremost, XR should align with the larger brand goals and be measured for ROI at various stages of the marketing funnel. Keep in mind that traditional measurement metrics aren’t always applicable to these mediums, however, XR is known to increase engagement and differentiation, with extended dwell time often leading to brand consideration. In addition, XR entertainment experiences should also provide an easily accessible and compelling value proposition, especially as a means to differentiate as the technology becomes more mainstream. Therefore, brands should attempt that ‘wow factor’ through entertainment content that provides an enhanced user experience, as well as exclusive benefits, rewards or offers. Finally, multichannel campaign deployment is often necessary for the success of XR campaigns, along with supporting media and promotional efforts. It’s also advantageous to leverage your XR assets where possible across various platforms, such as social, broadcast, in-retail and OOH in order to achieve efficiencies and scale.
Future Outlook
New technology will always push the boundaries of what is possible, and innovation with head-mounted devices (HMD) is about to hit a tipping point. Usage of HMD has increased and is expected to grow to 36.5 billion by 2026. As early as next year, Meta is expected to release their HMD product described as AR glasses, and while the technology is still in its infancy, it’s indicative of a shift to a spatial internet. In addition, Niantic’s Lightship ARDK and other multiplayer AR platforms allow users to shift seamlessly from solo to multiplayer for a more dynamic and social gaming experience. The long-term vision for Lightship ARDK is to build a global 3D map using a Visual Positioning System (VPS) where content and gameplay can be superimposed anywhere in the world. With new funding to invest in current games and new apps, Nianitc plans to expand the Lightship developer platform and build out its vision for the real-world metaverse.
XR provides brands with a portal into the next iteration of the Internet, and brands are starting to familiarize themselves with this new storytelling language. According to a recent study by Meta, 75% of all brands plan to use AR in 2022. With the global XR market valued at $30.7B this year and expectations of a tenfold increase to almost $300B by 2024, it is critical for brands to start finding their voice in immersive XR experiences, especially those that deliver compelling entertainment value.
There have been two big shifts in technology since the rise of the Internet. First there was mobile, which allowed us to be connected constantly. Next there was the rise of social media, which allowed us to connect to each other and in a way, democratized the internet; we moved from passive consumption with traditional publishers as the arbiters of information to interactive participation and saw the rise of the creator economy. We’re now entering the third wave of the internet: the spatial internet where our physical and digital worlds will intersect more seamlessly and where we’ll transition from 2D to 3D as we leap from the flat screen into our world.
