How virtual technology will redefine music shows

Tech innovation has always pushed the boundaries of music shows, from Moog synthesizers in the 1960s to Elvis Presley’s posthumous hologram duet with Celine Dion in 2008. Industrial Light & Magic (the visual effects company behind the “Star Wars” franchise), takes that another step with the mixed reality experience for ABBA’s triumphant Voyage concert residency fronted by its de-aged, virtual avatars. Creative director and visual effects supervisor Ben Morris says early feedback points to the show changing the way we think about live music. 

Kate MacArthur: Can you help set the record straight on what this technology is?

Ben Morris: I can emphatically say it's not holograms. We chose to consider how we could present physical performance on a very advanced LED screen. But there are moments in the concert — without giving too much away — where the characters also are projected onto what's called a scrim or a gauze.

It's a very old technique where you project onto a fine mesh that you can’t see when it has no illumination on it. When you project onto it, suddenly an image hovers in space. The great thing about the new screen technologies is that they have an incredible dynamic range. It is way beyond traditional films, cinema or theater, which allowed us to create a perception, or the presence of the avatars in a way that we've never been able to before.  

MacArthur: How do you think it went?

Morris: I don't think it will replace real gigs and concerts, but I think it's another element in the industry. All the signs from the professionals are saying, “This will change the way in which we think about public events and music events.” 

MacArthur: How is the suspended disbelief you inspired for this show similar to or different from a film or illusionists?

Morris: One thing is that the building was designed for [this] purpose. It's the level of detail and perfection that we aim for that allows the viewer to just succumb to it. The other thing is ABBA’s music resonates in a very odd way. People are literally crying and just utterly thrilled, and they're singing and they're all together. It's a human experience, and that's the key.

MacArthur: Did you build things into the show to help it feel spontaneous?

Morris: There are moments where each of the individuals actually talked to the audience. After the song, how long does the crowd cheer for? Is there a little interaction with the crowd? We layer in all those ideas, similar to a real gig.

MacArthur: Did you have any uncanny valley issues?

Morris: Yeah. Trying to create humans is really hard. We had all sorts of people sanity checking what was going on. The uncanniness of it all is understanding why people recognize characters and what we didn’t want to do. You don't want to make a caricature, but you need to understand what a caricaturist is doing when they pull out the salient elements that make you recognize a person.

MacArthur: In our survey, most Americans don’t think virtual concerts are worth the price of a live concert.

Morris: There have been a lot of examples of really bad versions of what we have just delivered. It’s soured the audience. There are guys who I’m working with who do all the biggest gigs all over the world, Beyoncé — everyone. And they’ve sat there, and they’ve said, “I’ve never seen anything like it in my life. This feels like a normal gig.”

MacArthur: What does this tell you about the connection between the artist and the audience?

Morris: Some artists are chameleons. Some artists go through entire creative, different looks: Bowie, Madonna, Gaga. That visual language, that visual emotion that you are presenting, if you present it right, will connect with the audience and the music will then reinforce it. If the technology can break the barriers to you worrying about if that person is physically there or isn’t, then you’ve got something really interesting.  

MacArthur: How is this going to change music concert production?

Morris: Cinema, home entertainment and live venues are going to change. And the ability to create human performance is something I don't want to get rid of. But the ability to capture human performance and restage it is an exciting proposition. The technology that we use to do that is literally changing on a daily basis. So, will it bleed into the metaverse, into even more augmented concerts, into cinema becoming more dimensional? All those things are happening at the same time. You don't really want people to feel the technology, you just want them to feel the excitement and the presence.