Watch Local 4 News at 10 p.m. streaming here on ClickOnDetroit or on the Local 4+ channel on Apple TV, Roku or other Smart TV device.

DETROIT – A green screen, live actors, live musicians, digital cameras and a real-time gaming engine: is this the latest sci-fi fantasy hitting theaters? Nope, it’s Detroit Opera’s latest venture opening this weekend at the Detroit Opera House.

While it might be a bit more difficult to give an elevator pitch for this production, it’s this type of experimental live theater that’s bringing Detroit to the forefront of the opera scene with Yuval Sharon at the helm.

We chatted with Sharon, as well as Kaitlyn Pietras and Jason Thompson of PXT Studio who created the virtual world which the live performers live on, about bringing this new technology to Richard Wagner’s 1870 opera: Die Walkürie.

First of all, how did you get Sigourney Weaver involved in this?

Sharon: Since we’re only presenting Act III, I wanted to make sure audiences didn’t feel like it’s okay for them to come to this production without having seen the first two acts of Die Walkürie, or even the opera that precedes it, Das Rheingold, which is the first part of the Wagner cycle.

So to help make this a standalone piece, it seemed important to have a voice introduce us into the world of the opera. Sigourney Weaver is the queen of science fiction so she’s the perfect person. I also thought it was a great opportunity to introduce us to the science fiction aesthetic that we’re after this kind of Tron meets Matrix-like world we’re creating.

How did this idea of working with a green screen come about?

Sharon: Well Jason, Kaitlyn and I have worked with green screen technology before. We’ve actually done two other projects together in which we’ve used green screens to help bring the singers into a dimension and a visual universe that’s hard to achieve within the three walls of a theater.

About a year ago, the LA Philharmonic asked me to direct Act III of Die Walkürie for the Hollywood Bowl. And the Hollywood Bowl has these fantastic LED screens for image magnification, and it seemed like the perfect place to think about a production like this that would focus on that image magnification. So this focus on the jumbotron creates a dissonance between the live performance of the piece and the imagined digital universe that they would be in.

Can you talk about the technology going on in real-time with the performers on-stage?

Sharon: There’s layers to this. We’re essentially offering the audience a video game in real time and the singers are in front of this green screen and you’ll always be connected to the singers in real time. But the space they’re performing in is very shallow and empty. The five digital cameras are transporting them into a digital environment that was created using video game technology and a video game engine that has a three-dimensional component to it. So while we have the live singers in a very shallow and empty space, they’re also participating simultaneously in the kind of Metaverse, this digital landscape that’s being created for them in real-time following the music. It’s pretty extraordinary.

Thompson: This is the perfect project where technology has gotten to the point where now we’re thinking about how video can react in real-time. So we’re using Unreal Engine and it’s so powerful that the world is in 3D. We’re sending triggers to move virtual cameras around inside of it and it’s all happening in real-time every time we do it. Shot-for-shot, we’re creating this movie experience and you get the duality of watching the performers on stage against a green screen and then also the composite that’s happening in real time. It’s pretty exciting and there are a lot of layers to it.

How has the technology evolved to get to the point where we can do events like this?

Thompson: Technology moves exponentially fast, so the computing power that we had when we first started working together versus now it’s completely different. We feel like we’re lightyears away. Before, we were waiting around for things to render out in 3D scenes. Now it’s happening in seconds. It’s happening so fast. And Yuval has always had such a cinematic mind when he approaches operas like this. While we listen to the music, he sees these cinematic jib shots and fly-throughs and all of these things and we’re able to manipulate virtual cameras based on what Yuval sees.

Pietras: The first one we did was more of a 2D-built environment that was a lot of Photoshop and AfterEffects. The second one, all the content was built on a program called Cinema 4D, which is more for motion graphics. That was more like the wait around render. This is the first time that we’re using video game engine technology to create everything in 3D and also have it generating things live. So even things like snowfall, it’s all completely different every time that we see the show.

Sharon: To that point, the technology is incredibly exciting, but what’s really fascinating is how it really unlocks the music and unlocks this original story that Wagner demands with his enormous and epic music. He’s trying to depict these eight women flying through the air on horses, and he’s trying to depict a magic fire that’s surrounding his daughter. These images can rarely be convincingly realized on stage. They’re often quite honestly pretty embarrassing where the music is so amazing whereas the visuals, of what’s actually possible on stages, is nowhere near the ideas that you can tell from the music. So this technology actually feels like it’s able to realize this kernel of an idea that Wagner had and has never been able to realize.

With green screen technology, you’re pretty much limitless when it comes to design. How did you decide on this retro look?

Sharon: It was a bit of trial and error. Our initial idea was a bit more naturalistic: mountain tops and riding through the air with blue skies. But it just didn’t feel resonant at all with the music that feels very futuristic, almost like science fiction.

So it all happened very quickly where we just said: Tron, we all love that movie. We all love The Matrix. And we all responded to this aesthetic called vaporware from the 80s and 90s with this whole notion of a digital network and living on a digital plane was first starting to emerge and now feels very close to home which now feels very close to home because we’re doing this interview on Zoom. But Tron and the like feels really appropriate for it being a fantasy, but in a science fiction world. It’s looking forward but looking backward at the same time.

What were some of the challenges in using all this technology with live theater?

Pietras: Well none of this technology was designed to be used in the way that we’re using it. Video game software is designed for video games that are exported a certain way and used by one user in a certain way. I’m leading a team of technical artists when we also didn’t know the parameters of the software and also trying to explain to them how theater works. So it was really cool using this technology in a way that’s not meant to be used. That’s sort of our thing.

How do you think technology adds on to the beauty of the opera?

Sharon: The technology, as exciting as it is, the live performance is really the heart of it. In fact, the technology amplifies what the singers are naturally bringing to this piece getting a chance to have close-ups of these incredible singers. Christine Goerke owns this role of Brüunhilde – she’s sung it all over the world. To have her do it here, but also to be able to have an intimacy through this close-up that you don’t get anywhere else is such a gift. Alan Held, who’s playing Wotan, is an international star and seeing every nuance of emotion on his face just brings the life of this character right to the forefront and it’s a really remarkable experience.

So with the live performers on stage and the giant screens showing the real-time rendered visuals, how do audiences know where to look?

Yuval: There is no right or wrong way to experience this: whether you only watch the screen, only watch the singers, whether you close your eyes and just listen to the music. I like the fact that there is a tension and you’ve got this choose-your-own-path through this performance. Ideally, you’re probably going back-and-forth from the live to the digital image quite a bit. That means that every single audience member gets their own experience of this. I’ve been excited about that because opera opens up so many possibilities of experience and the notion of these dual realities that you get to choose how you want to composite them yourself. I think it’s part of the fun.

What was it like to see this production at the Hollywood Bowl for the first time?

Pietras: The first time I watched it from out in the house, I actually cried. It was really powerful. We’re [Thompson] actually married and we have a son and he watched it with me.

Thompson: It’s one of the most impressive venues in Los Angeles and to be doing the show there is incredibly difficult but incredibly rewarding to be able to pull it off there. It’s just amazing.

Are there any changes being made to the show as it makes its way to Detroit?

Pietras: We made a lot of improvements. The Hollywood Bowl felt like a proof that we could actually do this crazy idea, like a great proof-of-concept. I feel like we’ve really taken it to a new level. I think that the whole concept we have is more well-suited to this environment. It’s indoors and the screens are a little more accessible. I think it’s really more designed for this venue here and we’ve had a lot more time to fine tune things and make little adjustments that actually made a huge impact on the overall piece.

Sharon: Jason, Kaitlyn and I have worked on so many projects that are incredibly difficult to explain. But when you come and see it, you kind of go, “Oh, that’s what they were talking about.” It’s already happened even with the people that work here at Detroit Opera. But then they walk into the theater and their jaws are on the floor because they can’t believe what we’re actually accomplishing. I hope it’s the same for audiences where it might still be very hard to understand what exactly we’re doing but if you trust us and come, I think you’ll enter this theater really experiencing something you’ve never seen before.

The Valkyries has an 87-minute runtime and plays at the Detroit Opera House on September 17, 18 and 20. For showtimes and tickets, visit DetroitOpera.org.

Copyright 2022 by WDIV ClickOnDetroit - All rights reserved.

If you need help with the Public File, call (313) 222-0566.

Copyright © 2022 ClickOnDetroit.com is managed by Graham Digital and published by Graham Media Group, a division of Graham Holdings.