Through the modern technology known as virtual reality (VR), people are able to interact with three-dimensional, computer-generated images so vivid that they feel as if they’ve been physically transported to another lifelike world. And while the technology has been used for some time to make video games more exciting and realistic, a Baylor University professor is discovering new VR applications that can enhance the experiences of theatergoers.
After earning a BFA degree in theatre design and technology from Baylor, Ryan Joyner received an MFA from Cal State-Fullerton in theatre design and technology, specializing in lighting and sound design. He then returned to Baylor to work as the master electrician in the theatre arts department and lecture part time in sound design. As more students signed up for classes in sound design, a tenure-track faculty position was created in the subject and Joyner was chosen for it in 2014. As assistant professor of theatre arts, he has decided to focus his research on ways to connect VR with live performance.
Virtual reality is interactive, meaning that it allows for immersive experiences in computer generated and simulated environments. Users wear specially designed headsets over their eyes to view VR content, and Joyner’s research focus was sparked by using such a headset.
Joyner received Google’s first mass market contribution to the world of VR, known as the Google Cardboard –– a simple viewer that can be used with a smartphone. Google Cardboard allows users to affordably observe the immersive technology of VR apps and videos at their convenience. Intrigued by the ease and accessibility of the product, Joyner was inspired to figure out ways he could use VR to enhance theatrical experiences.
“I was looking for a research track that I would be passionate about pursuing for several years,” Joyner said. “When I got my first Google Cardboard viewer, I was amazed at what you could do with your mobile phone and this piece of cardboard to really put the person in a new environment. So, that got my wheels turning about how we could use that same effect to share theatrical experiences. On a basic level, it could be a really easy way to share our own productions at Baylor with a wider audience.”
Joyner believed that applying VR to the theatre would transform shows into experiences that audiences could fully explore and interact with from wherever they were seated, whether that was in a theatre or on a couch at home.
“The epitome of this type of (VR) experience would be to attend a performance of a show virtually, so that you wouldn’t have to pay for travel, sit in the nosebleed seats or be a passive viewer,” Joyner said. “You can use VR technology to either be an audience member –– say, from third row orchestra –– or choose to be transported onstage right there next to the performers.”
With visions of the ways he could experiment with virtual reality, Joyner then began to take steps to acquire the proper equipment –– a special 16-lens camera that can capture 360° 3D imagery. Such cameras are expensive, so he and his department chair came up with an idea for a mailer (shown below) sent out to prospective Baylor students that could be folded into a VR headset (shown below). The viewers would allow the prospective students to view VR videos of Baylor events. The Office of Undergraduate Admissions thought it was an innovative way to recruit students and agreed to provide Joyner with the needed equipment.
The VR viewers were an instant success. Before, the rates of interaction for prospective Baylor students receiving direct mail pieces had ranged from 5 to 10 percent. But after students received the VR viewers, their interaction rate more than doubled to 26 percent.
“I’m really proud of it because of the widespread effect that it has touching so many prospective students, and that they have now been able to experience Baylor from across the world,” Joyner said.
Innovative Music Video
Another new use of VR has required collaboration between Joyner, the Baylor School of Music and the Department of Film and Digital Media in the College of Arts & Sciences. They have worked together to produce a virtual reality music video for VirtuOSO, Baylor’s premier A Capella vocal group. Joyner said that creating a virtual reality video of a musical performance is different from creating a standard video, due to the fact that the VR camera captures images and sound from all parts of a room at once.
“It works very similarly to any other type of film in the sense that there is preproduction planning, storyboarding, etcetera. However, since the technology is so different, there are special processes,” Joyner said. “One unique thing about the way VR is filmed is that since it is a 360 degree capture, the production team can’t actually be in the room with the camera and the performers while it’s recording, or else the technicians will be caught on tape.”
While planning for the VR music video shoot, Joyner used his knowledge of sound design to make sure that viewers would have a peak auditory experience by being able to hear each singer individually in the VR headset while watching the video.
“Because I am a sound design professor, I am best with the spatial audio. For the music video, I was able to spatialize each individual person’s vocal tracks with them as they moved around the camera so that their voice comes from where they are visually in the piece, while also making sure the harmonies blended well together,” Joyner said.
Because of the special requirements to correctly record virtual reality, producing and shooting the music video was a bit unorthodox, with every Baylor collaborator providing and exercising expertise in ways that had never been done before.
“We put the camera in the center of the action. The choreographer would direct the performers in how to move around the camera at different points of the song and adjust their movements and blocking, but every time we did a take, the whole production team would run and hide behind the audience wall. Then we would say ‘and action’ and duck down,” Joyner said. “A lot of it we wouldn’t actually be able to see in detail until the next day once the film was stitched together. We did several takes a day and about two and a half days of shooting until we felt we had the right take for that music video.”
Baylor film and digital media students under the supervision of Dr. Corey Carbonara, professor of film and digital media and Master Teacher, produced a behind-the-scenes documentary about how the VR VirtuOSO video was made. The documentary won first place in its category at a film festival sponsored by the Society of Motion Picture and Television Engineers (SMPTE) in Hollywood.
“Our students did an amazing job producing that piece, showing exactly how we made the VR video,” Joyner said.
The VirtuOSO VR experience video made its campus debut when Baylor had its “Give Light” philanthropic campaign kickoff in November 2018. Those in attendance, including Baylor President Linda Livingstone, got the chance to wear a headset and watch the video for the first time.
“There were probably about a hundred people who got to see the VR music video, and just watching them try it, and that being the first piece of VR that they experienced, was really amazing,” Joyner said.
Joyner’s research into new applications for virtual reality has introduced audiences to technology that many have neither seen nor heard about. He appreciates that he can produce high-quality VR content that can be enjoyed by those at Baylor and beyond.
“My favorite part is watching people’s reaction to the finished product, because even though VR has been around for a while and has had all of this hype behind it for the last few years, not a lot people have seen a fully produced VR piece, especially not in a nice quality headset,” Joyner said.
The original Baylor VR recruiting video, as well as 360° VR videos of special Baylor events such as Christmas on Fifth Street and Baylor Homecoming as well as a VR campus tour, can be viewed on the Baylor VR YouTube page.