Graduate Student Premieres Mind-Controlled Project May 12

A team of Rensselaer Polytechnic Institute students has created a system that pairs an EEG headset with a 3-D theatrical flying harness, allowing users to “fly” by controlling their thoughts. The “Infinity Simulator” will make its debut with an art installation in which participants rise into the air – and trigger light, sound, and video effects – by calming their thoughts.

Creative director and Rensselaer MFA candidate Yehuda Duenyas describes the “Infinity Simulator” as a platform similar to a gaming console — like the Wii or the Kinect — writ large.

“Instead of you sitting and controlling gaming content, it’s a whole system that can control live elements — so you can control 3-D rigging, sound, lights, and video,” said Duenyas, who works under the moniker “xxxy.” “It’s a system for creating hybrids of theater, installation, game, and ride.”

Duenyas created the “Infinity Simulator” with a team of collaborators, including Michael Todd, a Rensselaer 2010 graduate in computer science. Duenyas will exhibit the new system in the art installation “The Ascent” on May 12 at Curtis R. Priem Experimental Media and Performing Arts Center (EMPAC).

Ten computer programs running simultaneously link the commercially available EEG headset to the computer-controlled 3-D flying harness and various theater systems, said Todd.

Within the theater, the rigging — including the harness — is controlled by a Stage Tech NOMAD console; lights are controlled by an ION console running MIDI show control; sound through MAX/MSP; and video through Isadora and Jitter. The “Infinity Simulator,” a series of three C programs written by Todd, acts as intermediary between the headset and the theater systems, connecting and conveying all input and output.

 “We’ve built a software system on top of the rigging control board and now have control of it through an iPad, and since we have the iPad control, we can have anything control it,” said Duenyas. “The ‘Infinity Simulator’ is the center; everything talks to the ‘Infinity Simulator.’”

The May 12 “The Ascent” installation is only one experience made possible by the new platform, Duenyas said.

“‘The Ascent’ embodies the maiden experience that we’ll be presenting,” Duenyas said. “But we’ve found that it’s a versatile platform to create almost any type of experience that involves rigging, video, sound, and light. The idea is that it’s reactive to the users’ body; there’s a physical interaction.”

Duenyas, a Brooklyn-based artist and theater director, specializes in experiential theater performances.

“The thing that I focus on the most is user experience,” Duenyas said. “All the shows I do with my theater company and on my own involve a lot of set and set design – you’re entering into a whole world. You’re having an experience that is more than going to a show, although a show is part of it.”

The “Infinity Simulator” stemmed from an idea Duenyas had for such a theatrical experience.

“It started with an idea that I wanted to create a simulator that would give people a feeling of infinity,” Duenyas said. His initial vision was that of a room similar to a Cave Automated Virtual Environment — a room paneled with projection screens — in which participants would be able to float effortlessly in an environment intended to evoke a glimpse into infinity.

At Rensselaer, Duenyas took advantage of the technology at hand to explore his idea, first with a video game he developed in 2010, then — working through the Department of the Arts — with EMPAC’s computer-controlled 3-D theatrical flying harness.

“The charge of the arts department is to allow the artists that they bring into the department to use technology to enhance what they’ve been doing already,” Duenyas said. “In coming here (EMPAC), and starting to translate our ideas into a physical space, so many different things started opening themselves up to us.”

The 2010 video game, also developed with Todd, tracked the movements — pitch and yaw — of players suspended in a custom-rigged harness, allowing players to soar through simulated landscapes. Duenyas said that that game (also called the “Infinity Simulator”) and the new platform are part of the same vision.

EMPAC Director Johannes Goebel saw the game on display at the 2010 GameFest and discussed the custom-designed 3-D theatrical flying rig in EMPAC with Duenyas. Working through the Arts Department, Duenyas submitted a proposal to work with the rig, and his proposal was accepted.

Duenyas and his team experimented — first gaining peripheral control over the system, and then linking it to the EEG headset - and created the Ascent installation as an initial project. In the installation, the Infinity Simulator is programmed to respond to relaxation.

“We’re measuring two brain states – alpha and theta — waking consciousness and everyday brain computational processing,” said Duenyas. “If you close your eyes and take a deep breath, that processing power decreases. When it decreases below a certain threshold, that is the trigger for you to elevate.”

As a user rises, their ascent triggers a changing display of lights, sound, and video. Duenyas said he wants to hint at transcendental experience, while keeping the door open for a more circumspect interpretation.

“The point is that the user is trying to transcend the everyday and get into this meditative state so they can have this experience. I see it as some sort of iconic spiritual simulator. That’s the serious side,” he said. “There’s also a real tongue-in-cheek side of my work: I want clouds, I want Terry Gilliam’s animated fist to pop out of a cloud and hit you in the face. It’s mixing serious religious symbology, but not taking it seriously.”

The humor is prompted, in part, by the limitations of this earliest iteration of Duenyas’ vision.

“It started with, ‘I want to have a glimpse of infinity,’ ‘I want to float in space.’ Then you get in the harness and you’re like ‘man, this harness is uncomfortable,’” he said. “In order to achieve the original vision, we had to build an infrastructure, and I still see development of the infinity experience is a ways off; but what we can do with the infrastructure in a realistic time frame is create ‘The Ascent,’ which is going to be really fun, and totally other.”

Creating the “Infinity Simulator” has prompted new possibilities.

“The vision now is to play with this fun system that we can use to build any experience,” he said. “It’s sort of overwhelming because you could do so many things – you could create a flight through cumulus clouds, you could create an augmented physicality parkour course where you set up different features in the room and guide yourself to different heights. It’s limitless.”