The Download: Feature Articles

Yiyao Nie: Technological Interactions Beyond the Screen

By Victoria Lubas | April 8, 2020

Pushing Boundaries and Overcoming Limitations

Artist and designer Yiyao Nie has been working with interactive media since she began her digital media technology undergraduate studies at Zhejiang University seven years ago. Building off her 3D and computer graphics background, Nie’s early work consisted mostly of VR and AR simulation experiences — as “VR and AR were avant-garde at that time” — as well as gaming and apps.

After graduating, Nie came to NYU to pursue a master’s degree from Tisch’s Interactive Telecommunications Program (ITP). Nie describes her undergraduate work as more technology-focused and screen-based, explaining that devices and screens provided a “very convenient but not intuitive way of interacting with the world.” But at ITP, Nie says her artwork started to focus on “broader interaction and understanding beyond the screen — human-to-human, human-to-machine, and machine-to-machine relationships.” As an ITP student, Nie’s art focused on “what technology and art can bring to the people, their feeling, ...and the moment,” as well as “pushing the boundaries of interactive media and breaking the limitations of devices and machines” to facilitate a more natural and intuitive interaction between humans and machines.

Nie was also a member of the NYU IT Student Tech Innovation Team. As a UX designer, Nie conducted research and helped redesign the NYU Hartford Institute of Geriatric Nursing’s website. For Nie, the greatest takeaways of the project were the “chance to talk with the user, ...do research ...[and] learn from other UX designers from the team.”

Now working as a creative technologist at Apple, Nie’s current job requires her to make immersive and intuitive work. For Nie, the “combination of tech and art is very natural,” so her artwork has always involved innovative uses of technology. When creating a new project she decides on a concept to depict and then seeks out the technology that will allow her to tell that story. While Nie doesn’t categorize her work into pre- and post-grad categories, she has found that her most recent works are more collaborative and larger-scale than her previous projects.

River: Technological Art at London Fashion Week

Nie teamed up with fashion designer Qiongxin Kou to create the collection River to Ocean which debuted at London Fashion Week 2020. River to Ocean combines Kou’s textile work with Nie’s 3D printed sculptures that resemble splashing liquid frozen in time. The models wore Nie’s 3D prints as hats, vests, and decorations on their shoes. Nie added to the show by performing with her instrument River, a technologically-enhanced tambourine that responds to the users motions and dance movements with sounds. River, which Nie created in 2019 while she was an ITP student, has switches on its surface that allow the user to choose from various different modes, resulting in new, seemingly random sounds that are meant to allude to the flowing of water in a river.

Attached to the base of the tambourine is a long piece of fabric that encourages the user to make large gestures and interact more wholeheartedly with the instrument. Nie explained that interactive media used to be synonymous primarily with gaming and animation, and then with AR and VR; all of which require screens to interact with the virtual world. River is one way Nie’s work evolved beyond the screen to explore the human-machine relationship. River will be exhibited in the annual Currents New Media Festival in Santa Fe, New Mexico, which highlights the work of new media artists. Nie said she is “very happy to be accepted into this institution.”

Mitochondria: Combining Visual Graphics and Traditional Art

In 2018, Nie created Mitochondria, a project that combines dance and interactive technology to produce real-time visual graphics in response to the dancers’ movements. Using customized shader, codes, and graphics simulation, Nie created a series of works consisting of white and cool-colored strokes that flow around a black background. Performers, wearing white, floor-length costumes, dance in front of a wall onto which Nie’s designs are projected. A Kinect camera detects points on the dancers’ bodies and tracks their movements, causing the projected image to react in real-time. Working on Mitochondria as an ITP student gave Nie the chance to explore the “interaction between [the] body and interactive technology.”

Nie’s experience with audio-visual mediums and performances drew her attention to the role of the instrument and other forms of interaction. In December 2019, Nie collaborated with percussionist Shiqi Zhong by designing the visual graphics¹ that accompanied his solo performance at Carnegie Hall. For this piece, Nie designed a series of circles and ovals that simulate water as they expand and float to reflect Zhong’s music.

Life in a Nutshell: 3D Printing and Motion Graphics

For her ITP thesis², Nie created sculptural representations of 12 ancient Chinese characters and oracle bones that represent different stages of life, titled Life in a Nutshell. Nie took advantage of the services, expertise, and student discount offered by the LaGuardia Studio, to create high-quality 3D prints of anthropomorphized Chinese characters. The prints stand on wooden blocks and are labeled with both the Chinese and English word that is being represented.

Each character also has an accompanying video that was created using motion capture technology and serves to motivate the audience to get involved in the story. Nie explained, “by making the poses embed the Chinese character, the audience will interact with the interactive installation to experience the life.” For the month of February, Nie’s sculptures were on display at Space p11 in Chicago as part of the “Fixity Equals Death” exhibit.

Future Endeavors

Reflecting on her recent projects and current commitments, Nie explained that lately she hasn’t been able to collaborate with as many artists as she’d like, instead focusing on her design work for Apple. “It is fun working with a big company to create innovative products and solve actual real-world problems. But still, I would like to explore more in-depth the dreamy world of being able to create art pieces that express me as an emotional, vulnerable and also strong human being,” states Nie.

In her next project, Nie wants to continue exploring the storytelling potential of art and technology in the interactive media field. Nie describes herself as “a girl that dreams about using her artwork to speak to the world, to leave something to the world, to have a bigger impact on the world.”

Sources

1. Blendtw, “Carnegie Hall Celebrates 40th Anniversary of Sino-US Diplomatic Relations with Recital” (Video)
2. ITP, Life in a Nutshell