Machinima

Filmmaking in Second Life

Mechthild Schmidt

During Fall 2007, I taught a “Machinima” course through the undergraduate degree-granting Paul McGhee Division within the School of Continuing and Professional Studies. The course was one of several in an interdisciplinary study of virtual environments (such as Second Life) across three disciplines — media production, media studies, and political economy. The study was supported by the 2007-08 Curricular Development Challenge Fund, a program administered by the Center for Teaching Excellence to promote innovative curricular programs and projects at NYU.

The word “machinima”¹ is a fusion of the words “machine” and “cinema”. It refers to a set of filmmaking techniques as well as a genre of film produced in real-time, 3-D interactive engines such as Second Life. My course culminated with an eight-minute machinima based on Shakespeare’s “Macbeth”. Designed and shot in Second Life, the machinima was intended to give students of digital film and animation an introduction to production and communication structures on a virtual, open source platform.

My students collaborated with students in Professor Hulley’s “Media Genres” class (see article) by participating in joint criticism meetings, supporting students in special assignments such as screenwriting, and attending several of our class and recording sessions. Half of the students in my class were also registered for Professor Anthony Pennings’ “Political Economy of Digital Media” course, which included a unit on virtual environments and helped to provide a broader analytical perspective on Second Life than possible through only my production-based course.

In this article, I discuss the technical findings and challenges that I and my students experienced during the machinima course in order to provide organizational and workflow recommendations for future NYU Second Life projects.

A Brief Introduction to Second Life

Second Life² is a software application that operates across a grid of computers to produce an animated three-dimensional virtual world populated by an international community of seven million “residents”, or members. Founded by Linden Research, Inc. in 2003, Second Life occupies a virtual space which, when scaled to earthly proportions, is about twice the size of the San Francisco Bay area. Each participant has a virtual self (called an avatar) residing in Second Life (or “in-world”), and each avatar may create or purchase assets from other avatars, using the in-world currency, Linden dollars.³ Residents can communicate with each other in-world via text chat, or voice chat using a headset.

ITS Space in Second Life

Approximately 150 educational and cultural institutions own parcels or whole islands in Second Life. These are used for such diverse activities as student advising; classes in literature, architecture, and programming; science simulations; foreign language learning; and — as in our case — instruction in media and economic analysis, as well as many aspects of media production.

The New Media Consortium (NMC),⁴ of which NYU is a founding member, is an important umbrella organization in Second Life’s academic community. NMC leases over a dozen Second Life islands for educational use and in-world conferences. ITS owns a large parcel on the NMC Outreach Island, and ITS’ support was essential to the class’s machinima project. Heather Stewart, ITS Director of Academic Technologies, directed me toward ITS Director of Student Technology Services Vincent Doogan and ITS Lab Manager Liam Fry, a Second Life member for four years. Doogan leased NMC land for ITS and generously provided a large share for our machinima “set”. When wired Second Life access was hard to find, ITS Lab Facilities Manager Robyn Berland set up wireless access for us in the ITS Multimedia Center located at 35 West 4th Street, and ITS provided technical support throughout the project.

Virtual Storytelling and Design

We intentionally chose to produce a classic text, William Shakespeare’s tragedy “Macbeth”, in this non-traditional production environment. We strove to drive our filmmaking, as much as possible, through this summary of the play’s narrative:

A flashback at the eve of his final fight leads Macbeth through the vicious cycle of power, betrayal and predictions of fate that brought him to his demise.

Production stills from "Macbeth, a Virtual Study".

We approximated historical set and costume design for an authentic look without claiming accuracy. We used tools only available in Second Life as stylistic elements to portray the paranormal elements of the story, such as insanity and magic. For example, in the first witch scene, where the “world is turned upside down”, we see the ground outlined and rotating, until Macbeth and Banquo fall to find themselves standing on clouds, looking up at the witches sitting upside down, who are unaffected by the “turn of events”. In post-production, we added lip synching, audio, and various effects to the captured clips.

The fifteen-week course was divided roughly into three phases: pre-production, production, and post-production. Students signed up for a team handling a specific aspect of production during each phase of the course. Each team leader was responsible for meeting deadlines, asking for help, and troubleshooting problems.

As we moved forward, challenges we encountered — such as the distribution of skill sets within the class (many students had a video background, but fewer had experience with 3-D) and technical limitations of Second Life — required us to stay flexible and reshuffle teams, adjust deadlines, and rethink the script. We found that extensive pre-production based on traditional filmmaking did not work in Second Life; instead, we had to jump in and test the limits, and base our shoot on those results.

Technical Considerations

We worked at the SCPS Mac-based G5 labs in the Woolworth Building and the ITS Multimedia Lab, using Second Life, Adobe Photoshop, SculptyPaint (a displacement texture modeling app), Poser, Snapz Pro, CrazyTalk, Pro Tools, Final Cut Pro, Adobe After Effects, Fetch (an FTP client), and iDVD to create our machinima project. We recorded our production to portable hard drives.

An early class meeting in the raw studio space.

Accessing Second Life

Each class member needed an avatar to participate in-world, and we easily created most of the avatars within minutes. There is an in-world self-paced orientation exercise required of new Second Life members that provides an opportunity to practice basic avatar motion skills. Second Life maintains a spam block that prohibits large numbers of simultaneous avatar registrations from one IP address, so it might be wise to require students to come to class having already created Second Life accounts from home.

During class sessions, Second Life posed few time delays; however, some students’ home computers did not have the necessary hardware to run the graphic-heavy Second Life client, making in-world meetings held after Woolworth lab hours difficult for some class members to attend.

Pre-Production

After outlining the project’s workflow, pre-production teams worked on script development, storyboarding, audio recording, and prop design. One student arranged a recording session with an audio engineer; another student with theatre and film experience served as the director and conducted a few rehearsals in preparation for recording. We scanned our storyboard, combined with a voice-over, into the computer, which served as an audible guide for the shoot.

The castle set, in progress.

We found it easy to learn to use the Second Life building tools. Students with a background in 3-D design built Macbeth’s castle by first doing photographic research, then creating sketches, a floor grid, and walls using textures in Photoshop. The props team built furniture, dinnerware, and accessories. Students searched for stores in the Second Life world that sold items we could not build. Costume and make-up design was work-intensive, as the faces and costumes had to be designed on special templates (Second Life avatars default to young faces, and wrinkled skin had to be specially designed).

The camera team tested movement and screen capture — an area in which we encountered difficulties, as the Second Life camera packages were not robust enough for our use. We purchased 3Dconnexion’s “SpaceNavigator”, a type of joystick that has been used successfully in other 3-D applications, and we agreed to become a beta site for the Macintosh version of the Second Life software. After setting up the movement parameters, we repeatedly experienced uncontrollable shifts and overshoots in motion curves. After a long consulting session with Liam Fry at ITS, and experiments on several computers, we decided that we could not afford to lose any more time experimenting and abandoned that path for a more static and reliable Second Life camera.

Production

Once production began (approximately three weeks later than we had planned), students began working within their production teams, two of which used software available from third-party vendors outside the Second Life platform:

Animation: Second Life provides a number of default animations, but specific character movements have to be custom built. We chose to use the Poser 3-D modeling and animation software to create our animation sequences, which we then imported into Second Life. The biggest challenge for the animation team was translating timings between Poser and Second Life.
Lip-synching: The lip-synching team used a third-party software application from Reallusion, called CrazyTalk. It allowed us to use a still frame and an audio track to animate lips and facial features. Adjusting the parameters took time, but the implementation was very successful.

Preparing the witch scene, with oversized "superprims"
as backdrops and a virtual rainstorm.
(A prim is a Second Life 3-D building element.)

We had planned to shoot on a tight schedule, but retakes and extra blocking time meant extended class sessions. Scenes requiring participation from all students were shot during class, but alternating groups logged onto Second Life from home and interacted with the camera and main actors in the lab to shoot smaller scenes. We had initially planned to use SCPS’s lab in the Woolworth Building, but because students’ schedules and lab hours did not always align, we were grateful for the opportunity to work in the ITS Multimedia Center when Woolworth was closed.

Post-Production

We used a variety of software applications during post-production: one team edited scenes using Final Cut Pro; another set of students were concurrently registered for a Pro Tools Sound Design class, and they assembled sound effects with the support of instructor Bob Schott; a third team worked on the score; and a fourth composited effects, credits, and titles using Adobe After Effects and Photoshop.

The first rough cut was shown at our final class meeting with Professor Hulley’s class for feedback and critique. All final elements were delivered to the student who took on all last minute tweaks and the final sound mix. A polished version is now playing as streaming video within Second Life; residents of Second Life may view the stream at http://slurl.com/secondlife/Outreach/61/72/26e.

Conclusion

The course was intensely demanding in time, as well as in student initiative and skill sets. Students needed to work together with continual communication, which grew as the semester progressed and students realized the impact their actions and inactions had on their teammates.

Shooting the last fight scene at the Woolworth lab.

This observation changed my expectations for the project. I moved from a final product-centered approach to a more procedural approach, built on teamwork and communication. With only one semester to explore a very focused use of Second Life, many other areas remained untouched; in a future course, I would like to add to this experience and explore communication and interactivity in Second Life more broadly, with smaller final projects that allow more time for experimentation.

Second Life’s strengths enabled us to produce a video that would have been financially impossible for our class members to achieve in the “real world”, and impossible to create in only one semester in a more conventional, key frame-based 3-D animation environment.

Although using Second Life gave us a measure of freedom in the production of our video, we also learned its limitations. It is not yet feasible in Second Life to recreate intricate human emotion and subtle movements for a narrative production. Considerable computer literacy and access to the appropriate equipment are necessary for a successful Second Life media production; however, a course using Second Life for the study of content, unrelated to production, would face few limitations after the initial set-up and orientation phases.

The McGhee Division hopes to use Second Life as a community platform, termed “McGhee at Midnight”, connecting our off-campus undergraduates with one another. We see great potential for Second Life as a virtual campus for our online students and those at NYU’s global locations.

While virtual platforms and social networks will change, and while they may not become a production environment for most students, collaborative learning, peer-centered virtual communication, and other skills learned in Second Life are important professional assets. It is my hope that students who learn to shape their media environment actively can assume a critical and creative role in analyzing or producing content in the framework of their future professions, and I believe virtual environments like those provided by Second Life can serve as inspiring training grounds for that pedagogical goal.

Footnotes

For more information about machinima, read the Wikipedia article located at www.wikipedia.org/wiki/Machinima.
For more information about Second Life, see Vincent Doogan’s article in the Fall/Winter 2007 issue of Connect, available at www.nyu.edu/its/pubs/connect/fall07/doogan_secondlife.html.
Exchange rate approximately L$270-300 = US$1.
For more information, visit www.nmc.org.

Author Biographies

Mechthild Schmidt is a Master Teacher for Animation and Visual Effects in the Bachelor of Science degree in Digital Communications and Media (DCoM) at the Paul McGhee Division within NYU's School of Continuing and Professional Studies.