Arts Technology

Streaming Media

Jeff Bary

Point, click, wait ... This is how viewing or hearing art on the Internet always seems to go. Why does it have to be so slow? In order for you to see or hear a piece of artwork, your browser must wait until the entire file arrives at your computer before it can be displayed or played. This makes sense for a large image or a complex website. But why wait so long for time-based artwork like video and audio? Why not start playing the beginning while the later parts continue to arrive at your computer? Sound like a good idea? It is. It's the basic concept behind streaming media.

Quality Control

Before you go ahead and stream your new feature film, though, you will first need to consider the limitations of putting video and audio into a digital format. Video quality is determined by frame rate, frame size and compression type; audio quality is determined by sampling frequency, word length and compression type. The settings that you choose for these elements should depend on how your work will be seen or heard.

Streaming video looks good at frame rates starting at about 10 frames per second. This means that the movement will appear smooth, without any of the jerkiness associated with slower frame rates. In order to get a higher frame rate, you will have to sacrifice image size. The smallest image size is typically 240 pixels wide by 160 pixels high, or about three inches by two inches. Would you sit and watch a one or two-hour movie on a two-inch by three-inch screen?

If you have the RealPlayer from RealNetworks (available at www.real.com), you can view an example of a full-length movie on a small screen: a recording of the keynote address from Educom '97 (previously at http://www.educom.edu/conf/97/contents.html).

I tried to watch these broadcasts on a very fast Silicon Graphics workstation. Despite the superior hardware I had available to me, I was disappointed in the output, because I was using streaming media as a substitute for broadcast television. I felt that it didn't deliver a satisfactory viewing experience. But streaming media is not television and it's not radio. It's a whole new medium that bears a strong resemblance to broadcast television. Because of these similarities, we want to force streaming media into molds that current technology cannot support. As the technology advances, we will be able to support broadcast-quality television and radio for all Internet viewers.

Audio quality is determined primarily by sampling frequency. A minimum rate for video with audio would be mono at 10 Khz. Any less, and the audio quality becomes very poor. Audio without video streams much better; you can get transfer rates close to that of a CD.

On the practical side, there are a number of techniques that can be used to control the quality of your final presentation. The most important is the quality of the original material.

Noise does not compress well. This is true in audio as well as in video. Compression is based on analyzing the source material for similar and repetitive sections. These sections can then be reduced to simpler but identifiable reproductions of the original material. Keep in mind that almost all of these compression schemes discard information from the original. This effect is called "lossy," because information is lost and cannot be recreated.

Reaching Your Audience

When your work is going on the Internet, you need to predict what kind of connection the majority of your audience has. If you are delivering your stream to computers directly connected to the NYU network, you will be able to deliver very high quality audio and video, perhaps approaching broadcast quality. Viewers who are connected with a lower-speed modem will not be able to receive a high-quality output, and a different format might serve them better.

To avoid this guesswork, you can store your piece in several different formats, so the viewer can choose the appropriate version for his or her connection speed. Some types of media allow you to provide the same piece in several different formats. The player software then requests the proper version to play, depending on the connection speed, browser and server.

This requires an understanding of how web pages and media files are distributed on the Internet. There are two popular methods in use today for sending media to your browser program.

The first is the same way that all web pages and inline graphics are provided. Normally, web pages and graphics are requested by your browser from the HTTP server on a network computer. These are stored as files on that computer along with all the other web page and graphics files. When you click on a link to request a media file, the web server locates and sends the file containing an HTML, movie or audio file.

This works great for web pages, but not for streaming media. The web server merely starts sending the file as fast as it can until it has sent the entire file. The viewer has no control. It is not an interactive experience because you cannot stop, start or restart the piece.

There are several choices for this all-at-once method. The most popular are video files in QuickTime or AVI format. Audio has a number of formats: AU, WAVE, AIFF or QuickTime. QuickTime is supported on both Apple Macintosh and Windows platforms. Both QuickTime and AVI can have audio tracks without video and can be used as audio-only media. QuickTime can be made to stream even when sent from a web server rather than from a special server.

The second method is to use a proprietary streaming media server and custom player software. The server stores information about the media, usually the name of the streaming media server that the media file is on and the name of the file on that streaming media server. The web server sends that information to your browser. In turn, your browser passes this to a helper application, which in this case is the player. The player then makes a request to the streaming media server for that specific file.

A streaming media server is different from a web server in that it has the ability to listen to requests from the player program to control the playback of the media. These allow the player to act more like the controls on a tape player or VCR -- Stop, Pause, Play, Rewind, Fast Forward.

There are also a number of different methods for delivering streaming media. Some of the newer ones are Java-based, and some require additional separate players. In all cases the original media must be compressed and translated into the format for that player or server.

The Arts Technology Group of ACF currently operates two different types of proprietary streaming servers: RealMedia from RealNetworks, and Mediabase from Silicon Graphics. These servers are still experimental, so the format, availability and support change often. If you would like to put material on them, a member of the Arts faculty will have to request it for you. Please visit the Arts Technology Group Streaming Media website at www.nyu.edu/its/atg/media for current information and examples of material from this article. [ C ]

The Arts Technology Group (www.nyu.edu/its/atg) of ACF currently runs two streaming media servers. There is a tremendous amount of documentation on the Internet about streaming media. Go to the ATG Streaming Media site at www.nyu.edu/its/atg/media for links to these sites and examples of the techniques described in this article. In addition, we provide online gallery space to artists at NYU. Contact the author at jeffrey.bary@nyu.edu for more information on using these resources.

Jeff Bary integrates art and networks as a senior member of the Arts Technology Group at ACF.
Posted January 20, 1998