Included here in this article:

• Why use streaming video
• What is streaming video
• Standard Compression Technologies
• Proprietary Compression Solutions - The Big Three
• The Streaming Model
• Content Creation
• Streaming Servers
• The Client
• Availability
• Problems and solutions
• References

  Advances in computing and networking technology mean that it is now feasible to deliver sound and video across the Internet. However there are still many users with old computers and slow network connections, and care needs to be taken to ensure that streaming technology is not used inappropriately and without due regard for the target audience. This article is based on the experiences at the University of Bath where they experimented with both live and recorded video.

  Why Use Streaming Video?

  Streaming video can be used for live or recorded events. The main reason for broadcasting live is to reach a wider and/or more dispersed audience. Typical live broadcasts could be lectures, sports or entertainment events, and academic or other ceremonies. For a major academic lecture given at a university the number of people who could actually attend would be limited by the size of the lecture theatre, whilst the potential audience could be anywhere in the world. Live video is essential if the aim is to give a remote audience an experience as close as possible to being physically present at the event.

  If an event is broadcast live it is relatively simple to make a recording which can then be published on the Web for later viewing. However, there are many more possibilities with non-live broadcasts. A streamed broadcast should be considered to be a multimedia event, which could include full motion video if appropriate. Good examples of multimedia lectures can be found at the Boxmind site. Boxmind use synchronised broadcasts with several streams containing video, audio, scrolling text, pictures or diagrams, and hypertext links. The synchronisation ensures that the text display corresponds exactly to the spoken commentary. Although the pictures and diagrams would be generally considered to be an integral part of the lecture, it could be argued that the audio and video are not essential. It is however widely accepted that information retention rates are much higher when a student can see and hear a lecture in addition to being able to read the text.

  What is Streaming Video?

  Streaming technology is not new. Most people are familiar with it from an audio only point of view since Marconi invented radio in 1897. Streaming video followed with TV from the mid 1930s onwards. Most people would refer to this as broadcasting. This concept is well understood. A continuous stream of information is transmitted and receivers are able to tune in and receive the information in real time.

  Streaming across the Internet, although similar in concept, has its own specific issues that must be addressed. The Internet was not designed for real time streaming. The Internet is a shared medium and uses a best effort delivery mechanism, Internet Protocol (IP) to deliver content. There is no dedicated path between source and the sink. IP breaks content up into self contained packets and these packets are routed independently. Limited bandwidth, latency, noise, packet loss, retransmission and out of order packet delivery are all problems that can affect real time streaming over the Internet.

  In the main, traditional Internet traffic is not sensitive to these problems - or handles them higher up the protocol stack at its leisure. Live or on-demand streaming is a time critical application which is sensitive to the variation in delay that is normal for a shared access network like the Internet. Not only does the amount of bandwidth that you have access to matter but also the consistency or quality of this bandwidth. All Internet streaming technologies get around this by buffering a certain amount of content before actually starting to play. The buffer irons out the natural traffic variations inherent on the Internet. Many seconds worth of content can be buffered and in excess of 30 seconds worth is not uncommon. Note that after the initial buffering the streamed broadcast will start to play at the same time as more content is being downloaded. This is an improvement over earlier technologies where the whole file had to be downloaded before playing could commence.

  It is probably still safe to say that the majority of end users are accessing the Internet over very narrowband dial up links. Comparatively few people have the luxury of access at anything over 2Mbps, however over the last couple of years cable and DSL access has been increasing allowing bandwidths between 128kb/s to 512kb/s to be available to end users. At this bit rate near-VHS quality rich media can be achieved through modern compression techniques and sophisticated codec technology.

  It is not all about quality of bandwidth however. Content creation, serving, usability and availability are also challenges that need to be addressed.

  Standard Compression Technologies

  There are a variety of compression systems used today. The Motion Picture Experts Group (MPEG) has three open (ISO/IEC) standards that can be used for streaming.

  MPEG has standardized the following compression formats and ancillary standards:

  • MPEG-1: Initial video and audio compression standard. Later used as the standard for Video CD, and includes the popular Layer 3 MP3audio compression format.
  • MPEG-2: Transport, video and audio standards for broadcast-quality television. Used for over-the-air digital television ATSC, DVB and ISDB, digital satellite TV services like Dish Network, digital cable television signals, SVCD, and with slight modifications, as the .VOB (Video OBject) files that carry the images on DVDs.
  • MPEG-3: Originally designed for HDTV, but abandoned when it was realized that MPEG-2 (with extensions) was sufficient for HDTV. (not to be confused with MP3, which is MPEG-1 Audio Layer 3.)
  • MPEG-4: Expands MPEG-1 to support video/audio "objects", 3D content, low bitrate encoding and support for Digital Rights Management. Several new higher efficiency video standards (newer than MPEG-2 Video) are included (an alternative to MPEG-2 Video), notably:
    • MPEG-4 Part 2 (or Advanced Simple Profile) and
    • MPEG-4 Part 10 (or Advanced Video Coding or H.264). MPEG-4 Part 10 may be used on HD DVD and Blu-ray discs, along with VC-1 and MPEG-2.

  In addition, the following standards, while not sequential advances to the video encoding standard as with MPEG-1 through MPEG-4, are referred to by similar notation:

  • MPEG-7: A multimedia content description standard.
  • MPEG-21: MPEG describes this standard as a multimedia framework.

  Moreover, relatively more recently than other standards above, MPEG has started following international standards; each of the standards holds multiple MPEG technologies for a way of application.

  • MPEG-A: Multimedia application format.
  • MPEG-B: MPEG systems technologies.
  • MPEG-C: MPEG video technologies.
  • MPEG-D: MPEG audio technologies.
  • MPEG-E: Multimedia Middleware.

Despite the open standards of MPEG most people use one of the big three proprietary formats. These are RealMedia, Quicktime and Windows Media. All three have specific advantages which have allowed them to gain ground in the market - mainly because they are free, and support the Real Time Streaming Protocol (RTSP).

A very popular player which is very widely distributed and available for all major OS platforms.

RealPlayer is up to version 10 and the latest generation codecs (developed with Intel), coupled with their SureStream technology, will probably keep them in a dominant position. RealSystem 8 supports over 40 media formats. Surestream is an automatic multi bit-rate technology that will adjust the streamed data rate to suit the client's connectivity. In practical terms this means that a single encoding will suit all users from dial-up to corporate LAN. Also supported is Synchronised Multimedia Integration Language (SMIL) which allows mixed multimedia content to be delivered in a synchronised way.

RealServer is also available for most OS platforms but is only free for a basic 25-user licence. Streaming is RealNetwork's core business so they cannot subsidise the technology in favour of market share as Apple and Microsoft do. Serving more than a couple of hundred simultaneous streams can become quite expensive and one major drawback of the system.

Originally developed in 1991 version 6 now claims more than 100 million copies distributed world-wide. Quicktime's major advantages are its maturity and the large number of codecs available for it. It features an open plug-in feature to allow third party codecs to be added. MPEG1 and MPEG4 codecs are currently available.

The plug-in feature has allowed over 200 digital media formats to be supported by Quicktime 4 with companies such as Sorenson Labs producing very impressive codecs. As with RealPlayer, SMIL is available and now RTSP is also supported. (Prior to version 4 only progressive streaming, not true real time streaming was available in Quicktime). Quicktime 5, currently in beta, also has support for immersive virtual reality.

Quicktime server is supported natively in MAC OS. The open source Darwin Streaming Server is available for other platforms and is free.

Windows Media Player (currently at version 10) is the newcomer to the streaming world. Because of this, there have been fewer codecs available for it but because of their OS dominance, this is quickly changing. There is an MPEG4 codec and Microsoft's proprietary but very good ASF codec. Microsoft have put some work into their RTSP implementation and it is considered more efficient than others. SMIL is also supported.

Microsoft give the player away free in their operating systems and the company's marketing might means that the format is quickly gaining popularity. There are currently more than 250 million players in 26 languages in existence however, the European market is more difficult for Microsoft. Because of law suits in Europe, Microsoft has agreed to market their operating systems without Media Player.

Microsoft's streaming server (called Microsoft Media Services) is otherwise free and supplied as standard under Windows 2000 and 2003 Server and as a free download for Windows NT server. Microsoft have not open sourced the code which means that other platforms are not supported. This is considered a major disadvantage as far as flexibility is concerned.

The components of an end to end streaming system are the client or player, the server and some sort of content creation process. As always, content is king so the greatest amount of time will probably be spent on the creation process.

he designer of the content will use various production tools to create the content. These tools convert audio, video, or animation to a data type format that the server can stream. Because most servers can deliver content in many different formats, there are a number of tools that people can use in creating content. Production tools can epitomise the content for efficient delivery over the Internet, based on the nature of the material and the capabilities of the client computers.

Each of the big three provides tools for creating or converting content into a format that can be handled by their servers and epitomised for Internet Streaming. RealNetwork's RealProducer 8 will convert from a number of raw formats (AVI, MPEG-1, AU, AIFF etc) and is free for the basic version. Apple's Quicktime Player (free and pro) also provides content authoring and import/export facilities and Media On-Demand Producer is free from Microsoft.

Sonic Foundry's Stream Anywhere can be downloaded for about 125 US dollars and provides for the creation of streaming content in Real and MS Media formats. A more complete (and more expensive - upwards of 600 US dollars) product is Terran's Media Cleaner 5. Cleaner 5 is a complete suite of tools for preparing video and audio for the web and is considered the industry leader in this field.

The content creator can also create a Synchronised Multimedia Integration Language (SMIL) file to synchronise several clips within a presentation. A SMIL file co-ordinates the layout and playing of two or more media clips in parallel (simultaneously) or in sequence. A typical example of this is a lecture or presentation with associated slides where the presentation of the slides can be synchronised with the audio content of the lecture.

RealNetworks have put the most effort into developing SMIL for the web and have created proprietary formats of RealText, RealPix, RealVideo, RealAudio and RealFlash for use within a SMIL script. SMIL version 2.0 is currently in draft and will enhance the language significantly.

Creating content with SMIL (which is based on XML) takes more time and effort but the results are worth it. RealNetworks can supply the Oratrix Development program GRiNS for the creation of SMIL texts. There are many examples on the web showing how, for very little bandwidth, excellent media rich presentations can be compiled which are much more informative and interesting than the statically presented video.

Local experience has shown that it is not usually sufficient to simply encode existing video content for streaming. Content producers need to be cognisant of the tremendous compression ratios that are common in this arena. Subtle visual information is lost and picture sizes will be small. Limited camera movement is important as is good lighting, simple backgrounds and close ups of subject material.

All the systems have ways of making it easy to provide a single link for users encompassing multiple data rates. This means that your files can stream without the user having to specify a particular bandwidth. QuickTime's approach is to create a different file for each. This complicates the encoding process and does not address the issue of fluctuating bandwidths. However, having each file individually encoded does provide enormous flexibility.

RealNetwork's SureStream technology and Microsoft's Intelligent Streaming lets you put multiple tracks in a single file each with a different bit rate for delivery. Of the two Real's SureStream is the most sophisticated and flexible, and if bandwidth fluctuations are an important factor in delivery of content this will deal with it best. Combining SureStream with SMIL is also possible.

The content creator can either prepare media clips in advance or encode a live event as it happens. In this the term encoder refers to the software (such as RealProducer, for example) that converts live or pre-existing media into a format that the server can deliver.

Just as a Web server delivers pages to Web browsers over the Internet, streaming servers deliver media clips to clients (clips are created with the production tools described elsewhere). Real time streaming requires specific servers. RealNetworks, Microsoft and Apple all provide streaming servers. These servers give you a greater level of control over your media delivery but can be more complicated to set up and administer than a standard HTTP server. Also, real time streaming uses special network protocols, such as RTSP or MMS (Microsoft Media Server).

Ideally the user should have a simple hypertext interface and have to do no more than click on a link. Any upgrade or download of a client player utility should be automated and transparent. In practice client downloads tend to be large and complicated procedures with too many options available for the average user.

Streaming availability on the global Internet should ideally mean a server ready to stream content to any clients who have an interest in receiving it. Unfortunately the demand and availability of media rich content has lead to a breakdown of the traditional client server model. Single servers streaming content to diverse groups of clients distributed across the Internet are ineffective in terms of both server load and network congestion.

Over the last couple of years strategies have evolved in the commercial sector to address these problems. Content Delivery Networks (CDNs) are an attempt to introduce a coherent approach to building an infrastructure of caching proxies, mirror servers and proxy accelerators to enable a more efficient and speedier delivery of streamed content to end users. The ultimate goal is to replicate content and bring it closer to the end user in a transparent fashion. In this way the user sees no URI changes and has no knowledge (nor interest) in the actual source of the content.

There are several commercial CDNs already offering these services. Probably best known is Akamai with its FreeFlow technology. Adero use what they call their GeoTraffic Manager and Omnicast technology to move fresh content closer to the audience. Digital Island does very much the same with its Footprint technology. They claim a 10-fold speed increase by distributing content to their world-wide network of servers.

iBeam use their MaxCaster media serving system located in points-of-presence around the world. They use proprietary software and satellite networks to push content through their network. They claim more than 500,000 simultaneous streams now and will be capable of serving millions of streams in the future. Edgix use their Edge Delivery Platform which includes edgeMedia, edgeNews, and edgeStream to ensure high performance delivery of content to end users.

Although the above are commercial ventures, the notion and requirement for the CDN model has been appreciated generally. Over the last 12 months work has been carried out within The United Kingdom Education & Research Networking Association (UKERNA) to look at providing a similar distributed resource for delivering streamed content within UK academia. The recent upgrade to JANET (SuperJANET4 ), providing 2.5Gbps backbone links increasing to 10Gbps in the next two years, provides a huge leap in bandwidth availability. This offers excellent opportunities to experiment with streaming media but is also cause for concern as without proper management even large bandwidths like this can be swamped.

UKERNA intend to pilot an implementation of a content management system using the JANET core network. Content will be replicated at the edge of the (core) network and clients automatically directed to their nearest edge node. In this way core network resources are far more efficiently managed than in a centralised server model, and the end user should benefit from better and faster access to the resources they require.

SuperJANET4 new backbone contains eight Core Points-of-Presence (C-PoPs) geographically located throughout the UK. Here bandwidth and switching converge and offer the capacity to accommodate additional services and opportunities above pure transmission and routing.<

Given a good a network connection streaming video works well, although in many ways it is surprising that it works at all. As mentioned earlier, the nature of the Internet and its use of IP means that a broadcast is competing with other data transmissions, and in general there is no way of guaranteeing sufficient bandwidth to ensure an uninterrupted broadcast. Video conferencing systems usually use other network technologies such as ISDN, which has a relatively low but guaranteed end to end bandwidth, or ATM which can be set up with channels with guaranteed Quality of Service (QoS). Bandwidth over the Internet is increasing rapidly but unfortunately demand seems to be keeping up with supply, so increasing bandwidth alone is unlikely to solve the problem. Various developments are taking place which should ultimately result in QoS being available over IP and this, together with the emergence of CDNs, should result in a rapid growth in the use of video over the Internet.

A more mundane, but nevertheless important, difficulty in our experience is that many users have trouble setting up their client machines to receive audio and video. In the case of PCs most users seem to need a PC expert to help them install, for example, Real Player. A more fundamental problem in many educational establishments is that teaching rooms have frequently been set up without audio hardware. Where there are a large number of machines in one area it is generally necessary to use headphones.

Because "live" broadcasts are not really live but are typically delayed for around 30 seconds it is difficult to set up remote feedback. For example where an on-line lecture has been publicised in advance it would be beneficial to allow questions from the remote audience. One way to achieve this would be to set up a Web page so that people could type in their questions, which could then to relayed to the lecturer by another person in the room.

Many organisations have their networks protected with a firewall and, even if normal Web traffic is allowed, special provision may have to be made to allow access to the ports used to receive streaming video. The same applies when serving video to the Internet from inside a firewall.

Despite the problems, streaming has shown that it is practicable to deliver multimedia broadcasts across wide area networks, providing the end user is connected to the network with a reasonably fast connection such as Ethernet, DSL or cable modem. We do not consider it feasible to use a dial-up modem connection to view full motion video streamed broadcasts although it should be adequate for audio only or slide-show presentations.

This article has concentrated on the technology needed to produce and deliver multimedia, and in particular video presentations. However, careful consideration should be given as to whether video is needed as part of a multimedia presentation. Although easy to produce, a continuous shot of someone talking direct to camera is technically demanding on bandwidth and probably adds relatively little to the presentation. In particular, when producing material for education and training a combination of slide shows, animation and recorded computer session together with a commentary is easier to deliver and in many cases more effective than full motion video.

• Motion Picture Experts Group (MPEG): <http://www.mpeg.org/>
• Real Networks: <http://www.real.com>
• Quicktime: <http://www.apple.com/quicktime>
• Sorenson: <http://www.sorenson.com/>
• Darwin:< http://www.opensource.apple.com/projects/streaming>
• Microsoft Windows Media: <http://www.microsoft.com/windows/windowsmedia>
• Sonic Foundry: <http://www.sonicfoundry.com>
• SMIL: <http://www.w3.org/TR/REC-smil/>
• Ligos: <http://www.ligos.com/>
• Akamai: <http://www.akamai.com>
• Talking Point : <http://www.talkpointcommunications.com/>
• SuperJANET4: <http://www.superjanet4.net>