It is possible to create tools for both the analysis of the media streams and the post-processing the media streams in order to create new modified streams. For video it is envisaged that the media streams are post-processed to generate periodic full intra frames in order to provide multiple entry points into the inter coded original stream. This will enable playback from arbitrary places in the stream.
With analogue video a new whole frame is sent to the display every 25th of a second. Due to the way digital video is compressed using intra frames and inter frames, whole new frames are rarely (if ever) sent to the display. These intra and inter frames are the differences from the previous image. A whole frame is reconstructed from the differences of the image contained in the intra and inter frames. Our technique will have the benefit of having a full image at all times during playback. Without having these full inter frames it will be possible to replay the video from an arbitrary place but only see a few flickering blocks of changing inter coded video. Furthermore as this technique will generate whole images at a regular frequency it will be possible to view a video sequence in a fast- forward mode.
When using video data compression, such as the H.261 encoding, the encoded data stream contains changes from the previous image. The changes take the form of Inter frames and Intra frames. Inter frames are used for small local changes, and Intra frames are used for large changes.
By doing an analysis on the Intra frames it is possible to determine when large changes have occurred within a video sequence because a large change in the motion or a scene change will cause the encoder to generate Intra frames. The analysis could determine where Intra frames occur within a compressed video stream and automatically create a new index into that video.
For fast perusal video, the compressed domain again provides some benefits. As it is known where scene changes or major motion events occur, then an indexing mechanism could be devised based on this information which will create a new index with references only to the specified points. When such an index is created, the playback mechanism could go to each indexed point in the video and play back a few seconds to the user, and then skip to the next index point. In this way the user will see all of the main events of the original video without the need to do a random search. The user could then choose to go back to any point and start playing the video in full.
To aid searching a continuous media stream such as video we plan to do automatic scene-change analysis to determine when a scene has changed to a new one and do motion analysis to determine if there has been a lot of motion within one scene. It is at these points that the user will be interested.
Next: Index References as Data Up: Indexing Techniques Previous: Using Indexes For Segment Jon CROWCROFT