Abstract
In a home video archival scenario, it is desirable to store only a summary of the best quality portions of the media. This equates to less required storage capacity and removal of regions which typically hinder the overall semantic intent of the document (scenes with excessive brightness or random, chaotic movement typical to home video). Towards constructing this summary, psychological studies have demonstrated that if the video is coupled with a professional-quality musical soundtrack to accompany the visual material, there is an overall increase in perceptual impact of the media. A novelty score according to self-similarity analysis is computed from any chosen audio track, which is segmented at maximum novelty points. Pictures together with the video material, constrained to the higher quality samples, are segmented temporally and aligned to match the audio segments, producing an initial audiovisual digest. A further enhancement is to semantically synchronize the specific media elements and events, based on the premise that artistic composition is the montage of passages that construct a tension/resolution progression. This is quantified using feature vectors for describing temporal and spectral features of the media and implemented as recommended by the MPEG-7 Standard. The result is a summary in which the different media elements are synchronized semantically, enhancing perception and providing a consolidated audiovisual experience.