PhD Thesis
"Higher Level Techniques for the Artistic Rendering of Images and Video"
John P. Collomosse
University of Bath, May 2004
Abstract
This thesis investigates the problem of producing non-photorealistic renderings for the purpose of aesthetics; so called Artistic Rendering (AR). Specifically, we address the problem of image-space AR, proposing novel algorithms for the artistic rendering of real images and post-production video.
Image analysis is a necessary component of image-space AR; information must be extracted from two-dimensional content prior to its re-presentation in some artistic style. Existing image-space AR algorithms perform this analysis at a ``low'' spatiotemporal level of abstraction. In the case of static AR, the strokes that comprise a rendering are placed independently, and their visual attributes set as a function of only a small image region local to each stroke. In the case of AR animation, video footage is also rendered on a temporally local basis; each frame of animation is rendered taking account of only the current and preceding frame in the video. We argue that this low-level processing paradigm is a limiting factor in the development of image-space AR. The process of deriving artwork from a photograph or video demands visual interpretation, rather than localised filtering, of that source content --- a goal challenging enough to warrant application of higher level image analysis techniques, implying interesting new application areas for Computer Vision (and motivating new Computer Vision research as a result).
Throughout this thesis we develop a number of novel AR algorithms, the results of which demonstrate a higher spatiotemporal level of analysis to benefit AR in terms of broadening range of potential rendering styles, enhancing temporal coherence in animations, and improving the aesthetic quality of renderings. We introduce the use of global salience measures to image-space AR, and propose novel static AR algorithms which seek to emphasise salient detail, and abstract away unimportant detail within a painting. We also introduce novel animation techniques, describing a ``Video Paintbox'' capable of creating AR animations directly from video clips. Not only do these animations exhibit a wide gamut of potential styles (such as cartoon-style motion cues and a variety of artistic shading effects), but also exhibit a significant improvement in temporal coherence over the state of the art. We also demonstrate that consideration of the AR process at a higher spatiotemporal level enables the diversification of AR styles to include Cubist-styled compositions and cartoon motion emphasis in animation.