Irfan Essa’s Academic Activities

ACM Siggraph 2008 Class on Computional and Journalism

• Date and Time: Wednesday, 13 August 2008 | 1:45 pm - 5:30 pm
• Location: Room 502 A, Los Angeles Convention Center, Los Angeles, CA, USA

Fundamentally, journalism is the process of collecting news information and disseminating that information with a layer of contextualization and understanding provided by journalists in the form of a news story. Recent advances in computational technology are rapidly affecting how news is gathered, reported, and distributed, and how stories are authored and told. New technologies for aggregating, visualizing, summarizing, consuming, and collaborating on news are becoming increasingly popular. They are challenging the traditional practices of journalism and directly affecting the future of news production and consumption. Computation and journalism share a deep interest in information and the value it provides to society, and they are deeply involved in the future of storytelling in various contexts, especially current events. This class summarizes how these new technologies affect journalism, both at the core of the journalism discipline and in its practice and business. Topics include: the technologies that have empowered citizen journalism and related citizen media production and authoring; mobile and sensing technologies that allow journalism to become ubiquitous and pervasive; the changes in photo, video, and broadcast journalism; and how web, online, and science journalism are changing the basic processes of reporting. Instructors focus especially on areas of special interest to the SIGGRAPH community: photography and video, large-scale information visualization, and social networking.

Presentations will be made by:

• Irfan Essa (Georgia Institute of Technology) [website]
• Brad Stenger (WIRED NextFest)
• Maneesh Agrawala (University of California, Berkeley) [website]
• Jeffrey Heer (University of California, Berkeley, Stanford University) [website]
• Eric Ulken (LA Times) [website]
• Chloe Sladden (Current TV) [website]

This course is open to all registrant of ACM SIGGRAPH 2008 and has not pre-requisite requirements. See the info on ACM SIGGRAPH Site

ACM SIGGRAPH 2008 Paper’s Committee Meeting was held at GA Tech in Atlanta, March 29-30, under the leadership of Greg Turk. Following is a picture of all of us at work, with our sigs, as a note of thanks for Greg

Original Photo by myself, this version with sigs by Fredo Durand.

Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra (2005), “Texture optimization for example-based synthesis” In ACM Transactions on Graphics (TOG) Volume 24 , Issue 3 (July 2005) Proceedings of ACM SIGGRAPH 2005, Pages: 795 - 802, ISSN:0730-0301 (DOI|PDF|Project Site|Video|Talk)

ABSTRACT

We present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the synthesis problem as minimization of an energy function, which is optimized using an Expectation Maximization (EM)-like algorithm. In contrast to most example-based techniques that do region-growing, ours is a joint optimization approach that progressively refines the entire texture. Additionally, our approach is ideally suited to allow for controllable synthesis of textures. Specifically, we demonstrate controllability by animating image textures using flow fields. We allow for general two-dimensional flow fields that may dynamically change over time. Applications of this technique include dynamic texturing of fluid animations and texture-based flow visualization.

James Hays and Irfan Essa (2004) “Image and video based painterly animation” In Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering (NPAR 2004), Annecy, France, June 7-9, 2004, pages, 113 - 120, ISBN:1-58113-887-3, 2004 (DOI|PDF|Project Web Site).

ABSTRACT

We present techniques for transforming images and videos into painterly animations depicting different artistic styles. Our techniques rely on image and video analysis to compute appearance and motion properties. We also determine and apply motion information from different (user-specified) sources to static and moving images. These properties that encode spatio-temporal variations are then used to render (or paint) effects of selected styles to generate images and videos with a painted look. Painterly animations are generated using a mesh of brush stroke objects with dynamic spatio-temporal properties. Styles govern the behavior of these brush strokes as well as their rendering to a virtual canvas. We present methods for modifying the properties of these brush strokes according to the input images, videos, or motions. Brush stroke color, length, orientation, opacity, and motion are determined and the brush strokes are regenerated to fill the canvas as the video changes. All brush stroke properties are temporally constrained to guarantee temporally coherent non-photorealistic animations.

Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, Aaron Bobick (2003), “Graphcut textures: image and video synthesis using graph cuts” In ACM Transactions on Graphics (TOG), Volume 22 , Issue 3, Proceedings of ACM SIGGRAPH 2003, Pages: 277 - 286, July 2003, ISSN:0730-0301. (DOI|Paper| SIGGRAPH Video (160 MB, 50 MB) | Video Results 87 MB | Project Site)

ABSTRACT

In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample image or video are transformed and copied to the output and then stitched together along optimal seams to generate a new (and typically larger) output. In contrast to other techniques, the size of the patch is not chosen a-priori, but instead a graph cut technique is used to determine the optimal patch region for any given offset between the input and output texture. Unlike dynamic programming, our graph cut technique for seam optimization is applicable in any dimension. We specifically explore it in 2D and 3D to perform video texture synthesis in addition to regular image synthesis. We present approximative offset search techniques that work well in conjunction with the presented patch size optimization. We show results for synthesizing regular, random, and natural images and videos. We also demonstrate how this method can be used to interactively merge different images to generate new scenes.

Arno Schödl and Irfan Essa (2002), “Controlled animation of video sprites” Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. ACM Press, Pages: 121 - 127 July 2002, San Antonio TX, ISBN:1-58113-573-4.  (DOI|PDF|WebSite)

ABSTRACT

We introduce a new optimization algorithm for video sprites to animate realistic-looking characters. Video sprites are animations created by rearranging recorded video frames of a moving object. Our new technique to find good frame arrangements is based on repeated partial replacements of the sequence. It allows the user to specify animations using a flexible cost function. We also show a fast technique to compute video sprite transitions and a simple algorithm to correct for perspective effects of the input footage. We use our techniques to create character animations of animals, which are difficult both to train in the real world and to animate as 3D models.

Gabriel J. Brostow and  Irfan Essa (2001) “Image-based motion blur for stop motion animation” In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (ACM SIGGRPH) Pages: 561 - 566 August 2001, ISBN:1-58113-374-X ACM New York, NY, USA (DOI|PDF|Video|Project Site)

ABSTRACT

Stop motion animation is a well-established technique where still pictures of static scenes are taken and then played at film speeds to show motion. A major limitation of this method appears when fast motions are desired; most motion appears to have sharp edges and there is no visible motion blur. Appearance of motion blur is a strong perceptual cue, which is automatically present in live-action films, and synthetically generated in animated sequences. In this paper, we present an approach for automatically simulating motion blur. Ours is wholly a post-process, and uses image sequences, both stop motion or raw video, as input. First we track the frame-to-frame motion of the objects within the image plane. We then integrate the scene’s appearance as it changed over a period of time. This period of time corresponds to shutter speed in live-action filming, and gives us interactive control over the extent of the induced blur. We demonstrate a simple implementation of our approach as it applies to footage of different motions and to scenes of varying complexity. Our photorealistic renderings of these input sequences approximate the effect of capturing moving objects on film that is exposed for finite periods of time.

Video textures

Abstract

This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying stream of images. While the individual frames of a video texture may be repeated from time to time, the video sequence as a whole is never repeated exactly. Video textures can be used in place of digital photos to infuse a static image with dynamic qualities and explicit actions. We present techniques for analyzing a video clip to extract its structure, and for synthesizing a new, similar looking video of arbitrary length. We combine video textures with view morphing techniques to obtain 3D video textures. We also introduce video-based animation, in which the synthesis of video textures can be guided by a user through high-level interactive controls. Applications of video textures and their extensions include the display of dynamic scenes on web pages, the creation of dynamic backdrops for special effects and games, and the interactive control of video-based animation.

• Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of the 27th Annual Conference on Computer Graphics and interactive Techniques International Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, 489-498. [DOI][PDF]
• Project WebPage
• Presentation Slides
• SIGGRAPH 2000 Video