Irfan Essa’s Academic Activities

I was invited to participate and present at the CETEE 2007, Islamabad, November 27-28, 2007.

This meeting has recently been postponed.

I have just been invited to give an Invited Talk at Advanced Visual Interfaces (AVI) 2008, May 28-30, 2008, in Napoli, Italy.

Here is a tentative title/abstract for this future presentation. Thanks for the AVI 2008 organizers for inviting me.

Computational Photography and Video: Interacting and Creating with Videos and Images

Abstract

Digital image capture, processing, and sharing has become pervasive in our society. This has had significant impact on how we create novel scenes, how we share our experiences, and how we interact with images and videos. In this talk, I will present an overview of series of ongoing efforts in the analysis of images and videos for rendering novel scenes. First I will discuss (in brief) our work on Video Textures, where repeating information is extracted to generate extended sequences of videos. I will then describe some our extensions to this approach that allows for controlled generation of animations of video sprites. We have developed various learning and optimization techniques that allow for video-based animations of photo-realistic characters. Using these sets of approaches as a foundation, then I will show how new images and videos can be generated. I will show examples of Photorealistic and Non-photorealistic Renderings of Scenes (Videos and Images) and how these methods support the media reuse culture, so common these days with user generated content. Time permitting, I will also share some of our efforts on video annotation and how we have taken some of these new concepts of video analysis to undergraduate classrooms.

Computational Photography and Video: Spatio Temporal Analysis for Synthesis of Novel Images and Videos.

ABSTRACT

Digital image capture and processing has recently had a significant impact on the computer graphics quest for rendering novel scenes. In this talk, I will present an overview of series of ongoing efforts in the analysis of images and videos for rendering novel scenes. First I will discuss (in brief) our work on Video Textures, where repeating information is extracted to generate extended sequences of videos. I will then describe some our extensions to this approach that allows for controlled generation of animations of video sprites. We have developed various learning and optimization techniques that allow for video-based animations of photo-realistic characters. Then I will describe additional approaches for image and video synthesis that builds on optimal patch-based copying of samples. I will show how our methods allow for iterative refinement, with a variety of optimization criteria, and all for extension to synthesis of both images and video from very limited samples. Using these sets of approaches as a foundation, then I will show how new images and videos can be generated. I will show examples of Photorealistic and Non-photorealistic Renderings of Scenes (Videos and Images) and how these methods support the media reuse culture, so common these days with user generated content. Time permitting, I will also share some of our efforts on video annotation and how we have taken some of these new concepts of video analysis to undergraduate classrooms.

WIAMIS 2007: “Data-driven and Procedural Analysis and Synthesis of Multimedia”

Abstract

In this talk, I will outline the changes that have come about in the analysis and synthesis of multimedia, due to the availability of large amounts of data. I will present several of the recently successful methods that have been introduced in the last few years for example-based synthesis for animation and rendering of videos. I will also show how these methods have been extended to other modalities. I will also show how these approaches need to be extended by developing parametric and procedurals models to represent temporal variations. Using example from my groups work and also other efforts, I will discuss how video is becoming an accessible medium for all and I will also discuss some newer work on authoring of multimedia content.

• Irfan Essa (2004), “Temporal Reasoning from Video to Temporal Synthesis of Video” Talk at USC’s IRIS-Vision Seminars (Fall 2004).

Temporal Reasoning from Video to Temporal Synthesis of Video

Irfan Essa (2003), “Analysis and Synthesis of Video for Special Effects and Animation:
A showcase of research and educational endeavors.” Invited talk at Ohio State University’s ACCAD Center, October, 7, 2003.

• Irfan Essa, “Temporal Reasoning from Video to Temporal Synthesis of Video” CMU’s Robotics Institute: Seminar, February 15, 2002

Temporal Reasoning from Video to Temporal Synthesis of Video

Abstract

In this talk, I will present some ongoing work on extracting spatio-temporal cues from video for both synthesis of novel video sequences, and recognition of complex activities. First I will discuss (in brief) our work on Video Textures, where repeating information is extracted to generate extended sequences of videos. I will then describe some our extensions to this approach that allows for controlled generation of animations of video sprites. We have developed various learning and optimization techniques that allow for video-based animations of photo-realistic characters. Then I will describe our new approach for image and video synthesis that builds on optimal patch-based copying of samples. I will show how our method allows for iterative refinement and extend to synthesis of both images and video from very limited samples. In the next part of my talk, I will describe how a similar analysis of video can be used to recognize what a person is doing in a scene. Such an analysis of video, aimed at recognition, requires more contextual information about the environment. I will show how we leverage off contextual information shared between actions and objects to recognize what is happening in complex environments. I will also show that by adding some form of grammar (we use Stochastic Context Free Grammar) we can recognize very complex, multi-tasked activities. Finally, I will describe (very briefly) the Aware Home project at Georgia Tech, which is one primary area of ongoing and future research for me and my group. Further information on my work with videos is available from my webpage at http://www.cc.gatech.edu/~irfan

Computer Vision and the Art of Special Effects