• Matthias Grundmann, Vivek Kwatra, Mei Han, and Irfan Essa (2010) “Discontinuous Seam-Carving for Video Retargeting” (a GA Tech, Google Collaboration)
• Matthias Grundmann, Vivek Kwatra, Mei Han, and Irfan Essa (2010) “Efficient Hierarchical Graph-Based Video Segmentation” (a GA Tech, Google Collaboration)
• Kihwan Kim, Matthias Grundmann, Ariel Shamir, Iain Matthews, Jessica Hodgins, and Irfan Essa (2010) “Motion Fields to Predict Play Evolution in Dynamic Sport Scenes” (a GA Tech, Disney Collaboration)
• Raffay Hamid, Ramkrishan Kumar, Matthias Grundmann, Kihwan Kim, Irfan Essa, and Jessica Hodgins (2010) “Player Localization Using Multiple Static Cameras for Sports Visualization” (a GA Tech, Disney Collaboration)

Check out the list of final projects for this term’s (Spring 2010) class on Computational Journalism.  Final reports expected in last week of April.  Stay tuned.

Computation + Journalism (CS 4464 / CS 6465)

This class is aimed at understanding the computational and technological advancements in the area of journalism. Primary focus is on the study of technologies for developing new tools for (a) sense-making from diverse news information sources, (b) the impact of more and cheaper networked sensors (c) collaborative human models for information aggregation and sense-making, (d) mashups and the use of programming in journalism, (e) the impact of mobile computing and data gathering, (f) computational approaches to information quality, (g) data mining for personalization and aggregation, and (h) citizen journalism.

Computing, Society and Professionalism (CS 4001)

Although Computing, Society and Professionalism is a required course for CS majors, it is not a typical computer science course. Rather than dealing with the technical content of computing, it addresses the effects of computing on individuals, organizations, and society, and on what yourresponsibilities are as a computing professional in light of those impacts. The topic is a very broad one and one that you will have to deal with almost every day of your professional life. The issues are sometimes as intellectually deep as some of the greatest philosophical writings in history – and sometimes as shallow as a report on the evening TV news. This course can do little more than introduce you to the topics, but, if successful, will change the way you view the technology with which you work.

Abstract

This paper investigates the recognition of human actions from three-dimensional (3-D) point clouds that encode the motions of people acting in sensor-distributed indoor environments. Data streams are time sequences of silhouettes extracted from cameras in the environment. From the 2-D silhouette contours we generate space–time streams by continuously aligning and stacking the contours along the time axis as third spatial dimension. The space–time stream of an observation sequence is segmented into parts corresponding to subactions using a pattern matching technique based on suffix trees and interval scheduling. Then, the segmented space–time shapes are processed by treating the shapes as 3-D point clouds and estimating global point feature histograms for them. The resultant models are clustered using statistical analysis and our experimental results indicate that the presented methods robustly derive different action classes. This holds despite large intra-class variance in the recorded datasets due to performances from different persons at different time intervals.

© Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2009

Overview of the approach.

Keywords: Action recognition, point cloud, global features, action segmentation

Abstract

We introduce methods for augmenting aerial visualizations of Earth (from tools such as Google Earth or Microsoft Virtual Earth) with dynamic information obtained from videos. Our goal is to make Augmented Earth Maps that visualize the live broadcast of dynamic sceneries within a city. We propose different approaches to analyze videos of pedestrians and cars, under differing conditions and then augment Aerial Earth Maps (AEMs) with live and dynamic information. We also analyze natural phenomenon (clouds) and project information from these to the AEMs to add the visual reality.

Check out the media coverage of our new paper to appear in ISMAR 2009, in October.

Also see

• “Latest videos makes Google Earth cities bustle” New Scientist (Sep 30, 2009 Issue)
• “Video: Google Earth animated with real time human and vehicular traffic” Endgadget (Sep 30, 2009)

I am co-organizing the First IEEE Workshop on Computer Vision for Humanoids in conjunction with ICCV Conference in Kyoto, Japan.  This workshop will be held September 27, 2009. (9:30am – 6:00pm).

The goal of this workshop is to bring together experts from the fields of computer vision and robotics that are working on humanoid robots with vision as one of the primary modalities. Topics of interest include and are not limited to:

• Visual Learning in Robots
• Human Robot Interaction
• Grasping and Manipulation
• Learning by Demonstration
• Task Learning for Robots
• Activity Recognition and Discovery for Robot
• Humanoid Navigation in Real Environments
• Vision Devices and Systems for Robot Applications
• Application of Humanoid Robots (Indoor/Outdoor, Entertainment)

This is the first attempt at a workshop that crosses from Humanoids Research to Computer Vision Research.

The workshop includes six invited talks as well as an open poster session, where all participants are expected to present a poster describing their recent work.

Location: Kyoto University, Faculty of Engineering Bldg.#3, 2F, Room W201, in conjunction with ICCV. (See http://www.iccv2009.org/workshops/index.html).

For schedule, abstracts and other information, see the workshop website. More information about ICCV at http://www.iccv2009.org/.

Invited Speakers and Organizers after the Workshop

Abstract

We present an algorithm for creating realistic animations of characters that are swimming through fluids. Our approach combines dynamic simulation with data-driven kinematic motions (motion capture data) to produce realistic animation in a fluid. The interaction of the articulated body with the fluid is performed by incorporating joint constraints with rigid animation and by extending a solid/fluid coupling method to handle articulated chains. Our solver takes as input the current state of the simulation and calculates the angular and linear accelerations of the connected bodies needed to match a particular motion sequence for the articulated body. These accelerations are used to estimate the forces and torques that are then applied to each joint. Based on this approach, we demonstrate simulated swimming results for a variety of different strokes, including crawl, backstroke, breaststroke and butterfly. The ability to have articulated bodies interact with fluids also allows us to generate simulations of simple water creatures that are driven by simple controllers.

]]> http://prof.irfanessa.com/2009/06/10/swimmer/feed/ 0 http://prof.irfanessa.com/2009/06/08/times-2008-article-can-computer-nerds-save-journalism/ http://prof.irfanessa.com/2009/06/08/times-2008-article-can-computer-nerds-save-journalism/#comments Mon, 08 Jun 2009 14:08:13 +0000 Irfan Essa http://academics.irfanessa.com/?p=512 Can Computer Nerds Save Journalism?, TIME Magazine, by MATT VILLANO, June 8, 2009

EXCERPT

“At the Georgia Institute of Technology in Atlanta, a three-year-old program in “computational journalism” helps computer-science majors study how journalists gather, organize and utilize information, then take these workflows and see how technology can make the processes easier.”

Full article here. Also see CnJ site.

(Slides, NO Video)

Abstract

I will present a variety of temporal models of video that we have been studying (and developing on) for analysis and synthesis of video. Forsynthesis of videos, we have been developing representations that support example-based re-synthesis and spatio-temporal re-targeting. These approaches build on graph-based methods and we present techniques for similarity metrics for video, segmentation in video, and merging of different video streams. I will showcase a series of examples of these approaches applied to generate new videos.

For analysis of videos, we have developed a series of representations to observe and model activities in videos. Building on low-level measures of movement and motion in videos, we have incorporated higher-level temporal generative models to represent and recognize observed activities. I will discuss the strengths of a variety of State-based, Markovian, Grammar-based and Network-based representations that we have employed for recognizing activities from video. I will also discuss approaches for unsupervised discovery and recognition of activities.

Time permitting, I will describe some new efforts that move towards understanding mobile imaging and video, and video authoring and video on the web, Within these I will discuss issues of collaborative imaging, collective authoring, ad-hoc sensor networks, and peer production with images and videos. Using these concepts, to focus the conversation, I will discuss how all of these issues are impacting the field Journalism and Reporting and how we have started on a new interdisciplinary research and education effort, we call Computational Journalism.

Irfan Essa
Georgia Institute of Technology
School of Interactive Computing, GVU and RIM Centers
April 21, 2009.

(see the video of this presentation)

Abstract

Our consumption of images (photography/video) continues to grow with the pervasiveness of computing (networking, mobile and media) technologies into our daily lives. Everyone now has a mobile camera, and digital image capture, processing, and sharing has become ubiquitous in our society. This has led to a significant impact on we want to (a) create novel scenes, (b) share our experiences with images, and (c) interact with  large amounts of images and videos from many sources. In this talk, I will start  with a brief overview of series of ongoing efforts in the analysis of images and videos for rendering novel scenes, interacting with images/videos and collaboratively authoring new content. I will describe some work on video-based rendering and synthesizing novel videos (and scenes) and highlight the technical contributions being made in areas of Computational Photography and Video.

Using these sets of efforts as a foundation I will showcase where things are headed in terms of user generated content, media sharing, annotation, and reuse with large scale networks. In essence, everybody is a content, producer, distributor, and consumer. I will describe some new efforts that move towards understanding mobile imaging and video, and also discuss issues of collaborative imaging, collective authoring, ad-hoc sensor networks, and peer production with images and videos.  Using these concepts I will discuss how all of these issues are impacting the field Journalism and Reporting and how we have started on a new interdisciplinary research and education effort, we call Computational Journalism.  The concept of Computational Journalism includes more than just imaging, and relates to media and information in general and is aimed at the study of how we remain informed in this connected world. I will outline this new field and relate it back to imaging, with examples from some of our recent work in this new area.

Abstract

Matthew Flagg, Atsushi Nakazawa, Qiushuang Zhang, Sing Bing Kang, Young Kee Ryu, Irfan Essa, James M. Rehg (2009), Human Video Textures In Proceedings of the ACM Symposium on Interactive 3D Graphics and Games 2009 (I3D ’09), Boston, MA, February 27-March 1 (Fri-Sun), 2009 [PDF (see Copyright) | Video in DiVx | Website ]

Abstract

This paper describes a data-driven approach for generating photorealistic animations of human motion. Each animation sequence follows a user-choreographed path and plays continuously by seamlessly transitioning between different segments of the captured data. To produce these animations, we capitalize on the complementary characteristics of motion capture data and video. We customize our capture system to record motion capture data that are synchronized with our video source. Candidate transition points in video clips are identified using a new similarity metric based on 3-D marker trajectories and their 2-D projections into video. Once the transitions have been identified, a video-based motion graph is constructed. We further exploit hybrid motion and video data to ensure that the transitions are seamless when generating animations. Motion capture marker projections serve as control points for segmentation of layers and nonrigid transformation of regions. This allows warping and blending to generate seamless in-between frames for animation. We show a series of choreographed animations of walks and martial arts scenes as validation of our approach.

Human Video Textures (Output Rendered as a Collage!)

ABSTRACT

The natural language for most deaf signers in the United States is American Sign Language (ASL). ASL has internal structure like spoken languages, and ASL linguists have introduced several phonemic models. The study of ASL phonemes is not only interesting to linguists, but also useful for scalability in recognition by machines. Since machine perception is different than human perception, this paper learns the basic units for ASL directly from data. Comparing with previous studies, our approach computes a set of data-driven units (fenemes) discriminatively from the results of segmental feature selection. The learning iterates the following two steps: first apply discriminative feature selection segmentally to the signs, and then tie the most similar temporal segments to re-train. Intuitively, the sign parts indistinguishable to machines are merged to form basic units, which we call ASL fenemes. Experiments on publicly available ASL recognition data show that the extracted data-driven fenemes are meaningful, and recognition using those fenemes achieves improved accuracy at reduced model complexity

Computation & Journalism: The Impact of Technology on Journalism, Information Quality, and Civic Literacy

Irfan Essa
Georgia Institute of Technology
School of Interactive Computing, GVU and RIM Centers 

Fundamentally, journalism is the process of collecting news information and disseminating that information with a layer of contextualization and understanding provided by journalists in the form of a news story. Recent advances in computational technology are rapidly affecting how news is gathered, reported, and distributed, and how stories are authored and told. New technologies for aggregating, visualizing, summarizing, consuming, and collaborating on news are becoming increasingly popular. Theses advances are challenging the traditional practices of journalism and directly affecting the future of news production and consumption. Both computation and journalism share a deep interest in information and the value it provides to society, and they are deeply involved in the future of storytelling in various contexts, especially current events. This requires us to consider how both Computation and Journalism can help each other. 

In this talk, I will present a vision for a new area of research and education that brings together the fields of computation and journalism together to enhance both these disciplines and supports a creation of a “Computationalist-Journalist.,” a new kind of participant in the public conversation. I will start by describing how imaging, video, and media production and consumption has changed with technology and then how similar technologies can be used for Journalism and related Civic Literacy issues. I will describe new technologies that have changed the landscape of both Computation and Journalism and use these developments to showcase, where we are headed to with both Computation and Journalism, and technologists and journalists together to create new computing tools that further the aims of journalism. 

Bio

by JOHN MECKLIN

Dec 23, 2008 in Miller-McCune

“If you pay passing attention to the media landscape, you know that most mainstream news outlets have had their business models undermined by the digital revolution. As their general-interest monopolies have been pillaged by niche online competitors, traditional news organizations have lost revenue and cachet, laying off journalists in waves that have grown into tsunamis. This process has created dire prospects for the future of investigative reporting, often seen as the most costly of journalistic forms.”

Goes on to mention Computational Journalism and our (at GA Tech) and recent Duke University’s efforts in this space and few others.

ABSTRACT

We propose the use of 3D (2D+time) Shape Context to recognize the spatial and temporal details inherent in human actions. We represent an action in a video sequence by a 3D point cloud extracted by sampling 2D silhouettes over time. A non-uniform sampling method is introduced that gives preference to fast moving body parts using a Euclidean 3D Distance Transform. Actions are then classified by matching the extracted point clouds. Our proposed approach is based on a global matching and does not require specific training to learn the model. We test the approach thoroughly on two publicly available datasets and compare to several state-of-the-art methods. The achieved classification accuracy is on par with or superior to the best results reported to date.

This academic year, I am spending some time working with the newly formed Disney Research, Pittsburgh, (Directed by Jessica Hodgins) formed next to CMU.  The press release is announcing this lab is here (Carnegie Mellon SCS Press Release). I am also hanging out with folks at the CMU Robotics Institute and have started some new collaborations.  So now depending on when, you can find me either in Atlanta (at GA Tech) or in Pittsburgh (at Disney Lab or CMU) [OR on a airplane between Pittsburgh and Atlanta].