Irfan Essa’s Academic Activities

GT Research Horizons — Fall 2003

Anne Eisenberg (2001)“A ‘Smart’ Home, to Avoid the Nursing Home”New York Times Circuits Section,

April 5, 2001 Issue

Quote from the Article: “Cameras are going to rule one day at the Georgia Tech house, though, staff members there say. Dr. Irfan A. Essa, a computer science professor at Georgia Tech, is one of the people building a tracking system, based on video cameras, that will one day replace radio frequency tags. ”We can locate where the person is,” Dr. Essa said, ”and make a first-level guess at where this person is heading using the optical sensors.””

Ubiquitous sensing for smart and aware environments

Essa, I.A.
Coll. of Comput., Georgia Inst. of Technol., Atlanta, GA, USA;
This paper appears in: Personal Communications, IEEE [see also IEEE Wireless Communications]
Publication Date: Oct. 2000
Volume: 7 , Issue: 5
On page(s): 47 - 49
ISSN: 1070-9916
CODEN: IPCME7
INSPEC Accession Number:6756447
Digital Object Identifier: 10.1109/98.878538
Posted online: 2002-08-06 23:40:31.0

Abstract

As computing technology continues to become increasingly pervasive and ubiquitous, we envision the development of environments that can sense what we are doing and support our daily activities. In this article, we outline our efforts toward building such environments and discuss the importance of a sensing and signal-understanding infrastructure that leads to awareness of what is happening in an environment and how it can best be supported. Such an infrastructure supports both high- and low-end data transmission and processing, while allowing for detailed interpretation, modeling and recognition from sensed information. We are currently prototyping several aware environments to aid in the development and study of such sensing and computation in real-world settings

The Aware Home

Irfan A. Essa “Computers Seeing People”, AI Magazine 20(2): Summer 1999, 69-82

Abstract

AI researchers are interested in building intelligent machines that can interact with them as they interact with each other. Science fiction writers have given us these goals in the form of HAL in 2001: A Space Odyssey and Commander Data in Star Trek: The Next Generation. However, at present, our computers are deaf, dumb, and blind, almost unaware of the environment they are in and of the user who interacts with them. In this article, I present the current state of the art in machines that can see people, recognize them, determine their gaze, understand their facial expressions and hand gestures, and interpret their activities. I believe that by building machines with such abilities for perceiving, people will take us one step closer to building HAL and Commander Data.

Award#9806822 - Experimental Software Systems: Automated Understanding of Captured Experience
ABSTRACT

9806822 Essa, Irfan A. Abowd, Gregory D. Georgia Institute of Technology Experimental Software Systems: Automated Understanding of Captured Experience The objective of this research is to reduce substantially the human input necessary for creating and accessing large collections of multimedia, particularly multimedia created by capturing what is happening in an environment. The existing software system which is being used as the starting point for this investigation is Classroom 2000, a system designed to capture what happens in classrooms, meetings, and offices. Classroom 2000 integrates and synchronizes multiple streams of captured text, images, handwritten annotations, audio, and video. In a sense, it automates note-taking for a lecture or meeting. The research challenge is to make sense of this flood of captured data. The project explores how the output of Classroom 2000 can be automatically structured, segmented, indexed, and linked. Machine learning and statistical approaches to language are used to attempt to understand the captured data. Techniques from computational perception are used to try to find structure in the captured data. An important component of this research is the experimental analysis of the software system being built. The expectation is that this research will have a dramatic impact on how humans work and learn, as technology aids humans by capturing and making accessible what happens in an environment.

Alex Pentland (1996), “Smart Rooms”Scientific American, April 1996

Quote from the Article: “Facial expression is almost as important as identity. A teaching program, for example, should know if its students look bored. So once our smart room has found and identified someone’s face, it analyzes the expression. Yet another computer compares the facial motion the camera records with maps depicting the facial motions involved in making various expressions. Each expression, in fact, involves a unique collection of muscle movements. When you smile, you curl the corners of your mouth and lift certain parts of your forehead; when you fake a smile, though, you move only your mouth. In experiments conducted by scientist Irfan A. Essa and me, our system has correctly judged expressions-among a small group of subjects-98 percent of the time.”