The ease with which current hardware and communication channels allow for the acquisition, processing and sharing of images has enabled a wide range of new usage scenarios. However, many of these applications are plagued by concerns for the privacy of the people visible in the scene. Examples include the recently introduced Google Streetview service, surveillance systems to help monitor patients in nursing homes, and the collection and distribution of medical face databases. While algorithms have been proposed to remove identifying information from images, currently available methods are lacking in many respects. In my dissertation I proposed a general framework for the de-identification of images which subsumes a number of previously introduced approaches. The goal is to remove as much identifying information from the image as necessary while preserving most of the original signal. To achieve this goal I developed factorization algorithms that separate the data into identity and non-identity related components using a framework which unifies linear, bilinear and quadratic data models. In experiments I was able to show that this approach preserves more data utility than other de-identification methods while maintaining privacy protection. The de-identification framework is tightly integrated with a generative face model, the Active Appearance Model, which enables real-time video face de-identification.
Ralph Gross, Latanya Sweeney, Fernando de la Torre, and Simon Baker. Model-Based Face De-Identification. IEEE Workshop on Privacy Research In Vision, 2006.
Ralph Gross, Edoardo Airoldi, Bradley Malin, and Latanya Sweeney. Integrating Utility into Face De-Identification. Workshop on Privacy Enhancing Technologies (PET), 2005.
Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to track faces in video. A variety of video applications are possible, including dynamic head pose and gaze estimation for real-time user interfaces, lip-reading, and expression recognition. To construct an AAM, a number of training images of faces with a mesh of canonical feature points (usually hand-marked) are needed. All feature points have to be visible in all training images. However, in many scenarios parts of the face may be occluded. Perhaps the most common cause of occlusion is 3D pose variation, which can cause self-occlusion of the face. Furthermore, tracking using standard AAM fitting algorithms often fails in the presence of even small occlusions. In this project we developed algorithms to construct AAMs from occluded training images and to track faces efficiently in videos containing occlusion. We evaluated our algorithms both quantitatively and qualitatively and showed successful real-time face tracking on a number of image sequences containing varying degrees and types of occlusions.
Ralph Gross, Iain Matthews, and Simon Baker. Active Appearance Models with Occlusion. Image and Vision Computing, 24(6), 2006, 593-604.
Ralph Gross, Iain Matthews, Simon Baker. Constructing and Fitting Active Appearance Models With Occlusion. First IEEE Workshop on Face Processing in Video (FPIV), 2004.
Generic vs. Person Specific Active Appearance Models
Active Appearance Models (AAMs) have been used very successfully to model faces. Anecdotal evidence, however, suggests that the performance of an AAM built to model the variation in appearance of a single person across pose, illumination, and expression (a Person Specific AAM) is substantially better than the performance of an AAM built to model the variation in appearance of many faces, including unseen subjects not in the training set (a Generic AAM). In this paper we present an empirical evaluation that shows that Person Specific AAMs are, as expected, both easier to build and more robust to fit than Generic AAMs. Moreover, we show that: (1) building a generic shape model is far easier than building a generic appearance model, and (2) the shape component is the main cause of the reduced fitting robustness of Generic AAMs. We then proceed to describe two refinements to Generic AAMs to improve their performance: (1) a refitting procedure to improve the quality of the ground-truth data used to build the AAM and (2) a new fitting algorithm. For both refinements we demonstrate dramatically improved fitting performance. Finally, we evaluate the effect of these improvements on a combined model construction and fitting task.
Ralph Gross, Iain Matthews, and Simon Baker. Generic vs. Person Specific Active Appearance Models. Image and Vision Computing, 2005. 23(11), 2005, 1080-1093.
Ralph Gross, Iain Matthews, Simon Baker. Generic vs. Person Specific Active Appearance Models. British Machine Vision Conference, 2004.
Participation in social networking sites has dramatically increased in recent years. Services such as Friendster, Tribe, or the Facebook allow millions of individuals to create online profiles and share personal information with vast networks of friends - and, often, unknown numbers of strangers. In a series of studies we examined the patterns of information revelation in online social networks and their privacy implications. We analyzed the online behavior of more than 4,000 Carnegie Mellon University students who have joined a popular social networking site catered to colleges. We evaluate the amount of information they disclose and study their usage of the site's privacy settings. We highlight potential attacks on various aspects of their privacy, and we show that only a minimal percentage of users changes the highly permeable privacy preferences. We furthermore surveyed a representative sample of the members of the Facebook and compared the survey data to information retrieved form the network itself.
Alessandro Acquisti and Ralph Gross. Imagined Communities: Awareness, Information Sharing, and Privacy on the Facebook. 6th Workshop on Privacy Enhancing Technologies, 2006.
Ralph Gross and Alessandro Acquisti. Information Revelation and Privacy in Social Networking Sites (or: Privacy on Facebook). Workshop on Privacy in the Electronic Society (WPES), 2005.