The ease with which current hardware and communication channels allow for the acquisition, processing and sharing of images has enabled a wide range of new usage scenarios. However, many of these applications are plagued by concerns for the privacy of the people visible in the scene. Examples include the recently introduced Google Streetview service, surveillance systems to help monitor patients in nursing homes, and the collection and distribution of medical face databases. While algorithms have been proposed to remove identifying information from images, currently available methods are lacking in many respects. In my dissertation I proposed a general framework for the de-identification of images which subsumes a number of previously introduced approaches. The goal is to remove as much identifying information from the image as necessary while preserving most of the original signal. To achieve this goal I developed factorization algorithms that separate the data into identity and non-identity related components using a framework which unifies linear, bilinear and quadratic data models. In experiments I was able to show that this approach preserves more data utility than other de-identification methods while maintaining privacy protection. The de-identification framework is tightly integrated with a generative face model, the Active Appearance Model, which enables real-time video face de-identification.

Publications:

Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to track faces in video. A variety of video applications are possible, including dynamic head pose and gaze estimation for real-time user interfaces, lip-reading, and expression recognition. To construct an AAM, a number of training images of faces with a mesh of canonical feature points (usually hand-marked) are needed. All feature points have to be visible in all training images. However, in many scenarios parts of the face may be occluded. Perhaps the most common cause of occlusion is 3D pose variation, which can cause self-occlusion of the face. Furthermore, tracking using standard AAM fitting algorithms often fails in the presence of even small occlusions. In this project we developed algorithms to construct AAMs from occluded training images and to track faces efficiently in videos containing occlusion. We evaluated our algorithms both quantitatively and qualitatively and showed successful real-time face tracking on a number of image sequences containing varying degrees and types of occlusions.

Publications:

Participation in social networking sites has dramatically increased in recent years. Services such as Friendster, Tribe, or the Facebook allow millions of individuals to create online profiles and share personal information with vast networks of friends - and, often, unknown numbers of strangers. In a series of studies we examined the patterns of information revelation in online social networks and their privacy implications. We analyzed the online behavior of more than 4,000 Carnegie Mellon University students who have joined a popular social networking site catered to colleges. We evaluate the amount of information they disclose and study their usage of the site's privacy settings. We highlight potential attacks on various aspects of their privacy, and we show that only a minimal percentage of users changes the highly permeable privacy preferences. We furthermore surveyed a representative sample of the members of the Facebook and compared the survey data to information retrieved form the network itself.

We found that an individual's privacy concerns are only a weak predictor of his membership to the network. Also privacy concerned individuals join the network and reveal great amounts of personal information. Some manage their privacy concerns by trusting their ability to control the information they provide and the external access to it. However, we found significant discrepancies between reality and some members' perceptions of the online community's reach and visibility of their profiles.

Publications: