Profile: Hany Farid

PROFILE: HANY FARID

PBS Airdate: June 25, 2008

NEIL deGRASSE TYSON: They say a picture is worth a thousand words, but what if those words are just lies?

With digital technology, anybody can appear to be or do just about anything. So what do we do when seeing is no longer believing? When it's gotten harder to tell fact from forgery?

Well, in this episode's profile, you'll meet a scientist who's developing the forensic tools to recognize a fake when he sees it.

Hany Farid is 42 years old, and spends much of his time making fakes and forgeries, images that are meant to trick the human eye.

HANY FARID: When I am creating a forgery, it is a little bit like being an artist. You have to think about, well, the colors and the palettes, and the lighting and the contrasts, and all the various things that artists probably think about.

I find it really fascinating to try to create these visual fakes. I think it's interesting, purely from an aesthetic and artistic point of view.

NEIL deGRASSE TYSON: But Hany is not a forger or a crook. He's a Dartmouth College computer science professor who likes taking chances. He rides a Harley.

HANY FARID: There's something really exciting and risky and slightly dangerous about it. I wouldn't give it up for anything.

NEIL deGRASSE TYSON: And Hany is anything but predictable.

HANY FARID: So on any given morning, I might take a shower, brush my teeth, shave, get dressed and leave. And on another day, I'll shave, shower, brush my teeth. Or I'll brush my teeth, shower and shave. Or I'll brush my teeth, have my coffee, and then go back and do it. What happens is that when I fall into a routine, I think my thinking falls into a routine.

NEIL deGRASSE TYSON: Thinking is his obsession.

HANY FARID: You have to wake up and you have to go to sleep thinking, thinking about the problem and turning it around and spinning it.

So everywhere I go, I have a pad of paper and a pen.

NEIL deGRASSE TYSON: What he's thinking about is something called digital forensics, an entirely new field that Hany helped pioneer.

HANY FARID: The basic idea is... I mean, think forensics the way you would think CSI forensics, but now it's pixels instead of hair.

NEIL deGRASSE TYSON: As a digital detective, Hany takes what he learns from making forgeries...

HANY FARID: So imagine you're somebody who is trying to catch a counterfeiter. How do you learn? How do you learn how to detect that? Well, you have to know how they do it.

NEIL deGRASSE TYSON: ...and he creates powerful software. When he runs that software on a photograph, he can show if it's real or fake. Like this Time Magazine cover of O.J. Simpson, well known to have been manipulated.

HANY FARID: I think we do want to believe what we see; it's our first reaction. I, however, think that is changing. More and more, when people see photographs that are fantastic, there is an initial, "Oh, how do we know it hasn't been Photoshopped?"

The reality is that photographs have been manipulated since the 1800s, since the start of photography, actually. So you're talking about something that's been going on for hundreds of years. And if you look throughout history, in fact, all the great dictators doctored photographs to change history. Stalin did it, Mussolini, Castro, Mao. I mean they all manipulated photographs.

NEIL deGRASSE TYSON: In the digital age, we're flooded with images, and the problem of the tampered photograph has only gotten worse.

Hany and his team of graduate students have joined forces with the FBI and other law enforcement agencies, but how can they beat the forgers?

HANY FARID: Most of what we think about is authentication. Can you show that something has been added to the image? Has it been removed from the image? Has this person been airbrushed? Is it somebody else's head on somebody else's body? So think about all the types of digital fakes you've seen: you know, the guy holding the really big cat, the really big hogzilla. On all those things, we ask the question, "Is this a legitimate photograph?"

NEIL deGRASSE TYSON: They approach it like a detective would.

MICAH K. JOHNSON (Dartmouth College): In a sense, images contain natural fingerprints. So we build tools that can detect these fingerprints.

HANY FARID: One of the most common ways of manipulating a photograph is to remove something or somebody from a photograph. And the most common way to do that is to typically take one part of an image, copy it and paste it into another part of the image to hide that person or that object. And when you do that, you've left behind a very specific statistical trace. You've left behind two regions that are virtually identical.

NEIL deGRASSE TYSON: And that process is called cloning.

HANY FARID: Probably the most famous example of cloning was the photograph that Reuters published that came out of Lebanon after an Israeli strike. It showed smoke billowing out of a building.

NEIL deGRASSE TYSON: To Hany, something about the smoke looked unnatural. But how could he confirm his suspicions?

Hany created software that detects identical pixel patterns.

HANY FARID: So we've developed a very efficient algorithm that can detect whether two parts of an image are identical. We call this the anti-cloning tool. The software analyzes it, and what comes back is a color-coded image that says all of these pixels which we color-code in red, for example, are the same as all of these pixels which were color-coded blue. So there are these two highlighted areas. It's very distinct, and it's very recognizable.

NEIL deGRASSE TYSON: Hany's software confirmed that the photographer had digitally added more smoke after the photo was taken. Portions of the image had been cloned.

Hany was able to develop his next technique, thanks to the budding relationship of Brad Pitt and Angelina Jolie, in 2005.

HANY FARID: This is when they were still rumored to have a relationship, and everybody was dying to get that photograph of them together.

NEIL deGRASSE TYSON: So was this infamous photo real or fake? Hany could see with his own eyes that something was very wrong. His clues were the light and the shadows.

HANY FARID: Interestingly, by the way, most people don't notice it. They look at the image, two beautiful people. Who cares, right, where the light was? It doesn't matter to us.

NEIL deGRASSE TYSON: Just by looking, Hany could tell it was a fake.

HANY FARID: The light was off by a good 120 to 130 degrees. It was completely opposite directions. So for Brad Pitt the light was on his left, and for Angelina Jolie, the light was on her right.

NEIL deGRASSE TYSON: Which means there were two separate photographs made to look like one.

HANY FARID: So that made us think, well, this would really be a really cool digital forensics tool. What if we could estimate, from an image, where the light was?

NEIL deGRASSE TYSON: Thanks to "Brangelina," Hany used linear algebra and physics to develop his light-direction software. It can tell you where the light came from when the picture was taken.

It was the perfect tool for the 2004 presidential campaign.

HANY FARID: One of the most damaging political forgeries was the image of John Kerry sharing a stage with Jane Fonda at an anti-war rally. So this was when Senator Kerry was still trying to get the Democratic nomination for President, and when this image broke, it made the headlines. People were talking about it for weeks.

What was interesting about that particular image is that when you look at it, you can't tell that the lighting is different.

NEIL deGRASSE TYSON: The photo looked real, but when Hany used his light-direction tool, it revealed a completely different story.

HANY FARID: It was absolutely a fake. We were able to detect differences in lighting between Jane Fonda and John Kerry by as much as a thirty degrees difference.

NEIL deGRASSE TYSON: Which means, once again, this image was a combination of two photos.

Hany's light-direction software could determine lighting in two dimensions. But he wanted to find a way to work three-dimensionally. The key was in the eyes.

HANY FARID: The eyes are a wonderful thing because they are a partial mirror into the world in which you were photographed.

MICAH JOHNSON: So we had this idea that you could actually measure properties of the lighting environment from reflections in somebody's eye.

HANY FARID: If you look closely at my eyes, what you can probably see is that there must be a light off to my right and slightly above. And it's actually rectangular in shape. And you can see that because of where the white is in my eyes. And you can probably see a reflection of that directly in my eye.

NEIL deGRASSE TYSON: Using the geometry of the human eyeball, Hany developed a tool that exposed the position of the subject and the light source in 3-D. Armed with his new eyeball software, he was faced with this photo of the American Idol hosts.

HANY FARID: The Associated Press was about to run this. And there was something about the photograph that bothered all of us when we saw it.

NEIL deGRASSE TYSON: Once again, Hany's new software confirmed his suspicions.

HANY FARID: When we went in and looked at the eyes, we could tell that the light in which each was photographed was completely different.

The people who were photographed with two lights, we saw two dots on their eyes. For the person with the flash, we saw a very small dot in the center of the eyes. And for the other person, we saw a different shape and a different location. And so it was radically different lighting in that case. And it was clearly a composite of three photographs.

I think, from the point of view of the Associated Press, this is not an appropriate manipulation, and they just didn't publish the photograph.

NEIL deGRASSE TYSON: In this digital age, Hany's new technologies are indispensable. But developing a new field of science is never an easy mission.

HANY FARID: When I first got to Dartmouth, my post-doc advisor said to me, "I don't think it'll work." And when somebody who's a lot smarter than you says something's not going to work, you better pay attention.

If somebody tells me something won't work, I'm more determined than ever to make it work.

NEIL deGRASSE TYSON: Hany's strong will started a long time ago, when he was a young boy growing up in Rochester, New York.

HANY FARID: I think my grades and my abilities in school were either very good or very, very bad. And there was really nothing in between. There are things that really resonate with me, and I will commit myself to it, and there are other things that I find uninteresting, and I can't for the life of me make it work. And I just couldn't really get myself to do the work. I was in the slow lane a little bit.

My mother called me one day when I was in college, and—I think I was a junior at the time—I still didn't have a major. I was a C student. She'd read an article in the Time Magazine saying computer science is going to be big. She said, "You know, you should think about taking a class." And I thought, "Yeah, right, Mom." Like, what else would I say? But you know what? It sat with me, and I ended up taking a class, and it was great. It was one of the first things I was actually good at, and it changed my entire life.

Listen to your mother!

NEIL deGRASSE TYSON: In less than a decade, Hany has taken his rule-breaking brand of computer science and mathematics and helped pioneer the field of digital forensics. And he's caught the attention of major players along the way.

Now Adobe, developer of the software Photoshop, has invited Hany to work with them.

HANY FARID: They were interested in collaborating with my lab at Dartmouth, thinking about whether we can transfer that technology into software that could actually be used by law enforcement, media outlets, scientific publishers, to bring some of the techniques that we developed to a broader audience.

NEIL deGRASSE TYSON: Because of Adobe, the use of Hany's software is likely to become widespread in the battle against the forgers. Soon we may all know if what we're looking at is real or fake.

KEVIN O'CONNOR (Adobe): The research that Hany is doing is going to enable people to continue believing in what they see.

NEIL deGRASSE TYSON: So in the end, can Hany win against the forgers?

HANY FARID: I think the game of forensics is not you stop forgeries. That's an unrealistic and naive goal. What it is is you make it increasingly more difficult. I think, at the end the day, the average forger—the fourteen-year-old, in Ohio, in his bedroom, making forgeries and posting them on the web—they will lose. We will win that game. There is no doubt about it. But if I wanted to play the other side, I will always be able to beat the authentication game.

I'll be able to create a fake, but you won't.

Profile: Hany Farid

Edited by

Cherry Enoki

Produced by

Joshua Seftel and Ann Kim

Directed by

Joshua Seftel

NOVA scienceNOW

Executive Producer

Samuel Fine

Executive Editor

Neil deGrasse Tyson

Senior Series Producer

Vincent Liota

Supervising Producers

Stephen Sweigart
Joey David

Editorial Producer

Julia Cort

Development Producer

Vinita Mehta

Senior Editor

David Chmura

Production Assistant

Fran Laks

Compositors

Brian Edgerton
Yunsik Noh

Music

Rob Morsberger

Associate Producers

Fran Laks
Molly Longstreth
Anthony Manupelli
Win Rosenfeld
Alison Snyder

Assistant Editor

Susan Perla

Camera

Jim Ball
James Callanan
Austin deBesche
Brian Dowley
Vincent Liota
Steve McCarthy

Sound Recordists

Bernie Beaudry
Lauretta Molitor
Roger Phenix
George Shafnacker
Tom Williams

Sound Mix

David Chmura

Animation

Sputnik Animation
James LaPlante
Brian Edgerton
Anthony Kraus
Yunsik Noh
"The Forgetting," Twin Cities Public Television

Production Manager For Crowd Segment

Candace White

Segment Production Assistants

Mona Damluji
Grisha Enikolopov

NOVA scienceNOW series animation

Edgeworx

Three dimensional brain animation

Courtesy Dr. Arthur W. Toga, Laboratory of Neuro Imaging at UCLA

Archival Material

ESA
Jet Propulsion Laboratory
Library of Congress
NASA

Special Thanks

Briarwood Healthcare and Rehabilitation Center, Needham, MA
Douglas Clowe
Bamidele Faboyede
Prasad Jayanti
Kimo Johnson
Eric Kee
Sue Moen
Soudan Underground Mine - A Minnesota State Park
Twin Cities Public Television
Jeff Woodward

Neil deGrasse Tyson

is director of the Hayden Planetarium in the Rose Center for Earth and Space at the American Museum of Natural History.

NOVA Series Graphics

yU + co.

NOVA Theme Music

Walter Werzowa
John Luker
Musikvergnuegen, Inc.

Additional NOVA Theme Music

Ray Loring
Rob Morsberger

Post Production Online Editor

Spencer Gentry

Closed Captioning

The Caption Center

NOVA Administrator

Ashley King

Publicity

Carole McFall
Eileen Campion
Lindsay de la Rigaudiere
Victoria Louie
Kate Becker

Senior researcher

Gaia Remerowski

Production Coordinator

Linda Callahan

Paralegal

Raphael Nemes

Talent Relations

Scott Kardel, Esq.
Janice Flood

Legal Counsel

Susan Rosen

Post Production Assistant

Darcy Forlenza

Associate Producer, Post Production

Patrick Carey

Post Production Supervisor

Regina O'Toole

Post Production Editors

Rebecca Nieto
Alex Kreuter

Post Production Manager

Nathan Gunner

Compliance Manager

Linzy Emery

Supervising Producer

Stephen Sweigart

Business Manager

Joseph P. Tracy

Producers, Special Projects

Lisa Mirowitz
David Condon

Coordinating Producer

Laurie Cahalane

Senior Science Editor

Evan Hadingham

Senior Series Producer

Melanie Wallace

Managing Director

Alan Ritsko

Senior Executive Producer

Paula S. Apsell

This material is based upon work supported by the National Science Foundation under Grant No. 0638931. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

NOVA scienceNOW is a trademark of the WGBH Educational Foundation

NOVA scienceNOW is produced for WGBH/Boston by NOVA

© 2008 WGBH Educational Foundation

All rights reserved

Image credit: (Hany Farid) Courtesy Joe Mehling

Hany Farid

Computer Scientist, Dartmouth College www.cs.dartmouth.edu/farid/

Micah Johnson

Graduate Student, Dartmouth College

Kevin O'Connor

Adobe