Paper Shows Airline Profiling IneffectiveBy Brian Loux
Many critics of airport profiling have derailed it as racist and an unlawful invasion of privacy. But Aaron B. Strauss G and Samidh Chakrabarti G added something new to the list last spring: an aid to terrorists.
The Computer-Assisted Passenger Prescreening System was instated first in 1997 to single out passengers that posed a significant threat for an explosive device check. After Sept. 11, 2001, CAPPS was expanded to secondary checkpoint screening.
Strauss and Chakrabarti decided to analyze the system in their Ethics and Law on the Electronic Frontier (6.805) class. Chakrabarti “had the idea of writing a report on profiling,” Strauss said. “We brainstormed how CAPPS works and if it could be successfully implemented.”
It soon became apparent to the two students that the system would not. They concluded that terrorist cells could send members on flights without any intention of hijacking the plane just to see whether or not members would be flagged. After testing the system enough, the cell could confidently send hijackers onto a flight knowing they would not be screened. The effect was termed “carnival booth effect,” because, according to Strauss, “just like a carnie, the FBI invites the terrorists to ‘step right up’ and be flagged.”
“The idea of a carnival booth effect was just a result of a logical step-by-step review of the process,” said Strauss. “The logical flaw just became apparent after researching and examining the CAPPS system.”
“I thought Samidh and Aaron were proposing a great project, because of its scope and ambition,” said Professor Harold Abelson PhD ’73. “Seeing students perform such outstanding work is one of the real joys of teaching at MIT.”
After they submitted the paper for the class, Professors Abelson and Michael M. J. Fischer urged the students to publish the paper.
“If you've discovered a flaw, chances are that the bad guys have discovered it, too, and publication is an effective way to get the flaw repaired,” Abelson said.
The paper made its way into the hands of government officials and airline security experts. The students were interviewed for a related story that aired on Boston’s National Public Radio station last Wednesday, and the paper will be featured in the Conde Naste Traveler magazine.
TSA says paper a non-issue
The information technology department at the Transportation Security Administration has examined the paper, but according to Spokeswoman for the TSA Heather Rosenker, the paper “doesn’t matter.”
“Their whole premise is inapplicable to the new process that we are going to [implement] in the near future,” said Rosenker. The new system “is more about the placement of a person in their community ... there will be more robust algorithms drawn from community databases.”
Strauss disagreed. “Waleed Al Shehri, a Sept. 11 hijacker, had lived in the United States since 1994. We should learn from our previous mistakes and learn that terrorists are willing to live in the United States for long periods of times in order to commit horrendous acts.”
“The risk that Samidh and Aaron uncover in the paper is pretty fundamental, and simply making the system more complex won’t get around that,” Abelson said. “Of course, with the TSA being secret about what they’ve actually done, it’s hard to judge.”
The TSA is moving ahead with CAPPS II, a more complex system that is expected to open to be implemented in the late fall. According to a press release, the new system will now probe “numerous databases from government, industry, and the private sector,” to determine whether a passenger is a threat.
Strauss and Chakrabarti are not seeking to push their case with the government. Asked about what they plan to do with the paper in the future, Strauss said they are “trying to have the paper published in a printed journal.”
Paper concludes CAPPS illegal
A logical argument is not all that the paper presents. “They did an analysis and developed mathematical and computer models, and then did a legal analysis of the consequences of their findings,” Abelson said.
The computer model in the paper, according to Strauss, varies the distribution of threat indices over the general population and over terrorist organizations.
For some runs “we gave the government the benefit of the doubt by basically saying that they have a good profile of traditional terrorists,” Strauss said. “But the key flaw of CAPPS is that even with a good profile, terrorists can beat the system by sending in dummy probes.”
On average, after a terrorist group successfully sends members on three round-trip test flights, the chance that they will be stopped by CAPPS on the next flight is no greater than through random screening.
The subsequent legal aspect of the paper examines Supreme Court rulings on security measure limitations. Security measures such as CAPPS have been upheld by the Supreme Court by passing the “stop-and-frisk exception” where an authority has probable cause that the person may pose danger. “Since random searches can catch more terrorists,” wrote Chakrabarti and Strauss, “airport security cannot therefore legitimately establish that those passengers flagged by CAPPS have an enhanced likelihood of harm. Consequently, CAPPS fails to meet the standards of the stop-and-frisk exception.”
Strauss also said that people should focus more on the recommendations of the paper. “We’re not just a wrecking ball destroying facets of airport security,” he said. “We should shift our resources away from CAPPS and put them towards security measures that work.”
Experts stress baggage checks
Strauss and Chakrabarti made suggestions for improvements to airline security. One suggestion was that screening bags would be a far more important step than CAPPS.
Daniel Biran, an airline security expert at Kroll, a Washington-based risk consulting company, said that he understood what Strauss and Chakrabarti were saying. Mentioning that the criterion for CAPPS is not openly published, Biran said, “when the only thing used for profiling is technology, there will always be ways around it.”
Biran said that based upon a decision by Congress, “basically by Nov. 29 this year, every checked bag will be screened so there will be no advantage to [CAPPS].”
Strauss also mentioned that a terrorist’s behavior before committing an act almost always serves as a warning sign.
Biran, a former member of El Al, an Israeli airline famous for its scrutiny and flawless record, security, also said that human intervention is important. “When you bring in a human factor, it not only looks at an output, but you observe behavior and you will upgrade the level of security you are going to give. Your chances of identifying a problem will be higher,” he said.
MIT Police Chief John Difava, who served on the Logan Task Force Committee after Sept. 11, said that the paper had a good argument, but not enough to abandon the system. “What we were doing then was treating everyone as a terrorist instead of figuring out who the terrorists were, and that point resounded to me,” he said, recalling the discussions about the pre-Sept. 11 system.
“The thing to remember about 9/11 is that there was no security breach,” he said. “The people that got on board to the best of the knowledge of the investigators were legally authorized by the FAA rules and regulations. What happened at Logan would have happened at any other airport.”
Difava said that multiple levels of security are key to avoiding such a disaster. “Security is in layers, there is no silver bullet,” he said. “Every 10th car going into Logan and every 5th car through the parking lot [is screened]. Could the terrorist be the 9th car? Positively, but the terrorists have to take that risk.”
Sheila E. Widnall ’61, an Institute professor also on the task force, declined to comment on the issue.