Long before the commercial success of the Internet, Brian J. Fox invented one of its most widely used tools.
In 1987, Fox, then a young programmer, wrote BASH, short for Bourne-Again Shell, a free piece of software that is now built into more than 70 percent of the machines that connect to the Internet. That includes servers, computers, routers, some mobile phones and even everyday items like refrigerators and cameras.
On Thursday, security experts warned that BASH contained a particularly alarming software bug that could be used to take control of hundreds of millions of machines, potentially including Macintosh computers and smartphones that use the Android operating system.
The bug, named Shellshock, drew comparisons to the Heartbleed bug that was discovered in a crucial piece of software last spring.
But Shellshock could be a bigger threat. While Heartbleed could be used to do things like steal passwords from a server, Shellshock can be used to take over the entire machine. And Heartbleed went unnoticed for two years and affected an estimated 500,000 machines, but Shellshock was not discovered for 22 years.
That a flawed piece of code could go unnoticed for more than two decades could be surprising to many. But not to programmers.
Many of the commercial tools individual users and large corporations depend upon are built on top of programs that are written and maintained by a few unpaid volunteers in what is called the open-source community. That community, along with big companies like Google, adjusts and builds new things on top of older work. The Macintosh operating system, for example, is routinely updated, but it is built on top of older programs like Unix.
Sometimes there are flaws in that code. And over the years, the flaw becomes part of all sorts of products.
Fox maintained BASH — which serves as a sort of software interpreter for different commands from a user — for five years before handing over the reins to Chet Ramey, a 49-year-old programmer who, for the past 22 years, has maintained the software as an unpaid hobby. That is, when he is not working at his day job as a senior technology architect at Case Western Reserve University in Ohio.
Ramey said in an interview Thursday that he believed he inadvertently introduced Shellshock in a new BASH feature in 1992, although he could not be sure because back then he was not keeping comprehensive logs. Through the years, he maintained Bash by himself and occasionally bug reports would arrive in his email inbox.
On Sept. 12, he was contacted by Stephane Chazelas, another open-source enthusiast, about a potentially dangerous bug.
Chazelas discovered the flaw after finding a similar issue in another system a few months back. He tested the bug — which he called Bashdoor — against his own servers and looked for ways to detect and fix it.
Working with Ramey and people who work on open-source security, Chazelas had a patch within hours. Then they contacted major software makers while trying to avoid tipping off hackers.
An official alert from the National Institute of Standards and Technology warned that the vulnerability was a 10 out of 10, in terms of its severity, impact and exploitability, but low in terms of its complexity, meaning that it could be easily used by hackers.
Security researchers say that as soon as the bug was reported they detected widespread Internet scanning by “white hat” hackers — most likely security researchers — as well as people thought to be cybercriminals. The worry is that it is only a matter of time before somebody writes a program that will use Shellshock to take them over.
Researchers noted that it would be much easier for this to happen with Internet-connected servers than with a personal Macintosh laptop, because individuals would have to connect their laptops to a public network that hackers knew they were connected to in order to exploit the vulnerability.
Apple did not return a call seeking comment.
The Department of Homeland Security’s Computer Emergency Readiness Team, US-CERT, advised users and technology administrators to refer to their Linux or Unix-based operating systems suppliers for an appropriate patch. For users at home, security experts advised them to stay abreast of software updates and check manufacturer websites, particularly for hardware like routers.
Even as some question the open-source community, its biggest advocates say the bug’s discovery - even after 22 years - at least proves that programmers never stop trying to get things right.
In an interview Thursday, Fox, the BASH inventor, joked that his first reaction to the Shellshock discovery was, “Aha, my plan worked.”
After the Heartbleed bug was discovered last spring, the nonprofit Linux Foundation worked with major technology companies like Amazon, Apple and Google on the Core Infrastructure Initiative, an effort to identify and fund core pieces of open-source infrastructure. Contacted Thursday, Jim Zemlin, executive director of the Linux Foundation, said the initiative was contacting Ramey to see how it could help.
“I don’t think this is an open-source problem,” Zemlin said. “Software is eating the world. The bad news is software is hard and complex.”
The mantra of open source was perhaps best articulated by Eric J. Raymond, one of the elders of the open-source movement, who wrote in 1997 that “given enough eyeballs, all bugs are shallow.” But, in this case, Steven M. Bellovin, a computer science professor at Columbia University, said, those eyeballs are more consumed with new features than quality.
“Quality takes work, design, review and testing and those are not nearly as much fun as coding,” Bellovin said. “If the open-source community does not develop those skills, it’s going to fall further behind in the quality race.”