Open Source: The Story of the Great Code Giveaway

Open-source software is now so embedded in daily life, it is difficult to remember a time without it. Even laypeople—non-developers—have an idea of what it is. Software such as Linux, Struts, Spring, JUnit, Maven, jQuery, Node, Angular, and React is used not just by Internet companies but the vast majority of traditional corporations, too. And this has all happened in a very short period of time.

What is the real open source?

Although the Wikipedia entry on open source makes reference to Donald Knuth’s TeX (in 1978), this was just a language specification that was shared with other academics in the same way language specifications before it had been shared, and various individuals created their own implementations of that specification. What we think of today as open source is more than just a specification. We think of snippets, or even entire libraries, of source code that can be downloaded and used in a project.

The second reference on the Wikipedia page is Richard Stallman’s GNU (an alternative to Unix and a programmer’s joke: GNU’s not Unix), the operating system he created in 1984. This is a much better candidate for being named a progenitor of open source, but it is still not exactly what we know as open source today. This is because Stallman has been a lifelong activist for free software, and, as it says on his own Wikipedia page, has “spent most of his time advocating for free software, as well as campaigning against software patents, digital rights management (which he referred to as digital restrictions management, calling the more common term misleading), and other legal and technical systems which he sees as taking away users’ freedoms.”

Stallman embedded something like a virus into GNU. Not a computer virus written in code, though—a legal virus, the “copyleft” General Public License (GNU GPL). GNU’s copyleft ensures that any software that uses any free GNU GPL source code is legally bound to be distributed for free. So GNU was more about ensuring software would be free than it was about the comunal-code ownership and the unrestrictive licensing models that characterize today’s open-source movement. We will look at these two things in greater depth later.

Today’s normal was yesterday’s unthinkable

The tension between commercial purveyors of software and those who would see it shared with the world for free has been present almost from the beginning of the personal-computer revolution. It can be seen as early as 1976, when a young Bill Gates wrote his open letter to computer hobbyists.

The background to this letter was that there was a sense that MITS, the company that created the Altair 8800 (the first commercially successful microcomputer) was taking people’s money for Altair BASIC without actually delivering the software. So a pirated copy began circulating (on paper tape!) for free between hobbyists at the Homebrew Computer Club. One of the club members, Dan Sokol, said that he felt that MITS’s price for BASIC was “excessive and greedy.” It was a similar feeling that led Stallman to form the Free Software Foundation in 1985 in response to MIT’s licensing of the Lisp machine in 1981.

However, by the late 1980s, it was clear that Gates’s philosophy had won out, with everyday computer users ready and willing to buy software from companies such as Microsoft, Novell, and Oracle. Software was big business. By 1995, a mere six months after the financial world was shaken to its foundations by the £827m failure of Barings, the oldest investment bank in the UK, the financial world was ready to bet on software. Netscape’s IPO generated more than US$2 billion in just one day, driving a 5-year bubble that burst in early 2000. By 2002, the Nasdaq technology index dropped back down to where it was before the Netscape IPO.

Apache takes over the web

Netscape made a web server as well as a browser, but it never achieved much market share. In 1995, the most popular web server on the Internet was the National Center for Supercomputing Applications’ HTTPd. But in less than one year, the newly introduced Apache assumed the dominant position. It was free, it fixed numerous bugs and security holes, and was much faster. And it came with a super-simple, permissive, one-page license agreement.

After a few more years, the Apache Software Foundation was created and new projects were started. Among the first were the Tomcat server and the Perl and PHP modules that replaced the inefficient Common Gateway Interface (CGI) for server-side processing.

A few years after that, Apache took on the Struts project, but first let’s go back to Netscape, which was to do something momentous: On January 22, 1998, CNET reported that Netscape Communications would give away its web browser, but also that it would allow free access to the source code of that browser with a license that was substantially more permissive then the GNU GPL. Most significantly, similar to the Apache license, it allowed users of the licensed source code to combine it with other code under a different, even proprietary or commercial, license. Tom Paquin, the founder of Mozilla.org, wrote (with Jim Hamerly and Susan Walton):

“The decision to give away the browser came as no surprise, but the release of the source code stunned the industry. It hit the pages of newspapers around the world, and even the Open Source community was surprised at the move. Never before had a major software company opened up its proprietary code.”

Perkele! Tultiinko tänne juomaan vai puhumaan?!

This is the punchline to a very well-known Finnish joke; “Dammit! I came here to drink, not to talk”.

To understand the Netscape decision, some background is in order. Netscape executives were strongly influenced by a white paper by Frank Hecke, who in turn had been influenced by Eric Raymond’s 1997 essay The Cathedral and the Bazaar, which he had written based upon his experience of helping to develop Linux.

Linux started in mid-1991 as a small but ambitious personal project by Linus Torvalds, then 21. He was creating a new operating system, and he was doing it very publicly, posting on comp.os.minix, a newsgroup on Usenet. The project gained momentum extraordinarily quickly, and by 1993, a company named Red Hat had been formed, shocking everyone with the idea that a profitable commercial company could be run on the basis of publicly developed free software it didn’t “own” (Linux used/uses the GNU GPL license).

By 1998, when Netscape made its announcement, Linux had its own newsgroup—comp.os.linux—which was at version 2 and so accepted and popular that implementations existed not only for Intel CPUs, but also for the high-end workstations: DEC Alpha, Sun SPARC, and the MIPS CPU used by SGI. It had also proven that software written by thousands of anonymous programmers connected only by the Internet could be wildly successful.

That is what made Linux different from GNU, and that was the subject of Raymond’s paper. He likened the creation of Linux as a “bazaar,” as compared to the “cathedral” of GNU (which Raymond had also been involved in) that had been carefully planned by hand-picked “high priests.”

Another key detail that must be mentioned is the notion of security through obscurity, which was enjoying popularity at the time. Linux pretty much settled a debate that had been raging in the world of computer software by proving, once and for all, that open source could be just as secure, if not more secure, as proprietary code, because of what Raymond coined as Linus’ Law.

Open source becomes “official”

Although there was already a thriving open-source community with Apache and Linux, it was linked in the public’s mind to free software, as in Stallman’s Free Software Foundation.

The month after Netscape’s announcement, Raymond applied (unsuccessfully) for a trademark for “Open Source,” created the Open Source Initiative, and registered opensource.org. Raymond wanted to promote this revolutionary new way of creating software, this bazaar, and he wanted a different brand under which to promote it. Open Source Initiative founding member Michael Tiemann said that the intention was to “dump the moralizing and confrontational attitude that had been associated with ‘free software’” and instead promote open-source ideas on “pragmatic, business-case grounds.”

Next, he published Homesteading the Noosphere, which proposed a model for how economics of open-source software can work. It might well be considered the “Open Source Manifesto.”

In 1999, a company called VA Linux decided to launch SourceForge, a website hosting the work of open-source software developers and the first free public repository. SourceForge hosted popular programs such as phpMyAdmin, SugarCRM, and JBoss, and for many years was the most well-known public repository. However, most larger projects, such as Apache or Linux, maintained their own distribution on their own websites.

Such was also the case with two of the oldest open-source frameworks, Struts and Spring. The O’Reilly Struts book was published in 2002, and Spring started to gain in popularity a few years later.

Democratization

Struts and Spring were attracting attention in enterprise IT departments where Java was king, but considering what Raymond tells us in The Cathedral and the Bazaar, it should come as no surprise that open source really exploded once it was democratized.

Git was created in 2005 to support the Linux-distributed development process, with GitHub being created two years later by Chris Wanstrath, P. J. Hyett, Tom Preston-Werner, and Scott Chacon. By 2010, GitHub was hosting more than 1 million repositories, including jQuery, Erlang/OTP, Ruby on Rails, node.js, Redis, and CakePHP. Other popular frameworks were to follow over the years: Node, Express, Bootstrap, Angular, and React for JavaScript, Django and Pandas for Python, and Sinatra and Crepe for Ruby—and these are just a few of them.

But that is not all. There are now more than 100 million repositories on GitHub, and only a few of them are frameworks. Many of them are utility libraries. Need an EPUB Reader? Find one on GitHub. Need a DNS resolver? Get one on GitHub. This resource finally made open source convenient by providing one-stop shopping.

At the same time as GitHub was getting started, the Stack Overflow website was created as a question-and-answer site for professional and enthusiast programmers. Using reputation points and badges, it quickly became a de facto support site for every language, every framework, and every problem a programmer could have. As of 2011, 92% of the questions were answered, within a median time of 11 minutes.

Open-source software on GitHub can often be incomplete, idiosyncratic, or difficult to integrate within an existing code base. Stack Overflow is a crowdsourced support platform. It was the last piece of the puzzle, because democratized support made democratized code-sharing possible.

Putting it all together

To summarize: the movement to share source code and make it freely available has been going for a long time, at least since the early beginnings of the personal computer, probably even before that. However, for the first few decades of the personal-computer revolution, most commercial software was built around source code that was jealously guarded as a company trade secret. Even shareware and freeware was rarely distributed with source code, so this was not an issue of free versus paid. It was a fundamental general consensus that there was more value in keeping one’s source code private than there was in sharing it.

Apache, Linux/Red Hat, and Netscape/Mozilla challenged those fundamental beliefs in a way that was very dramatic for the times and opened people’s minds to a new way of thinking: That software could be created by large, decentralized teams that all had access to publicly visible source code.

Finally, the Internet, did two things:

• It crowdsourced security very effectively, thanks to its ability to reach huge numbers of people.
• It created two major clearing houses with its organically occurring monopolies of information.

These two things massively accelerated the adoption of open source by providing the holy trinity: Convenience, support, and security.

There are many possible futures for open source. Python and JavaScript are leading the way into that future with repository tools such as pip and npm that help standardize the packaging of open-source code. What the future demands is more standardization, which will pave the way to being able to assemble applications out of components in much the same way automobiles are assembled today—something one might call IT industrialization.

This industrialization is inevitable as the demand for software applications continues to outstrip the availability of skilled programmers. What is important to bear in mind is that this does not mean the replacement of skilled programmers. What it means is having standards and tooling that allow unskilled or semi-skilled workers to assemble working applications out of components that were designed and written by skilled programmers. In other words, it provides the opportunity to leverage the work of skilled programmers more intelligently. Open source has laid the foundations for this future.

This article is part of Behind the Code, the media for developers, by developers. Discover more articles and videos by visiting Behind the Code!

Want to contribute? Get published!

Follow us on Twitter to stay tuned!

Illustration by Victoria Roussel