How Python Became the Popular Choice

According to a StackOverflow survey carried out in 2019, Python outstripped Java to become the second most-loved programming language by developers, and was ranked fourth after JavaScript, HTML, and SQL as the most-used programming language. During the 30 years since it was launched, Python has managed to evolve while staying true to its core principles, illustrated in the Zen of Python. With its popularity with programmers still growing, it seemed a good time to examine its history to understand how it became one of the most impactful languages in the world.

Strong roots

Python was originally developed by Guido van Rossum in the late 1980s to follow on from, and improve on, ABC. With van Rossum leading the project until July 2018 as the benevolent dictator for life, as Python matured, it stayed true to its central tenets, as outlined below:

• Readability:  Python’s simple syntax and grammar, which is close to natural language, increased programmers’ productivity and brought the fun back into programming.
• Extensibility:  Python is designed to have a small core language with a large standard library and an easily extensible interpreter. This allows programmers to add programmable interfaces to existing applications.
• Maintainability:  Python’s approach of “there should be one — and preferably only one—obvious way to do it” allows coding standards, styles, and recommendations to be enforced. At the same time, it increases the readability and maintainability of projects.

Many programmers credit Python’s popularity to its strong adherence to these 3 philosophies. For Thomas Hatch, the creator of Salt open-source project and CTO of SaltStack, extensibility in particular has been key to its success. “Python is great for many things, but the best part is its ability to play nice with many languages,” he says. “This is why it’s successful. You can use Python in conjunction with other languages that might have strengths in other areas. This makes for full-stack programming solutions for a variety of programming problems.”

Early development

In January 1994, version 1.0 was released, strongly influenced by Lisp, and in which functional programming tools such as lambda, reduce(), filter(), and map() were introduced. Five years later, van Rossum launched the CP4E initiative—Computer Programming for Everybody—with the aim to make programming more accessible and encourage basic knowledge of coding; thanks to its clean syntax, Python was able to play a principal role in this.

During this period, Perl was the popular scripting language. Python was still not ready to be used by many and, while many of its maturing loadable modules mirrored Perl’s most popular features, many developers overlooked the former. However, those developers who sought alternatives to Perl felt they found one in Python.

In the object-oriented programming world, meanwhile, this was the era of Visual Basic and C/C++, with Java on the rise. Indeed, by the late 1990s, supported by Sun Microsystems, Java was hailed as a replacement for C/C++ and was destined to dominate.

A period of maturing

Python 2.0 was released in 2000, not long after the core Python development team moved to BeOpen.com to form the PythonLabs team. With this release, list comprehension was introduced, while many of Haskell’s features had been adopted for functional programming. At the end of the following year, Python 2.2 brought unification of the language’s types and classes under one hierarchy, allowing Python’s object model to be purely and consistently object-oriented.

Even though Python was an object-oriented language, it never competed with Java or other company-sponsored languages. Instead, its developers focused their efforts on developing the language and its community.

At this time, many programmers were interested in using Python as an alternate scripting language, or prototype language for their projects. With the proliferation of Java in the enterprise application development world, Python found niches in applications that managed data and infrastructure. And thanks to it being interoperable with other languages, even Java programmers started to use it for parts of their applications to increase their own productivity.

In 2007, Dropbox launched, and within a year, it had received funding, quickly becoming one of the biggest projects written entirely in Python. By this time, many start-ups were using Python for various parts of their application development, but it had been slow to catch on as the language of choice for large-scale enterprises. The success of the Dropbox project changed that, propelling many technology managers in that world to consider Python more seriously.

The following year, Python 3.0 was released. Arguably one of the most important releases of Python to date and the one that pushed it into the limelight, it fixed some fundamental flaws in the language. It also retained backwards compatibility with the 2.x series, while its guiding principle was to “reduce feature duplication by removing old ways of doing things”.

Python was also beginning to garner attention from the scientific community, particularly in neuroscience research, where it started to be used extensively. With the development of the library NumPy, Python started to be seen as an alternative to R, Matlab, and other scientific computing languages for research projects.

The rise of big data, machine learning, and data science

In the middle of 2000s, there had been an exponential increase in the amount of data that companies stored, tracked, and managed. In addition, from 2005 onwards, social media companies such as Facebook, Instagram, YouTube, and Twitter had also become popular. By 2019, 74% of adults in the US were using at least one social-media platform. Marketers were ecstatic at the large amounts of data suddenly available from social media, and start-ups were beginning to crop up everywhere to take advantage of it all, resulting in many B2B and B2C applications being developed in the start-up world during this time.

Due to the ease of use of Python, these young companies found it to be the most cost-effective solution for early development, leading it to become the language of choice for managing social-media data. Compared to languages such as Java, Python increased programmers’ productivity and, as mentioned, injected the fun back into programming, saving time as well as money.

Following the 2008 financial crisis, a tidal wave of automation took place in the financial world. With increased regulatory scrutiny in place, large financial institutions needed to find ways to better manage their trading, risk management, operations, and sales workflows to alleviate systematic risk. Economic recovery was on the horizon and hedge funds led the way in using machine learning and data science to find trading opportunities. Soon, these institutions opted to replace components of “human” intervention in their workflows with AI equivalents, resulting in cleaner ways to manage the massive amounts of data that the financial world generates daily.

With data rapidly multiplying in both the financial and start-up worlds, machine learning and data science soon became the new “hot” career for programmers. Suddenly, there was an increased need for technologists to program algorithms such as neural networks and reinforcement learning algorithms. At the beginning of 2019, after analyzing its user data, the job-search website Glassdoor showed that data scientist ranked as the most promising job in the US.

Along with this shift in demand from software programming to data science came the need for the consolidation of tools, and the efforts started in research and finance became championed by large technology companies such as Google and Facebook, including the below 3 Python libraries:

• Pandas: a project started by Wes McKinney in 2008 to perform quantitative analysis with financial data.
• Scikit-learn: started in 2007 by David Cournapeau, it enables programmers to use machine learning and data science algorithms.
• Matplotlib: started in 2008 by John D. Hunter to port a lot of the graphics functionalities in MATLAB into Python.

Between 2010 and 2015, Python was competing with R as the language of choice for machine learning and data science, with a lot more statisticians using R for analysis and visualization. But the Python ecosystem grew quickly, partially out of necessity. As a programming language, it was a lot more robust than R and the only choice when it came to productionizing machine learning code into full-scale systems.

Large technology companies such as Facebook, Google, and Netflix soon became the pioneers in the field of machine learning and data science. Tools, coding conventions, best practices, algorithms, and scaled-up solutions were developed by these companies using Python, including the below examples:

• TensorFlow: a project started by Google in 2015 to enable better data visualization in the data science and machine learning workflow.
• Seaborn: started by Michael Waskom in 2012 to improve tools for data visualization in the scientific computing community.

With Python now the language of choice in machine learning and data science, programmers started to work closely with data scientists, machine learning engineers, and researchers to develop more sophisticated AI applications. “We’ve had machine engineers and researchers come in and get going from day one,” says AI researcher, software engineer, and founder of productivity app ObjektivApp Andy O’Harney. “When they are done with their work, it’s very easy to just pass along the code to software engineers and say, ‘OK, now productionize this.’ I think this is where the biggest advantage of using Python is.”

Rapid proliferation

With such swift adaptation of Python, there were many questions in its community regarding implementations such as conventions, standards, and best practices. In some ways, van Rossum’s original philosophy of “there should be one — and preferably only one—obvious way to do it” was very useful during this time. As developers “experimented” with Python, those more experienced were quick to enforce Python coding styles and conventions, meaning large code bases involving hundreds of developers all over the world could be maintained.

Books such as Fluent Python (2015), Effective Python (2015), and Python Tricks (2017) helped to guide programmers to code more effectively, while blogs such as Python Software Foundation, Planet Python, and Real Python played an important part in educating the development community. By 2017, Python questions were dominating StackOverflow.

On the web development front, frameworks such as Django and Flask solidified Python’s place as the language of choice for developing the backend and middle tier of web applications. As David Foster, author of Generative Deep Learning, explains: “For our website Applied Data Science, we use Django for our backend. That has worked out great.”

Python as a beginner programming language

From 2014, it had become noticeable that Python was being used as the primary programming language for students attending universities, high schools, and coding camps worldwide. Anticipating a rapid increase of AI-enabled application development, everyone was learning to code.

High schools and middle schools were developing programs to teach children to code using Python. At the same time, there was an increased interest for math teachers to use Python’s visualization tools in the classroom, including Peter Farrell, the author of Math Adventures with Python. “When you are graphing and visualizing data, you are thinking about concepts such as boundaries, domains, and ranges, which essentially allows you to think about limits,” he explains. “Limits in math is a powerful concept that will set the foundation for learning calculus and other more difficult math concepts later on. Visualization also provides ‘proof’ in the child’s mind of how things really work. In the child’s mind, they can rotate visualizations to look at them from different angles. This unleashes creativity in their thinking.”

In a survey compiled the same year by Philip Guo, 8 out of the top 10 computer science departments at universities in the US were using Python to introduce students to coding.

The trend can also be seen in the StackOverflow statistics for seasonal traffic from colleges and universities. Within a few years of the move toward Python starting, it had overtaken Java, gaining the largest percentage of visitors.

Version 3.5 and beyond

In 2015, Python 3.5 was released. Then, in a sequence of releases from 2017 to 2019, the Python development community addressed a number of ongoing issues with the language, solidifying its place as one of the dominant programming languages across many industries.

During this time, many companies were embarking on scaling up their machine learning and data science applications. There were concerns about memory usage, concurrent processing, performance, code distribution, and so on. Solutions included Python 3.8 introducing asyncio for concurrent processing; process-based parallelism becoming possible with multiprocessing; and assignment expression becoming easier to manage with the Walrus operator.

Although PyPy remains the just-in-time compiler for certain Python code, if you want faster implementation, there’s now Numba, another open-source just-in-time compiler that allows you to translate a subset of Python and NumPy code to fast machine code using the industry-standard LLVM. And with PyInstaller, Python applications can be deployed as stand-alone executables.

By addressing these concerns as a community, with releases, documentation, and community-based tools, Python has gained more trust among developers.

The future of Python

In 2019 Python was the fastest-growing programming language. Its rapid ascent has been mainly due to its strict adherence to its philosophy: to be a dynamic, multi-paradigm, extensible, open-source language that is simple in syntax and programmed to be maintainable. It is no wonder that Python has one of the largest programming communities worldwide.

Its future is bright. Capitalizing on the exponential increase of innovations in our coming age of AI, machine learning, and data science, Python is dynamic and general purpose enough to help when used alongside other programming languages. Its continued success will depend largely on adhering to the same principles that have contributed to its success so far.

Embracing the programming community’s views on the future direction of Python and potential improvements is just one way Python will evolve. Says Salt’s Hatch: “I think what would push Python forward in a huge way is, first, a better built-in way to solve the single binary problem or continue to evolve on PyInstaller. And second, If we could take what the Julia developers have done from the LLVM perspective, and make it work in Python.”

Conclusion

Over the next years, Python will be used as a primary language alongside Java for enterprise application development in many industries. In the coming age of AI-enabled application development worldwide, Python’s place as a general purpose language will not change. However, the definition of the term “general purpose language” will evolve as our programming needs change. Understanding the evolution of Python to date, and sticking to its founding principles, will become increasingly important as the language develops.

This article is part of Behind the Code, the media for developers, by developers. Discover more articles and videos by visiting Behind the Code!

Want to contribute? Get published!

Follow us on Twitter to stay tuned!

Illustration by Victoria Roussel