Rust: Built to Last

In 2019, Stack Overflow reported that the Rust programming language was the “most loved” language for the third year in a row. Moreover, it is one of the fastest-growing languages. But why was Rust even created?

It’s true that every generation of programmers is inevitably seduced by the lure of new languages, and why not? Novelty and variety are the spice of life, and so far, the Online Historical Encyclopaedia of Programming Languages has identified 8,945 languages.

But what motivates language designers to create a new language in the first place? Usually it’s a desire to fix something that is broken, to remedy some deficiency in the languages already in existence, or to provide a different approach to writing code. So what deficiencies inspired Graydon Hoare, then a programmer at the nonprofit Mozilla, to start working in 2006 on a new language he called Rust? And how was Rust built to handle those deficiencies?

We looked at its history and turned to Steve Klabnik and Ashley Williams, two current members of the Rust core team, to discover the answers to these questions.

The deficiencies Rust wanted to solve

Concurrency

A hardware limitation had started to materialize shortly before the creation of Rust. Computer manufacturers were facing a physics problem: For every “clock cycle” of a central processing unit (CPU), a few hundred million “logic gates” were generating heat as they were either charged or their charge was released. As CPUs got bigger (more logic gates) and faster (more clock cycles), they reached the point where it was no longer possible to get rid of the heat using conventional, affordable technology (fans and heat sinks). In addition, there were two other factors—gate delay and the speed of electric transmission—although the heat alone was enough of a problem to prevent CPU clock speeds from going beyond 3.7 Mhz. The fastest CPU ever produced in a lab was the AMD Bulldozer, which reached 8.4 GHz and had to be cooled using liquid nitrogen.

When CPU speeds hit the wall, computer manufacturers lost no time in coming up with a new approach to making computers run at faster speeds. The answer was multi-processors. In the 1990s, only supercomputers were multi-processor, and programming them using concurrency was considered a black art. But by 2005, microprocessor manufacturers had developed techniques that exploited some of the speed advantage of multi-core CPUs, even if the code was not specifically written to be concurrent. However, to get the beat performance out of a computer, the code had to be concurrent. Only a few languages made this possible, and none of them made it easy.

Memory management

The other deficiency that motivated the creation of Rust was something a bit more difficult to explain: memory management. The dynamic scripting languages that had become so popular in the previous few decades, such as Python, JavaScript, and Ruby, all shared a mostly well-deserved reputation for performance problems. This was commonly attributed to them being interpreted rather than compiled. There are several arguments to be made about why this is a misconception, but one of the key issues was the impact of memory access patterns, and the way in which these languages tend to create “a scattered ‘mosaic’ of little objects scattered all over the place,” which they try to clean up using garbage collectors.

This has a few negative side effects. For one thing, processing is slowed down if the data the CPU requires is not “all in the same place.” Another thing is that every computer has a limited amount of “live” memory or RAM. So when a program uses memory to store data, it has to free this up as quickly as possible to make room for more, or else the program will run out of memory and crash. This is what is referred to as a “memory leak.” In older languages, such as C, programmers manually managed memory use. The scripting languages relied upon the garbage collector’s sub-routines to “figure out” if a bit of memory was still in use or not. Garbage collectors eat up CPU cycles when they run and they are not always reliable.

How Rust was built

An evolution, not a revolution

“Most of the features in Rust are rooted in ideas that are very old,” warns Klabnik, who is also a prolific open-source contributor. “While a lot of people think of Rust as some sort of new, advanced language, Rust is not trying to be a cutting-edge programming language theory [PLT] research language.”

It is pretty clear from his own statements that Hoare was seeking to improve the way things are done, not completely change them. There is indeed no new abstraction model proposed by the language, and RAII memory management can be considered a bit old-fashioned.

Hoare also avoided any kind of dramatically new syntax, which “was an affirmative choice, as at the time, the audience of a systems programming language was mainly systems programmers who are used to curly braces and semicolons,” says Klabnik.

A selection of the best features

Hoare sought to select the best features from other programming languages. According to Klabnik, “The original version of the language took heavy inspiration from Erlang and Newsqueak/Alef/Limbo, with green threads and channels for concurrency, for instance. It really took ‘let it crash’ to heart. [It] was even intended, [but] I’m not sure if it was ever implemented, [for there to be] a task-local garbage collector!”

So Rust was originally designed so that functions had a garbage collector running within, and dedicated to, the thread that was spawned to execute a function, inspired by the Erlang (and IBM TPF) model of functional programming, where every function/instance runs in its own process.

Another aspect of this style of functions running in separate processes is that there is no shared mutable state. In other words, no global variables and all states must be explicitly passed from one function to another. “The trait system is based on ‘type classes’ that date from 1988. The borrow checker is based on a concept called ‘linear types,’ which was introduced in 1990,” says Klabnik.

Linear types are a concept that Philip Wadler introduced in 1990 in his paper “Linear types can change the world!” (a typical hacker pun). One of Haskell’s designers and a specialist in type theory, Wadler built upon the linear logic of the contemporary logician J-Y Girard. In the late 1980s, Girard developed proofs that offered a means of formalizing models of parallelism and communication. Wadler saw in Girard’s work the possibility of overcoming some of the objections raised against Haskell and other functional programming languages, and how to approach the question of how a functional language can manage state change without sacrificing rigor.

A whole project

The bigger picture of Rust’s raison d’être is the need for systems developers to consider what happens to the code once it leaves the development environment. The maintainability of a language is influenced by far more than syntax, it is profoundly affected by things such as documentation, package management, deployment concerns, and curation of standard libraries.

For example, one of the more dismal facts of everyday life with Java programs is playing hide-and-go-seek with libraries that are standard in one release and and then optional in the next. It is a banal example, but the number of billable hours incurred by large enterprises because of this kind of thing is beyond measure.

“A lot of languages focus on, well, the language, and accomplishing certain goals with it. That’s good, of course, but Rust tends to view itself as a whole project,” explains Klabnik. “Rust cares about documentation, about package management, about the installation process, the upgrade process. A language feature might be great, but if it’s too new, or too hard to use, it might not be a good fit for Rust. But it just depends. Async/await is a relatively complicated feature at the language level, but is really easy to use from a user’s perspective. This makes it easier to add.”

So all of this underscores that Rust is really designed to properly support systems programming (used in the sense that Fred Brooks used it in his seminal book The Mythical Man-Month).

Making systems programming safe for developers

“One of the biggest goals of Rust is to make systems programming safe, but this goes further than first glance,” says Williams, who also leads the community team at Rust and was part of the Node.js and npm leadership. “Making systems programming safer makes more developers feel empowered to write systems programs. Getting more people involved in systems programming means a greater set of backgrounds and experiences can understand and produce systems programs. As it currently stands, the number of people who can do this today is very low. We want to see that grow. Rust views ‘safety guarantees’ as an accessibility feature, not just a technical one.”

One might compare this to Poka-yoke, a concept from the Toyota Production System that promotes the kind of mistake-proofing that you see with electrical connectors that are designed to make it impossible to plug devices into the wrong current, or to reverse the polarity. “We want Rust to be a language you can come to from any background, be it a scripting language like Ruby/JavaScript or a systems language like C++, or academia, or even as a first-time programmer,” says Williams. “Rust’s tooling, [such as its] first-class package manager cargo, version manager rustup, and education and inclusion programs RustBridge and Increasing Rust’s Reach, all express this value.”

Package management needs to evolve so as to standardize or automate integration concerns as much as possible. We are still a long way away from the kind of “Mass Produced Software Components” that Doug McIlroy (creator of Unix pipes) argued for back in 1968, but some argue that IT Industrialization is inevitable. Williams says: “Even at just the application level, there are myriad levels of abstraction. Asking a developer, or even a team of developers, to care about all of them at once is a poor use of resources and likely to be unsuccessful. Efficient package-management tooling and a vibrant, plentiful package ecosystem enable and empower teams to focus on building the thing they want to build. They can use their efforts to create new value, not reinvent someone else’s wheel.”

Conclusion

The growing popularity of Rust creates an opportunity for programmers to experience a mix of some of the greatest programming-language paradigms of past languages. It is very much worth digging into the background of the various features this article touches upon (RAII, linear types, Erlang’s “let it crash” approach), because there is a difference between learning a language and really understanding it.

The Rust core team is seeking input for the evolution of the language in 2020. They are asking “anyone and everyone” to write a blog post about how they’d like Rust to develop, and the core team will read all of them. They want to hear the community’s ideas for language features, tooling needs, community programs, ecosystem needs, or anything else you can think of. Be part of the evolution!

This article is part of Behind the Code, the media for developers, by developers. Discover more articles and videos by visiting Behind the Code!

Want to contribute? Get published!

Follow us on Twitter to stay tuned!

Illustration by Blok