The One Who Created Elixir

Creator of the functional programming language Elixir and cofounder of Plataformatec, a software consultancy specializing in Elixir and Ruby, José Valim discusses how it all started with programming, the lessons he learned while creating Elixir, and his involvement in its future.

Learning how to do things

When I was 8 to 10 years old, I had my first home computer, which was kind of privileged—it was relatively early on. And because my mother was studying data processing, I was able to play a little bit, and I remember getting one of her books and trying to do things in Visual Basic, or something similar. I would try for a couple of days, but I was a clueless kid and didn’t actually know what I was doing.

I remember way back then, I was probably around 14, we had M.U.G.E.N—I don’t think a lot of people remember it. It was a tool, an engine—I don’t even know what it was called—where you could create your own fighting game. So people would design the characters or the stages. You could download the game and you could customize it. It wasn’t necessarily what we would call programming, more tinkering, trying to put things together and seeing how they would work, kind of contributing to the whole idea. And I would have to change some files that were definitely code source files for some languages that I don’t recall.

But programming—programming for real—happened during my first year of university. I had classes in C. This time it was on a blackboard—we didn’t have a computer with us all the time. So the professor would write the programs on the blackboard. When we took our exams, we would write the programs on a piece of paper. And then, during my first year of university, some friends and I formed a band and I decided to make a website for the band in Flash ActionScript, that kind of stuff, during my vacation. And that was the first time I actually did some programming on my own.

Mastering Ruby first

So the first language I became really proficient in, really confident with, was Ruby. What I really liked about Ruby was this ability to explore, because the language is unrestricted and you can try out different things. And it’s very flexible—you can change it in many different ways. That was really what attracted me to Ruby at the time. And when I started with Ruby, it was because Rails was this big sensation. And Rails was leveraging the same kind of features that Ruby has.

The multi-core issue

Going back to my first computer that I got when I was 8 to 10 years old, it was a powerful computer at the time, but two years later, not even that… So I had a Pentium 100, and a boy at school told me he had a Pentium 233. And I was like, “How come? We just got this computer, we’ve barely finished paying for it. How come you have a machine that is more than twice as fast as mine?” It was because computers were getting twice as fast every two years. But now this is no longer true, and what is happening instead is that we have machines with 2, 4, 8, 64 cores. So I started to see interest growing in having machines that run with multiple cores.

And as I read up about it more, it became clear that we were going to have a multi-core future. We would not be running on very powerful machines with just one core, we would be having a bunch of cores and we would need to find a good way to write software that leveraged all those true cores. And working with Rails at the time would sometimes make things faster, and what would happen is that we would improve things, but someone would do a bug report and come back saying, “Hey, we’re running this new version and, under very specific situations, we are finding a bug.” And this bug would be hard to reproduce and hard to fix. And that’s not really the fault of Rails per se, I’m not criticizing Rails. But it is just what naturally occurs when you write software in languages such as Ruby, Java, and C.

Because the thing that changes with multi-cores—and it’s why this has had such a big impact on the industry—is when you have one core, you have just one thing running, but when you have two cores, you can have two things running at the same time. And then you have to address things like, “What happens if those two things running at the same time try to do the same thing or try to change the same place in the memory? Or they try to handle the same resource at the same time?” So we called it race conditions, because those two things are happening at the same time, which may not happen at all but may happen all the time—you just don’t know.

Making the problem disappear

So I found myself in that situation. I was like, if multi-cores become important and it’s hard for me to tackle those problems now, what solutions are there? Before I started working on Elixir, I started looking for solutions—like, “I have this problem, what’s the best way to solve it?” And I like to say that there were two points of no return in this journey of final solution. I felt like, “You know what? Maybe I have to change the way I write software altogether.”

The first time this occurred was when I found functional programming. So, for example, I said, “What happens when two cores running at the same time try to change the same place?” With functional programming we stop thinking about changing things, we think more about transformation. And the first revelation for me was, “Wait, if I were to think about transformation all the time, all that software I wrote in the past that I now know has concurrency bugs would just work.” The whole problem would disappear, which is the best way to solve a problem. You can find a solution to the problem, but if you can make the problem disappear altogether, then, yeah, I’ll take that. So that was the first point.

And the second point was when I found the Erlang virtual machine, the platform that Elixir runs on top of. Because if you look at what has happened in the past 10 years, a lot of the languages are focusing on concurrency. And that’s good, that’s great. They are all focusing on how we have multiple cores. But there is a point when you’re writing some software, when you’re writing a system, where one machine is not enough. If you have a powerful machine with 64 cores, you want to use everything in that machine as efficiently as possible. But previously, a lot of the time you would have to use 3 machines, 4 machines, or have a cluster. So while everyone was thinking about how to solve the concurrency problem when all those things were running on a single machine, Erlang had already solved the problem. And the problem that Erlang focused on is what if we have multiple machines as well? So, to me, Erlang was always ahead of the curve. When I looked at that, I thought, well, it’s great if they’ve resolved this and they’re already working on the next problem—that is where I want to be. At that point, I didn’t know that Elixir would be a thing. But I definitely felt that this was the moment in my career where I was going to change path. I had to try something else because, with the information that I had, I felt like, OK there are other areas to explore and venture into.

Exploring functional programming languages

So I did explore, both when I figured out that I wanted to explore functional programming languages and also when I decided I actually wanted to try writing a programming language. So there was a period when I was really trying to explore a bunch of different things, even languages that were not really that mainstream, sometimes niche languages that focus on a particular idea. For example, there’s this language called Frink, which is all about better ways of working with units of measure. So I tried a bunch of different things, a bunch of different functional programming languages to get ideas from them and see what would work with Elixir. Clojure, for example, was a big help with that—there are features in Elixir that were heavily inspired by features in Clojure. Of course, there were some ideas that came from Haskell—our naming comes from Haskell. And the reason why I didn’t go to those language was… What won me over was using the Erlang virtual machine as a platform. That was what I wanted. I wanted to write software using this platform. It didn’t really matter how, I just wanted this.

Getting good advice

I met the creators of Erlang—Joe, Mike, and Robert—and they were always very welcoming, very nice. Sometimes we would get to talk about technical challenges. Robert, for example, has a bunch of other languages running in the Erlang virtual machine, so sometimes I would ask him questions, sometimes he would be like, “Oh, how did you implement this in Elixir and consider it for other languages?” So whenever we had the opportunity to meet, we would have really great conversations.

And I also met some of the Erlang OTP team, and they were also very welcoming, very open to discussing ideas, which was very nice because you don’t feel like you are working against something. So today, Erlang is maintained by Ericsson—there’s an Erlang team within Ericsson—and I was able to meet different developers from this team from time to time. But I remember that, when Elixir had just come out, there was a bit of a push back—I got comments like, “Why do we need a new programming language? What’s the point?” And one of Ericsson’s developers sent me an email saying something like, “Don’t listen, just continue doing what you’re doing.” And that was really great advice—powerful but also reassuring, like, “If it doesn’t work, it doesn’t work, if it works, it works. Just go ahead, continue what you’re doing and have fun while you’re doing it.”

Defining an overall goal

So one of the things that helped me a lot is that I had a target. The Erlang virtual machine was my target and it has things you can do and things that you can’t. So, often, if I saw a feature in a language that I liked, such as protocols in Clojure, which is something we added to Elixir, I would be like, “OK, how would I implement this in the context of the Erlang virtual machine?”, which is a much smaller hurdle compared with starting a language from scratch. And when you start adding enough things, it’s a bit like Jenga, that game where you have to take pieces away and make sure the tower doesn’t collapse, because when you’re creating a language, things never stay separate when you add them. Everything has an overlap, so when you add something, it has an impact elsewhere. So that’s something that you also need to be aware of when you’re creating a language. And what helps is if you define your overall goal—what you actually want this language to be, and what kind of software you want it to write. And then it helps because whenever you have a question like, “Should this be part of the language or not?” you can always look at the overall goal and check whether it fits within it.

Learning from failure

My first prototype for Elixir—if you want a few adventures, go to GitHub and go way back in time—it didn’t work. It failed because I was trying to run Ruby in the Erlang virtual machine. So I was trying to bring behavior or features that are in Ruby into the Erlang virtual machine. And it didn’t work because the two are incompatible in a bunch of ways. And the important lesson I learned from this mistake is that, instead of saying something like, “Let’s just bring this over,” I would ask, “Why would I want to bring this over? What am I going to gain from this?” And then I would think about the problem in general terms—“Oh, I want to bring this over because it allows me to do this.” But maybe there were other things that would allow me to do that, too, meaning I could say, “OK, now I know what I am looking for, so I can look at different solutions and choose the one I think is best.” So that’s one process that I think I learned from and made me stronger when I was designing the language, and I carry this process with me today still.

Going out and talking to people

If I want a language to be used by other people, I need to go out and talk about it. I could write the greatest language in the world, but if I don’t tell anybody about it, it doesn’t have a purpose, nobody would use it. So I knew that I would have to go out, I would have to attend events, I would have to talk to people about it, I would have to write about it. And I think Rails, for example, has harnessed this idea of marketing really well. They know that they have to talk about it, they have to get people excited, they have to get people involved. So it’s something I knew I had to do, too, and I knew that from my previous experiences in the open-source community as well.

Adding great tooling to ease the learning curve

The best bits of Elixir came from Erlang, and that allowed us to develop as a language, because we have this great foundation. It allowed us to focus on specific things for Elixir itself. So what Elixir has in addition is great tooling. I realised from the beginning that we would also need to focus on that. Sure, I would have to talk to people about it, but I also knew that we needed to have great tooling. I could go and give a talk at a conference, and inspire people to try it out when they got home, but it would be pointless if they then got stuck, or couldn’t get started. So I know the getting-started experience should be simple, your product should be accessible, it should be easy to use, you should be able to go from zero to having something running relatively quickly. So that’s something Elixir focuses on, being very welcoming. And also education in general, or computer science in general, or programming in general, doesn’t focus on functional programming languages, as they are not in the majority, so I knew that people would have to learn a bunch of different things. They would need to learn about functional programming, they would need to learn about concurrency, and so on, so I wanted to ease this learning curve as much as possible.

The importance of not letting yourself get distracted

Every day when I am working, I try to make sure there are as few distractions as possible. So, unless it is important, there is pretty much no way to reach me and interrupt what I’m doing. And also, if you’re a regular reader of news sites such as HackerNews or Reddit—which are great—but if you have these places to go, your 5-minute break regularly becomes a 1½-hour break because you always have this rabbit hole you can disappear down. So I try to make sure I don’t have access to those, so that I can really keep a clear mind and be focused on whatever I need to be doing. And if there’s a topic I need to know more about, I look for papers on the subject, and if I have to read I take a Kindle, or something that has no Internet, nothing that could distract me.

Staying involved in Elixir

I will continue to be as involved as I am right now. Before, there was always the pressure of there being another thing to deal with, and another thing, and then another thing. But this doesn’t happen so much now, so I can have more time off, in a sense, but I prefer to stay as involved as I always have.

This article is part of Behind the Code, the media for developers, by developers. Discover more articles and videos by visiting Behind the Code!

Want to contribute? Get published!

Follow us on Twitter to stay tuned!

Illustration by WTTJ