Under Discussion: Behind JavaScript Standardization

While attending the 2019 edition of dotJS we got together with Daniel Ehrenberg and Sven Sauleau to talk about JavaScript standardization and how decisions are made within Ecma International’s TC39, the committee that evolves JavaScript. Ehrenberg is an engineer at Igalia, a free software cooperative, who used to work on V8, and who is currently a member of TC39. Sauleau is a software engineer working at Cloudflare who contributes to Babel, webpack, WebAssembly, Rust, and JavaScript; he is also involved with TC39 as an invited expert.

Could you tell us how you got involved with JavaScript standardization?

Daniel Ehrenberg: When I was working at Google, I saw an internal transfer and got a job on the V8 team [Google’s open-source high-performance JavaScript and WebAssembly engine], implementing ES6 [version 6 of the ECMAScript programming language], helping with async/await and things like that. I got involved with TC39 through that to try to help fix ES6 and to work on projects like SIMD.js. What I like about standardization is that the goal is really to find these inclusive answers to solve people’s problems. There’s a big deadweight loss due to people having to solve a problem over and over again at higher levels, when it could have been solved in the language, or in the tools, for instance. So I really like the idea of making the changes at those kinds of levels!

Sven Sauleau: I got involved in TC39 about three years ago as an invited expert from Babel, a compiler for writing the next generation of JavaScript.

How does the history of JavaScript have an impact on the current state of the language?

SS: Change proposals need to take backwards compatibility into account. So decisions that were made at the beginning still need to be preserved.

DE: Yeah, there’s just so much JavaScript code out there. It would be a shame to break it by changing what it does! But the biggest thing we get from history besides some early design decisions is just that JavaScript is a very minimal language. Even though we’re adding a bunch of stuff now, it remains much more feature-poor than other programming languages that have facilities for a lot of things that you kind of have to cobble together in JavaScript. And even when you do that, you don’t always get as good a result, so this gives us a surface to work with as language designers to improve the developer experience by filling in the gaps that other programming languages have.

This minimalism is partly related to the culture of TC39 and also to the history of how the standards committee operated with the ES4 effort being sort of canceled. But it also has to do with the history of web browsers—for a long time, there was a very uncompetitive browser market. And because everything that comes in JavaScript itself has to be distributed with the web browsers, if they decide that they don’t want to do things, that’s it, JavaScript doesn’t have those things. I think we’re now in a very good position with browsers, where they’re deeply engaged in the process. They’re brought into the process and into the incrementalism with vetting new features. So we’re at a really interesting point in history.

SS: But now you can also run JavaScript without a browser, thanks to Node.js or Deno, which brings JavaScript to the server.

DE: Actually, we did have that in the beginning, too! Right at the beginning, there were these Netscape and AOL server projects using JavaScript as a server-side language. It kept sort of rolling over and being replaced by other things. It’s more recent that JavaScript became the mainstream server programming language, but it’s always been there in the design of the JavaScript language. I think what’s changed is that, now, TC39 has a more inclusive process, so we actually listen to people who are working on JavaScript in these different contexts.

Could you explain a bit more about the process TC39, the committee that evolves JavaScript, follows for a change proposal?

DE: There’s a 4-stage process. Stage 1 is when something is under discussion. We’ve presented a problem statement and an initial solution that the champion group, the equivalent of a project manager, is working toward.

At stage 2, we are working out details. This is often a good time to do some prototype implementations, and this can be useful for getting feedback from people using it before we lock out all these decisions. Maybe it doesn’t make sense to implement it in a web browser or maybe it does, but only in certain cases. It’s a lot of work to implement in a full web browser and it seems better to iterate on what we call transpilers, these JavaScript-to-JavaScript compilers like Babel or TypeScript. Babel is especially friendly toward these early things being taken upstream and being worked on collectively, and with people having the option to opt into these earlier features or, by default, only the fully standard thing.

At stage 3, we’ve collected a lot of this community feedback and a lot of this experimentation, and we think about it a lot and discuss it a lot on GitHub issues, each one of these proposals having its own GitHub repository. So now we’ve reached a time when we can draw a conclusion on these big design decisions. And from there, there’s more implementation work that happens and more testing work. We have a shared test suite called Test262 that makes sure that the JavaScript engines match up with each other so that the same JavaScript code can be used in all the different engines.

And then, once you’ve carried out multiple implementations and tests, it’s ready for stage 4 and we can say it’s part of the standard.

Each one of these stage advancements is agreed on by the committee.

What is the frequency of the releases by TC39?

DE: In theory it’s once a year, but really it’s a continuous process. On the TC39 website, we maintain the current draft specification, and I think that’s what people should reference, because we only land things that are already at stage 4. I think releases make more sense as a concept for browsers, Node.js, or these other engines. With these standards, we’re trying to coordinate the whole ecosystem, so it’s kind of more amorphous.

What is the latest proposal you’ve worked on with TC39?

SS: My latest one is one we are doing together with Daniel. We want to fix a security issue with bloating JSON modules on the web. For example, if you rely on a web server that sends you JSON files, it could pass all your tests during development, but at some point in production maybe the web server decides to send you JavaScript instead. And the issue is that the web server can effectively execute JavaScript on the client side, forcing clients to execute the JavaScript because there’s nothing preventing it. We are suggesting adding attributes to modules when importing them, such as when you get JavaScript over HTTP, you won’t execute it anymore because it’s not JSON and it doesn’t pass your requirements. I like the tooling aspect—tools can use arbitrary attributes on those modules to attune to the code generation or to do certain things specific to them.

DE: Not everyone views this positively. Some people will feel that it’s really important that tooling doesn’t do this because then JavaScript would mean different things in different contexts. In a sense, though, we’ve already missed the boat on this. We already have these detailed configuration files that are kind of these webpack configs or things like that. They’re kind of inseparable from the code and it’s a mess to have them as separate files, but having them in line in a JavaScript code could be a big benefit and even allow the same sorts of attributes to be used across tooling. But at the same time I understand that you might want to keep the main JavaScript code sort of clean and have higher compatibility requirements. This is something that’s still at stage 1 in TC39, so we’re still at the early stages of talking about it.

How do you make sure that invited experts from a certain company work in the interests of JavaScript and not their company?

SS: As an invited expert, it is important to compartmentalize the commercial interests of the company separately from the interests of advancing JavaScript itself.

DE: In TC39 we use the term “good faith” a lot, which is just the idea that when somebody is talking about something, we should all assume the best intentions and try to move forward. I really do think that everyone in the committee makes a strong personal effort to do this. I feel like the incentives are well aligned so we can only do things if they’re actually for the good of the ecosystem. The consensus process and the TC39 instructor keeps us honest.

Do you think that JavaScript is more concerned with standardization than other programming languages?

SS: Yes, as JavaScript is implemented by many other environments, we need to make sure the specification is clear and works on every platform.

DE: Most programming languages that I know about have, for an initial period of some years, one canonical implementation. So for Python you had the main CPython implementation, and C was under development at AT&T for a long time in their labs. JavaScript wasn’t like that. Netscape developed JavaScript, and very soon Internet Explorer had its own JavaScript implementation. And so right off the bat it got into the question of standardization because they needed to be compatible, because it’s the same web that works the same on both of them.

Now we’re seeing that the number of web browser JavaScript engines has sadly decreased over time because we’ve lost Opera and Edge, as they’re just using V8 now. But there are more and more JavaScript engines that run in different contexts. And as the most commonly used programming language, it’s really important that it works in many different settings and with many different things. So I do think it especially makes sense to have a standards process for JavaScript.

Also, as a language that is frequently sent over a network in source form, as opposed to compiled at a separate stage, standardization is even more important, because you really have to make sure that it’s handled well at that other network endpoint.

SS: Regarding that subject, there is currently a proposal that aims to encode a JavaScript source in a binary format. That sounds interesting to me because not only should the source in text be stable but also its binary equivalent. I’m not aware of any other language that would support two formats.

What were the impacts of server-side JavaScript on standardization?

DE: We had some really serious communication issues with Node.js. There was some concern about their formal participation in TC39, and so, in the development of ES6, the online participation possibilities weren’t as strong and the norms about including the community in discussion weren’t as adhered to. Sometimes there was even the opposite feeling, of like, it’s us against the world—“Oh, they’re telling us not to do stuff, but we’re going to push ahead and make things great for JavaScript programmers.” And the communication problems happened on both sides. It wasn’t anybody’s fault in particular.

But I feel like there’s a lot of movement toward positively working together and talking openly about things. And now with the OpenJS Foundation, we’ve been getting more and more people from Node.js participating in TC39. And it’s been really positive to get lots of people so that we can get the whole spectrum of opinions, such as about the module attributes proposal that Sven was talking about. We had people on the committee claiming opposite things about what Node.js thought about the proposal and that’s the dynamic that we’re just going to have to keep working with.

SS: I’m working on Cloudflare Worker, which is similar to Node.js, but not open source. For a better developer experience, we want to make sure that JavaScript is consistent with other environments and we participate in standards as much as possible to ensure that. I think it’s important that implementers of any kind of JavaScript environment participate in standards to make sure that features align nicely with their platform.

What impact will WebAssembly have on JavaScript?

SS: WebAssembly and JavaScript can live together! We just need to make sure that the WebAssembly JavaScript API is conformed to the rest of JavaScript, so that you can feel it’s the same language everywhere. We need to make sure that WebAssembly plays well in the JavaScript world.

DE: I totally agree. And my understanding is that the adoption of WebAssembly among application developers remains pretty low. WebAssembly remains this pretty specialized thing that you just don’t use most of the time. It’s just difficult to use. The different flows for developing in WebAssembly are pretty difficult for people and there’s more work that could be done in tooling to facilitate this, to make it more easy to transparently use other languages in JavaScript. Unless you’re going to use something like C or Rust, which don’t need garbage collection, the other most popular languages are all garbage collected.

So there are some new additions that we’re thinking about both for WebAssembly and for JavaScript to improve the interaction between JavaScript and other garbage-collected languages that would be implemented in WebAssembly. One of these is WeakRefs, which lets WebAssembly or any JavaScript programs react when something has garbage collected to free up resources. Another is the WebAssembly GC [garbage collector] proposal, which would allow WebAssembly to allocate and manipulate garbage-collected objects without going through JavaScript. You can do this already in JavaScript, but only if the WebAssembly program imports a JavaScript function to create objects and other JavaScript functions to manipulate them. But at that point you’re not even really programming WebAssembly. You’re programming WebAssembly glue code to hook together JavaScript, and it ends up defeating the performance benefits.

I’m really excited about those efforts. My coworkers at Igalia are working on both of those, on tooling as well as incrementally working toward the WebAssembly GC proposal. And I’m really excited about where this will take us, but I’m not worried about it for JavaScript. I don’t see them as competitors. I see them as something complementary.

SS: While WebAssembly is not as easy to use as JavaScript today, a lot of effort is put into extending the interoperability between WebAssembly and the web platform, including JavaScript. Another example of that work is the JS-BigInt-integration that Daniel wrote. We worked on it together.

What are the biggest challenges you’re facing in the future regarding the standardization of JavaScript?

DE: The biggest high-level challenge I see is about unifying the language. JavaScript is used in so many different ways, and it’s going to keep being used in a lot of different ways. And sometimes the differences between the way that JavaScript is used in one place compared with another are inherent to the differences in the problem space. But sometimes they’re just kind of historical or we just haven’t thought through how to have a unified solution. One example of that is between Node.js and web browsers. And another difference is between using tools and just using the JavaScript engine. So we’ve historically had things like different semantics between what the tools did and what JavaScript engines did. And in both these cases there’s lots of really difficult and explicit and incremental effort needed to smooth out these differences. Which requires getting more involvement from the standards people who work on these different sides of JavaScript from the web, from Node.js, from different kinds of tooling and frameworks. If we want to make it so that you program the same JavaScript and it works in these different contexts, we all have to come together and make joint decisions about it.

This interview has been edited for space and clarity.

This article is part of Behind the Code, the media for developers, by developers. Discover more articles and videos by visiting Behind the Code!

Want to contribute? Get published!

Follow us on Twitter to stay tuned!

Illustration by Catherine Pearson