The decision process was actually pretty structured. The first decision we made was: we're not going to rewrite, we're going to port — because only by porting can we preserve the semantics and the algorithms and the exact behavior of our existing compiler, which everyone depends on for backwards compatibility. We could have cleaned the slate and started completely from scratch, but we would have come up with a different language, in the sense that it would give you different errors or behave differently in situations where it has to make choices between multiple possibilities. Porting also meant our code could keep certain assumptions — for example our code assumes the existence of garbage collection, and first-class treatment of functions: you can have functions within functions that close over state. As we evaluated all of these, Go was the one that checked the most boxes. It gives us very robust and mature native code generation on all major platforms, it has garbage collection, and it has excellent access to shared-memory concurrency. Those were the high-order bits we wanted to check, and every other language had something that kind of worked against those objectives. So for this particular workload, Go was the right choice for us, and it's worked out well.

I think there are two things. It doesn't have garbage collection — it's done manually, or through the borrow checker, and the borrow checker doesn't allow circular data structures. Our compiler is chalk full of circular data structures: we have trees with parent pointers, types that are recursive, symbols that refer to each other — it's everywhere. If we had chosen to rewrite, we could probably have solved all of that, but it wouldn't have been just porting the code; it would also have been solving a whole bunch of new problems we had bought ourselves by going that route. We tried, and it just wasn't feasible. We were just trying to optimize for doing as little work as possible to get us to the payout. And honestly, Rust is no better when it comes to the quality of the generated code or the concurrency gains we could have gotten out of it — Go is just as great. We got all the benefits for less work by going that route.

For certain classes of apps, they're writing 100% of the code, and there's an awful lot of that being written. Are they going to write 90% of the high quality code on the internet? I don't know about that. Ninety percent of the volume of written code is just going up like that right now, because AI writes so much code — so in a sense that's a self-fulfilling prophecy. But when it comes to the super high quality code that no one has ever written before, it's not clear to me that AI is going to write 90% of that.

There are so many things that in their time were challenging. On this last project, the easy part was getting the native code — just move it to another language. The hard part was taming this beast called shared-memory concurrency, and getting the compiler to use as many CPUs as there are available on your box. That is not a simple problem to solve in a compiler, because we're all taught to write sequential code: first you do this, then you do this. How do you do all of this at the same time, and then share the data structures afterwards, without anyone accidentally modifying the other guy's data structure? And in the cases where you have to do that, how do you synchronize it and avoid deadlocks and race conditions? It was tricky — there were some good technical problems that we licked that I think we can be proud of.

I usually recommend the same one. The book that was the "it" book for me when I was younger is Algorithms Plus Data Structures Equals Programs by Niklaus Wirth, the inventor of Pascal and later Modula and Oberon. It's a book that is light on math and symbolism and rich in instructive examples and good explanations about how data structures work. That's how I learned about hash tables. When you're self-taught, you don't know about hash tables — you sort of know about linked lists. In the original version of Turbo Pascal, all the symbol tables were just linked lists, and as you had many local variables you were going exponential in search time. Then I read about these hash tables — wow, you mean I can do this and it's basically linear lookup time? I implemented it and the compiler got twice as fast. Holy cow, that was useful reading. He also explains how to do error recovery when you construct a simple compiler. It was a great book for me — and now it's just a big PDF you can download out there; it's been out of print for 30 or 40 years.

A couple of things, maybe. Like we talked about, being a team player — I would probably have tried to educate myself a bit more about that to begin with. And one thing in retrospect: don't let people tell you that it can't be done. When they tell you it can't be done, it's because they can't do it. That doesn't mean that you couldn't possibly do it. So don't let that discourage you.