Why should programmers treat programming like a craft? In this episode, Max Brunsfeld, co-founder of Zed, a collaborative code editor written in Rust, joins Beyang Liu, co-founder and CTO of Sourcegraph, to share the apprenticeship-like pair-programming experience that taught him to appreciate programming, explain how he learned the fundamentals of parsing on the weekends and tell the story of presenting an application he couldn’t explain to Paul Graham at Y Combinator. Along the way, Max describes how the Zed team passes off in-progress branches to teammates in other countries and keeps development moving across time zones.
Show notes & transcript: https://about.sourcegraph.com/podcast/max-brunsfeld/
Sourcegraph: https://about.sourcegraph.com
Why should programmers treat programming like a craft? In this episode, Max Brunsfeld, co-founder of Zed, a collaborative code editor written in Rust, joins Beyang Liu, co-founder and CTO of Sourcegraph, to share the apprenticeship-like pair-programming experience that taught him to appreciate programming, explain how he learned the fundamentals of parsing on the weekends and tell the story of presenting an application he couldn’t explain to Paul Graham at Y Combinator. Along the way, Max describes how the Zed team passes off in-progress branches to teammates in other countries and keeps development moving across time zones.
Show notes & transcript: https://about.sourcegraph.com/podcast/max-brunsfeld/
Sourcegraph: https://about.sourcegraph.com
Beyang Liu:
All right. Welcome back everyone to another edition of the Sourcegraph podcast. Today, I’m talking with Max Brunsfeld. Max was one of the core contributors to Atom, the hackable web editor that pioneered the use of Electron for desktop applications. He is also the creator of Tree-sitter, the modern parser generator tool that powers syntax highlighting in an increasing number of editors and is also used for code navigation on GitHub. He’s now working on something new, a project and company called Zed, which is trying to bring real-time collaboration into the editor. He’s taking lessons learned over many years working on editors and developer tools. Max, welcome to the show.
Max Brunsfeld:
Thanks, Beyang, I appreciate you inviting me on.
Beyang Liu:
Awesome. So we like to kick things off around here by asking people, how did they get into this world of computers? So Max, how did you get into programming?
Max Brunsfeld:
Yeah, it took me a long time. Growing up I wanted to be an artist of some sort. My father is an artist and both my parents are really, and I thought I wanted to be some kind of realistic illustrator or jazz musician. And we were all kind of in the technical sphere of the arts, but I didn’t really think of it that way at the time. When I went to college, I started out studying art and music but I did know that I hadn’t yet found the right path. I hadn’t found something that I was really good at, and that was going to be a reliable way for me to take care of myself.
I was lucky that I went to a small college where most of the majors weren’t super competitive to get into, and it wasn’t super elite or very expensive. So I was able to just keep changing my major until I figured out what I wanted to do. So, every semester or two, I kept on changing what I was studying. And eventually I ended up taking a calculus course and it was when I took that class that I kind of realized I had always identified as an artist and not really thought of myself as a super scientific person. But when I got into that class… It was the first time I had learned a new technical set of ideas as an adult and there were these three things, the derivative and infinite series on the integral and it all made so much sense. And I just remember thinking like, “Wow, this is what scientists do all day? They build these theories that make sense.” It felt incredible. And I was like, “I get this. I’m actually good at this.”
I couldn’t believe that I didn’t realize before that that engineering and science were so cool. And that was a couple of years into college. So I took, for a couple of years, a combination of classes in environmental engineering and physics, and I eventually got my degree in physics. So, by the end of college, I had done at least some programming because in engineering school, we had these numerical analysis classes where it was very old school.
The professor would write the Fortran program on the blackboard and prove that it was solving a math problem correctly. And we would email the computer programs that we wrote as our homework to our professor and he would print them out and in red pen, underline the bugs.
And it was so different from how I think of programming now with all the tools and the methods. And I did realize in college that I liked that, but when going through all that, I didn’t think “Oh, I’m going to be a programmer.” It was just something that felt easy–unlike doing lab work or having to procure materials to build something. You could just do it and it was limitless what you could build. And I really appreciated that about it, but I still didn’t say to myself, “Oh, I’m going to be a programmer.”
And what finally took me to get there was really meeting the right person, a friend who was passionate about programming and who could talk about it in a way that made sense to me. Luckily, my girlfriend at the time, who I’m now married to, her brother is a software engineer–Nathan Sobo. We hung out during my senior year of college and he had already been working as a software engineer for a few years and he had quit his job to work on a side project that was a passion of his.
I won’t get too deep into what it is, but it had this mathematical component to it because it had to do with voting. There were these election algorithms that were involved in this product that he was building. He had this idea for exploring these new ways of collective decision-making through online voting. And he was taking time off to explore this idea and when we were hanging out, he said to me, “Oh, Max, you do physics. You’re into reading mathematical stuff. Maybe you would be interested in researching this literature with me. This literature is about election theory and different algorithms for computing election results. We could code together.”
And it was so funny because at the time I was figuring out what I was going to do after college and was thinking about applying to material science grad school and things like that. And so I almost had this reaction: “Oh no, I don’t have time to do this. I need to figure out what I want to do with my life.” And luckily, I stopped and was like, “Oh, well, maybe this will help me figure out what I want to do.” He shared a GitHub repository with me–it was a new thing to me at the time. This was 2011, I think. I was still in college.
So I began to read about these election algorithms. He was developing an application that implemented these algorithms in Ruby and so I was looking up how to program in Ruby and how to adapt these algorithms into Ruby code. Eventually, we talked more and he said, “Oh, wow. You’re really engaging with this. Well, maybe you and I should code together and I should come and visit you.”
I lived in Humboldt, California at the time and he lived in San Francisco, so he came and visited me. I remember we pair-programmed. That was something that he was really into at the time and I had never heard of it, but he set it all up. We had two displays and two keyboards, and all this equipment. We sat down and we coded and we ended up spending all this time pair-programming together.
For me, my experience programming had been in the engineering department at school using very primitive tools. I was using vi–the engineering lab teachers taught us to use vi, not Vim, but vi in the terminal. And so Nathan was like, “We are coding Ruby.” He had this Rails app, there was JavaScript, there was long polling, there was like Postgres–all this stuff. I was sitting next to him and taking it all in through this pair-programming process. I remember I was learning so much with all these technologies and tools for programming that I hadn’t been exposed to. We would take turns typing, mostly he would type, and I would be taking it all in because there was so much, but I would type.
And I remember one thing that he would do. We’d decide to make some kind of change and I would start typing and he’d be like, “Wait a second. No, no, no, undo all that you just typed. I want to show you that you can use the tool to make that happen. Hit Alt+F6 and we’ll rename this function. And then we’ll extract this part into another function also using some other command.”
We were using RubyMine at the time, the Ruby IDE by JetBrains. And there were all these moves that he taught me to do. He taught me not just the programming knowledge itself, but the tactics of how you physically get the code into the files.
And something about that at the time, I was learning all this kind of stuff about building applications, but then also learning that there were these amazing tools that you use as a programmer. I learned it was this craft, almost like, I don’t know, a trade, where you have to learn this rich set of tools. It was almost like an apprenticeship experience for me–like being a carpenter or something but for code. The experience of learning to code effectively with these tools–that was amazing to me. I came to think of programming in a whole new light. It was this skill that you could hone, not only in your understanding of software, but in your ability to create software effectively and in efficient ways that aren’t error-prone.
And that was mind blowing to me. That was the point where I was like, “Oh, okay, I’m going to be a programmer or a software engineer or whatever you call it.” And so that was during my last year of college and that was when I decided, “I’m not going to apply to engineering grad schools. I’m going to do this.” And then the funny thing is that it’s hard to explain what this actual application was that we were building on. Like I said, it had to do with voting and democracy.
Beyang Liu:
Was it for a voting system that people were going to use to vote or was it more of like a research project?
Max Brunsfeld:
It was a research-y project I would say. It was like an app to explore these ideas. But earlier in the year, Nathan had said, “I want you to apply to this thing with me in case we decide to keep working together. It’s this thing called Y Combinator and I’m going to see if they want to fund me to do this.” And at the time I was like, “Okay, I don’t know what Y Combinator is, but I’ll do this application with you.” And then we had all this time coding together that I described and then we got asked to come speak to them at Y Combinator for this idea that I have no idea how to explain how it would ever be a business–but Y Combinator does that. They mold your idea into a business. So it was right after college, I was still feeling this total shock of going from an engineering school to startup coding.
And we go to Y Combinator and we wait with all the other teams. And we got invited into this room and talked to Paul about it. Once we got in the room, it was kind of a shit show trying to explain how this thing was going to be a business. They weren’t really loving our explanation.
Beyang Liu:
That’s really like a short meeting too. It’s like 10 or 15 minutes.
Max Brunsfeld:
I remember it being very short. I think when I knew it was going badly was when they started asking, “Oh wow, that’s working in real time. How’d you make that happen?” At that point it was clear that there was not much business potential there or it was at least going to be a difficult road to try to turn that into a business. But the tech side of it was really interesting to them. So we got sidetracked off of that and then we pretty soon got a letter that said, “Hi guys. Really cool tech that you’re building, but we don’t see how we’re going to make that into a business.” But that was just a really early, interesting, trippy experience for me coming out of college.
Beyang Liu:
That’s awesome. So, where did it go from there? What was the path from that to working at GitHub?
Max Brunsfeld:
After college, I didn’t know what I was going to do. All I knew was that I wanted to code with Nathan on his project for a while and he had money. He was taking time off. So he was like, “I’ll give you some money so you can eat. You can come code with me.” I moved back into my room at my parents’ house. He was borrowing a desk from his old employer at Pivotal Labs. The founder of Pivotal Labs was kind enough to let Nathan and I have a desk there and work on his thing. So I was coding with Nathan all summer and then at a certain point, decided to call it–like, okay, we need to go get jobs now. So I applied to Pivotal and interviewed there and started working there after this.
The cool thing about working at Pivotal was that Pivotal was like a consulting shop. They had this very specific service that they provided, which was for if you wanted software built quickly and you wanted them to teach your own engineers how to build the software. They had this integrated pair-programming-based practice for making that happen. And we would handle hiring for people. It was sort of what the selling point was–Pivotal knows how to hire effectively and bring people on. And so it was this complete service for kicking off a project and then staffing up the team on it. So it was very focused on pair-programming all the time and rotating new people onto projects constantly.
So, after having had this summer where I had this crazy crash course in all the practices of programming and how to do it effectively, it was an opportunity to go mix with all these people in a really rapid way and learn and immediately start teaching other people the skills that I then knew about how to code effectively. And I remember a lot of the projects at Pivotal had a common approach. We were doing a lot of Rails apps with certain kinds of JavaScript that we would do. We’d do iPhone apps in Objective-C and so we would use the IDEs for those languages like I had learned with the JetBrains suite of IDEs.
And so there were all these cool things you could do to code effectively. All these Pivots (Editor’s note: “Pivots” are employees of Pivotal) knew all these tricks to get code fast and make changes fast and I started picking those up. But then there was a certain class of projects at Pivotal where the IDEs… There were codebases that were too large to be opened in IDEs or codebases in different languages. And so there were other Pivots who had all this deep expertise in VIM and using all these lower level tools for coding.
Beyang Liu:
Interesting. So they would use Vim on the larger codebases, because you would try to open it up in the IDE and it would slow to a crawl or something like that.
Max Brunsfeld:
Yeah. At that time, RubyMine couldn’t open–I don’t know what would happen–some of these client codebases in the IDEs. There’s this whole other parallel knowledge base at Pivotal of people who knew how to do crazy Vim tricks.
Beyang Liu:
Like the dark art of Vim.
Max Brunsfeld:
Right. And so I learned all of that as well. That was cool for me because I was interested in learning other programming languages too. I had learned a few on the job, but at the time I was interested in Clojure and Haskell and learning functional languages and there weren’t IDEs for those. So I was really happy to be learning how to use tools like Vim. But while learning those, I remember it was so much work to customize them and make them usable. Adding modern features to them was difficult and so was learning all the commands that you needed to know to make changes inside brackets and using all these commands to edit pieces of text.
It’s funny, Nathan and I had had all these conversations while working on this side project. Something else that Nathan was really interested in was like, he was like, “One day I want to build my own code editor that’s going to be better than all these code editors. And one thing about it, it’s going to be like the IDEs with all these syntax aware commands, but there’s going to be incremental parsing so that it’s fast. It’s not slow like these IDEs.”
And that was the extent of it, that was what I took away from those conversations. But I remember putting this idea into my head about, “Oh, incremental parsing that makes sense that you want to do.” And then I could see that once I switched to using VIM, I could understand, like, oh, VIM doesn’t really do syntax analysis. There’s this really rough syntax highlighting and then the rest–it’s all based on the power of image-based text processing, not really code processing. So it made sense to me then, once I switched back from using IDEs to using VIM.
So I put this idea in my head like, huh, there’s just this unsolved problem that we all don’t have a solution to, which is the idea of having a fast editor that also has the same syntax awareness that the JetBrains ones have. That was really interesting to me. It was my first realization that like, “Oh yeah, these tools that we all use as programmers? They’re changing all the time and you can just make your own stuff.” A couple years into working at Pivotal, I was looking for something hard to work on or something different than the app development cycle that I was in.
And I still had that thought about, “Oh, incremental parsing. Like, is that a thing?” And so I started on the side. Mostly in the mornings, when I was taking the bus from Oakland to San Francisco, I would do my research about this idea. What would it take to make that happen? I had this vision from early on. I really admired these software projects where it seemed like they were solving a dense, hard problem and making it sort of fade away. Like, once it had been solved by this library, you just didn’t even have to worry about it anymore. SQLite was the number one thing like that to me. It was this C library that if you wanted a managed, structured data you just use SQLite and it completely solved a problem. It was then a non-problem. And it was this really gnarly thing that it solved and other things like that.
Like libgit2 was a similar thing where it’s like, if you want to do good stuff just use libgit2 and you can do it from any language that has bindings to the C library.
I had it in my mind that, okay, well, I should figure out how parsing works. I didn’t know that because I never went to computer science school. Then once I do know that, then I have to find out how you do incremental parsing. There seemed to be papers about it. I didn’t understand them yet, but I’ll read them once I understand parsing. And then if you could just do it as a C library, then it would be solved and you could put it in an editor like Vim. That was the end game at the time. If it’s a C Library, you could even use it in Vim or any editor. You could portably solve it.
And so that was this long-term plan, but I had to start from just how does parsing work? What are the basics there? So I worked backward from trying to read what academic papers existed in the computer science literature about incremental parsing. And they were all older, from ‘90 or ‘92 or something like that. And this was 2012 that I was starting on this, I think. And so they were a bit older. I didn’t understand them yet, but I wanted to scan them and find out what the things were that they built upon so I could go learn that and then come back to them. And so I think I found out that all the incremental parsing work that was proven in research was based on LR parsing. And so I thought, “I know that’s what Yacc and Bison use. So I should go learn how that works.”
Beyang Liu:
Yacc and Bison are parser generators, right?
Max Brunsfeld:
Yeah. Like the old school Unix parser generators. So I read up on those–my go-to was just the Wikipedia pages about them. And then also trying to take what I did understand out of that, which is like a subset, and implement it and then see what didn’t work. And then that was what cued me that I needed to understand the rest of the article or gradually fill it in with a broken implementation.
Beyang Liu:
I love this. I love this because of two things.
One, you’re teaching yourself all these things that are the fundamentals of computer science. I feel there’s this perception out there that that’s a thing you still have to go to a four year computer science degree to get. But here you are just learning all this stuff with the power of Wikipedia and the internet on your own.
And then two, you chose to tackle a problem that I feel like a lot of people I’ve spoken to, who’ve been through four year CS programs have said, “Oh no, that would be too hard. If it hasn’t been done by now there’s a reason it hasn’t been done.” Like, parsing is hard and compilers are hard, but it’s almost like you, with your fresh eyes, you’re like, “Well, why not?” Like, I have this problem and it seems like this is a viable path and why not go down this rabbit hole?
Max Brunsfeld:
Well, I might not have done it if there weren’t at least one paper that said… So this guy, Tim Wagner, proved it, proved his algorithm correct. So I wasn’t completely walking in being like, “Oh, I could figure this out. I could prove that this is possible.” I didn’t know it was possible but someone in academia had proven it or supposedly proven it. I didn’t understand the proof at that time. But they graduated from their PhD and then they landed other things.
So I was like, “Oh, okay. Well, all someone needs to do then is just implement what they said and turn it into more of a modern library type of thing. Instead of being this complex, interfaced C++ project. Just turn it into a simple interface thing and then you’ll be good.” So, that was what gave me the confidence to approach it. It was like, okay, it’s been proven, but it hasn’t been implemented. So all I have to do is popularize it.
Beyang Liu:
Yeah. So you had this idea in the back of your mind and was this what prompted you to go to GitHub or was this something that you carried with you when you joined GitHub and later came to fruition?
Max Brunsfeld:
I worked on it for quite a while before I moved to GitHub. But what did happen was Nathan, who I’d been working with and who I talked about earlier, moved to GitHub. He was basically a founding member of the Atom team at GitHub and was working on that for a while. He’d told me how he had always wanted to build a text editor and so I knew he had gone off and done that.
I knew that with Atom they had this specific goal of making a hackable text editor based on web technologies. But I stayed working at Pivotal for a while and just kept on working on Tree-sitter on the side. But I think one thing that eventually prompted me to quit there and start working at GitHub… Well, there were a couple things.
One was that they let me beta test Atom. And I remember trying it at the time and Atom was really rough, but what they did really figure out early on was how you would write packages for it. It was almost like it wasn’t yet that good of an editor itself but the experience of making a package was super smooth. You could run the tests with this built-in command in the editor and there were already stubbed files for how you do your CSS and stuff. It was all set up for you and so I thought that was really cool.
As soon as I got that, that beta came out and I got on the list for it. I said, “Okay, now I’ll take Tree-sitter and whatever its current state is and I’ll do a Node.js binding for it. Atom used Electron and so Node.js was their package ecosystem that they were drawing from. I’ll hook it up and I’ll see. That’ll be my first way that I can actually run Tree-sitter live while editing and see how it works.”
So I built a simple package. All it could do was implement the extend-selection command from JetBrain’s editors so you can hit Alt+Up or something and your selection grows to grab the next syntax tree node in your source file. So, that was really exciting to me. And then I put that to the side, but then another thing that happened is we were expecting our first kid and the lifestyle…
Up until then, the Pivotal Labs style was that you’re in person all the time, no remote. Everybody used the same schedule–nine to six–and you were in-person pairing the whole time and I loved that. It’s a really fun way to work. All of a sudden I started not wanting to do that. I wanted to be home and able to help out with things and I was seeing that coming, even though the kid wasn’t born yet.
It was like, all of a sudden I realized it. I was seeing all the benefits of pairing all the time at Pivotal, but I wasn’t really, until then, experiencing why you might want a different workflow. All of a sudden I understood that and I wanted that and then at the same time, this stuff with Atom was happening. And so that was when I applied to work at GitHub and ended up joining that team.
Like I said, one of the reasons was to have this different working schedule and it was such a big change coming from working at Pivotal, where you’re literally talking out loud all the time to people, to GitHub, where I would occasionally commute to San Francisco from Oakland and go into the office. But even once I got into the office, it was still just everyone working in their own corner of the GitHub office. GitHub, at the time, its office was all bars and coffee shops and everyone was just picking their corner and typing away.
And I remember being so unfamiliar with that at first, but what I loved about it was the independence that you could work with. At the time I joined, the Atom team was pretty small and everyone on it was a pretty experienced developer at that point. So it was very much like everyone had their branch. It was kind of this infrequent code review.
People worked on whatever they thought was important at that time. It was super independent and self-structured. And I remember loving the PR review process at first. I hadn’t really engaged in that much before because when you’re pairing all the time, there isn’t as much of a need for code review. But on the Atom team, there were already all these Atom fans on the internet who were following all the pull requests. You kind of had this audience to write for. Anytime you made a code change, you’d have to write up this really nice, technical description of it. And I remember loving that at the time and thinking, “Oh, wow, this is so different from pairing where I just casually said everything.” Now it’s like technical writing is a part of your job and you have to describe everything in this batched way. So, that was really cool.
But then what changed about that was that we hired more people and the Atom team grew. So we had a little bit more of a mix of people with different coding experience levels on the team. And then also, really importantly, Antonio joined our team. Antonio, Nathan, and I are now founding this new company together and we’ve been working ever since this time. Antonio at the time found the Atom project on GitHub and started contributing these really deep technical improvements to Atom but Antonio lives in Italy. And so there was no way anymore to have very much of a synchronous conversation.
Beyang Liu:
Yeah, time zones.
Max Brunsfeld:
And at the same time, we started working on things that were not that way where everyone has their own branches and we’re all working separately and reviewing each other’s code. There would be some big, hard project–like we needed to rewrite a certain data structure in C++ or we needed to optimize something–and Nathan and I and Antonio started not wanting to just have our own branches and work separately and review PRs. We wanted to work together. But the only flow that we could use to make that happen was that we would have this continuous handoff where I would leave and I would just type a long, long message in Slack saying, like, “Okay, the code is not totally compiling. This part is broken. I just decided to change this to this and here’s why.” It was very different to the PR review workflow.
Beyang Liu:
Oh, wow. So you would hand off these things while they’re still works-in-progress.
Max Brunsfeld:
They were the same branch. We would just continue working. And one of us would get up and by the time one of us was going to go to bed, there’d be this short overlap and that’s how we work to this day.
Beyang Liu:
So the person who’s picking up where you left off, is there any live conversation in the transition point where you’re like, “Here’s what I did and blah, blah, blah, and watch out for this, because this is a big hack” or is it just like you write the message?
Max Brunsfeld:
We try. We did have that. When I was getting up in the morning or Nathan was getting up in the morning, Antonio would stay up late enough so that he could speak to us.
Because it would be the late evening for him and not like the middle of the night. But the other way around, when Antonio was waking up, nobody was there. So we just had to write this pretty complete description and we would try to not leave it broken, but we often did and so we had to explain everything in Slack and paste code into Slack. So it was like the most effective way we could find to work together but it was still challenging to have to write that way. We didn’t really use GitHub code reviews for that because it was all on the same branch and what we were having to capture was not in a complete state. It was this just where we left off.
That was a point when I realized, huh, none of our collaboration tools are quite addressing this problem right now but we made it work. I don’t know. Maybe I should fast forward at this point to other things.
Beyang Liu:
So I think a lot of this is foreshadowing, at least from my point of view, the eventual creation of the company that y’all started together–Zed. But before we get there, I just want to finish the time at GitHub. And I’m curious about how Tree-sitter got incorporated into Atom and different parts of GitHub’s architecture. Because in the beginning you just made this extension or package for, it was like this toy, and it wasn’t really doing much. At what point did it find an application inside GitHub?
Max Brunsfeld:
Yeah. It took a while. Fortunately we had this company summit where we all flew to San Diego and we stayed in a resort together and GitHub-
Beyang Liu:
How big was the company at this point?
Max Brunsfeld:
I think the number of software engineers at the company was around 150. So there were a lot of people I didn’t know because I’d really only met the Atom team, which was this small island. And Rob Rix had been at GitHub for a while and I didn’t know him, but he went up on stage in front of all the software engineers at the company and pitched this idea that we should be doing parsing of the code on GitHub and trying to build productivity features that made use of the parse tree. After he gave that talk I had to go and find him and say, “Hey I really think we should do that too. And I’ve been working on this parsing framework.”
My focus was on text editors at the time, but it was kind of a lot of common design constraints for a text editor and also for github.com because they’re both these tools where we, the tool creators, don’t know a ton about the code that the person is viewing with the tool. We just know the file extension and there’s a lot of different languages that we want to be able to plug in. And the parsing needs to be really fast in an editor because your display is updating while you’re typing and you need to see that. But in GitHub, it’s just because there’s a lot of pushes per second and so the parsing needs to be really fast.
And so I talked to him and he kind of got set up with Tree-sitter and they were doing code analysis in Haskell. And so they quickly created a Tree-sitter binding for Haskell and they explored a bunch of ideas doing stuff with diffs, but what eventually shipped was this first version of code navigation in GitHub. It was still simple. It was similar to Ctags where it’s not doing a semantic analysis of the code, but it’s finding functions and finding classes and types and finding call sites to functions and matching them up by their name. It was at least better than Ctags. Also, you could find not just the definition, but the call sites as well–although newer versions of Ctags might do that as well.
Beyang Liu:
This was back in like 2015 or something like that?
Max Brunsfeld:
Yeah, I started working at GitHub at the end of 2014. I think this might have been 2015 or 2016 when they started working on this. I was going to be involved from afar. I would help them debug Tree-sitter stuff and they were working on grammars and made a bunch more grammars for Ruby and PHP and Python.
But it was kind of separate for me. It was cool because it was this project that I had put all this time into and I cared a lot about. We weren’t really ready to use it on Atom yet because we had other product features that the team had prioritized, but this other team at GitHub was getting used to that.
Beyang Liu:
That’s awesome.
Max Brunsfeld:
So, that was nice. And then finally, it was almost through helping them with Tree-sitter and their work on grammars, it helped Tree-sitter mature enough to where it could highlight JavaScript and be more reliable than the grammars that we were using in Atom. It was more efficient. And so once we got to that milestone of “the highlighting is better and it’s faster” then we finally incorporated it into Atom.
Beyang Liu:
Awesome. And the thing it was doing in Atom, was that code navigation as well, or was that more on the syntax highlighting side?
Max Brunsfeld:
We were just using it for syntax highlighting and code folding. So, you could fold functions that weren’t necessarily indented in the way that Atom had previously required them to be. And also that command I mentioned earlier, extend-selection, we made that a built-in feature of Atom so you could grow your selection up the syntax tree. It’s funny, it’s this unassuming feature, but it was this feature of the IDEs that I had used to use a long time ago that I had loved so much. You could put multiple cursors on different codes that you wanted to refactor and grow them all up the syntax tree so they would all be selecting the right thing and then do refactoring.
Beyang Liu:
That sounds really satisfying.
Max Brunsfeld:
Yeah. I love it. So it didn’t have a huge bang but I just loved it.
Beyang Liu:
That’s awesome. What about Tree-sitter as an open source project? Because now it’s popping up in a lot of places. I think more and more people are using it inside other editors and building on top of it. I know GitHub continues to build on top of it for their code navigation. Have you kept up with the goings-on of the project and where it’s being used?
Max Brunsfeld:
I do. Yeah. I try to. It’s been taken up more and more. I hear about companies that are paying people to work on grammars and parsers and stuff, but a big one was Semgrep. They make tools for searching for certain patterns and especially anti-patterns in source code and correcting them. They switched to using Tree-sitter. And then one that made me really happy, even though I’ve moved on from using Vim, was that Neovim incorporated a built-in feature of Neovim that you could parse code with Tree-Sitter.
And what really, I think was the final thing that helped more people find a use for, was the library that we built. That was when I eventually left the Atom team while still at GitHub and went to work on the code navigation thing full-time and we built this feature in Tree-sitter that was about searching for patterns in a syntax tree. And our goal was to make it so that you could have one engine that was like a pattern-matching engine in a syntax tree that would both let us do syntax highlighting. You’d be able to say there was like a language that was based on S-expressions like Lisp where you could look for a function containing a return type of law and you could syntax highlight that. And then we can use the same engine for matching things like that for recording functions, calls, and definitions for code navigation and we just called it tree queries.
When we built that, it finally became: if anyone wanted to do syntax highlighting with Tree-sitter, it would be more like a one line thing now that there was that system and people could use the same tree queries even if they were using Tree-sitter from different application languages. Like, Neovim was using a C codebase. And then GitHub, we had like a Go/Rust/C codebase but we could all use these same queries to find the patterns in the trees. And once that sort of layer was moved down into the Tree-sitter library, then Neovim could really easily do highlighting and use people-made plug-ins for other text editors to do highlighting. I think that a lot of people started using that, which was really cool to see.
Beyang Liu:
Yeah. That’s awesome. I think the code navigation applications are particularly interesting because obviously Sourcegraph has its own code navigation implementation. But it’s kind of interesting to think about the different technical approaches that we’ve taken versus what GitHub has taken.
Max Brunsfeld:
Yeah. It’s really interesting. GitHub has taken a pretty… It’s definitely a difficult path trying to build your own code analysis pipeline that’s not feeding off of existing compilers and stuff to do the code analysis.
I think what you’re doing at Sourcegraph… It’s almost like it’s very easy to see why that approach makes a lot of sense. And GitHub has this interesting, different value, coming from different features where being able to do something ecosystem-wide is almost the only thing. It’s not the only thing that matters, but if a feature isn’t ecosystem-wide, it’s not going to hit that strongly on GitHub. Compared to Sourcegraph, if it’s your company’s codebase and you’re trying to search that, you’re fully capable of setting up a little compiler or CI pipeline where you run your own static analysis on your codebase and all of your own dependencies.
Beyang Liu:
Because GitHub is starting from this point of view where like whatever code navigation they provide, they want it to work across as many languages as they support as possible and regardless of whatever, like, build configuration you encounter. I think that that’s a lot of the trickiness of this problem. It’s like different languages and different build systems and so the Tree-sitter approach is nice because it’s sort of like this… I mean, it’s not language-agnostic to the point where one grammar works for all languages. You have language specific grammars, but it is one kind of parser generator. And then you’re not compiling each language and you’re not trying to be build-system-aware. It’s almost like you get 80% of the way there in terms of accuracy and hope that the set of heuristics that reduce to patterns, like you were saying in the AST, for what looks like a definition or what looks like a valid reference for a given definition.
Max Brunsfeld:
Yeah. That’s the goal of this team. I think it’ll always be a lower fidelity accuracy than what the compiler will give you. But it’s like trying to operate with this constraint of being able to run on every random repo in a given language that exists instead of having to have the repo conform to certain conventions.
Beyang Liu:
Yeah. Makes sense. Cool. Well, I want to get to Zed because that’s the thing that you’re working on now. And it looks like from the way you’ve described your entire career as a programmer, it feels like everything has built up to this moment in terms of the way you got into it through collaborating and communicating. You’re always thinking through workflows and you have such a deep appreciation for editors and dev tools in terms of the impact they have on the way people work. So what is the backstory of Zed and what is Zed? Why are you building it and what is it intended to do?
Max Brunsfeld:
Yeah. So Zed is, like I said, founded by Nathan and myself and Antonio from the Atom team. From the 10,000-foot level, we just wanted to build the text editor of our dreams. We’ve built up this dream set of features and qualities from working on text editors and programming tools for a while. When working on Atom, we started with a hackable text editor that wasn’t very polished or good in most other ways, but besides being the most hackable it could possibly be, we were then trying to evolve it to become fast and powerful incrementally after shipping it and having all these locked-in APIs that we had to support. And so we learned from that so much about how we wish the application worked in terms of what type of UI framework we even wanted and what the constraints were of using the DOM or using web technologies to render the UI.
Beyang Liu:
Thinking about performance issues mainly or are there other considerations there?
Max Brunsfeld:
A lot of it is performance. It’s like two things. We want to take a lot of what we learned about the different ways people collaborate on code together. We’ve all had a mixture of experience from doing these really fine collaborations, this pair-programming, but also doing this really asynchronous workflow of pull requests and code review. And then this third thing where there’s like many people on one branch doing handoffs, where it’s like a mix of the two.
Now we know about some of the technologies that are needed to collaboratively code. We built this collaborative coding plugin to Atom–it was called Teletype–that used what Nathan and Antonio did, this huge research deep dive into CRDTs, communicative replicated data types, and what the most efficient kind of latest, greatest flavor-
Beyang Liu:
For listeners, my understanding is that those are the data structure algorithms for doing live updating conflict resolution, like in Google Docs when there’s two people typing and sometimes they overlap. I don’t know if they actually use CRDTs in Google Docs, but this is a solution for that problem.
Max Brunsfeld:
Right. Yeah. It and Operational Transformation are like the two kinds of approaches and CRDTs require less central coordination than Operational Transformation. And there’s been all this work in recent times to make them efficient enough to use all the time in an editor. Yeah, after GitHub, Antonio went and worked out of this company called Ditto. That was all about data replication and he learned even more about CRDTs. We’re trying to take that, take what we learned there, and also take what we learned about building a fast text editor. Something we all kind of became passionate about when working on Atom was wanting to make an editor that was responsive all the time and feels like it’s never in your way to use. And to build an editor that sort of lets you collaborate on code in a way that we have not yet been able to do.
So, there’s some common features that exist now in other tools but I want to come in, in the morning, when I come to work, and not necessarily open up Slack or GitHub, but open up my editor and see who’s there. I also want to see what conversation has taken place and then be able to look through that conversation and see where in the code it applies. I want to click on what somebody said and be there in the code at that moment that they said that at that time and also see where that code is now and how it’s evolved since then.
And be able to for myself, on the other flip side, like at the end of my day when I’m handing off to Antonio, rather than going over to Slack and typing this long thing and pasting back code snippets in Slack, I want to click on some code or go to some code that I’m talking about and just start talking. And that association between this code at that moment in time and what I’m saying to be captured in the tool, or be able to click on a compiler error and start talking about that.
Beyang Liu:
It’s like you’re creating this tour of some recently written code and being able to link dynamically into specific snippets. Is that kind of the idea?
Max Brunsfeld:
It’s linked back and forth and in a way that’s sort of useful in a real time situation where if we’re there together, instead of typing in Slack about the code, we’re typing in the code editor. We’re also simultaneously making code changes, but also for conversations that happen at separate points in time. Like right now, if I want to find out about the history of a certain piece of code, I would go to GitHub and see these closed pull requests that now have discussions in them about that code at different points in time–but it’s hard. Like I would go to the pull request and search for certain words that would come up in the discussion but I can’t just go to that file and say, tell me the discussions that relate to this.
Today, comments are on diffs that are committed and pushed. They’re not on just any code and they’re not… So what we envision is a more fluid, collaborative environment where we can code together. We can both be making edits to the code together when we are both around, but we can also diverge and do our own thing and pull each other in when needed. It doesn’t have to be this very intense pair-programming situation, but it’s a more lightweight pulling in of somebody. It’s not like, “Hey, can you Zoom and screen share?” or like “Can you fire up this tool?” It’s just like, they’re always there to hear what I’m saying. One way that we talk about it is we’re trying to recreate the vibe of a pod of people sitting next to each other casually going over to each other, forming groups ad hoc, and then splitting and stuff.
Beyang Liu:
It’s like that mind-meld feeling.
Max Brunsfeld:
Yeah.
Beyang Liu:
When you’re hacking with a group of close-knit people.
Max Brunsfeld:
Yeah. But we’re trying to do that for remote.
Beyang Liu:
Yes. Right. Even if you’re in a different time zone and that person’s asleep, you want the context and information transfer to be as smooth as possible.
Max Brunsfeld:
Yeah. And then also to make all that collaboration that happens in that close-knit setting discoverable in the future anytime. It’s almost like you wouldn’t have TODO comments in your code, it would just be so easy to find comments in code that could just be comments in this tool. I could be looking at some code and be like, why is this like this? And then I don’t have to go do a lot of work to dig up the conversation that is about that code it’s connected to in the tool. That is our vision.
Beyang Liu:
It sounds like there’s a lot of overlap, at least on a feature basis, with various other things. Hearing you describe it, one thing that comes to mind is pair-programming tools like VS Code Live Share that allows you to get into the same editor and see someone else’s cursor move around or CodeStream, which was this other startup that we had in the podcast last season, which was all about having discussions inside your editor. Code Spaces, also–maybe there’s a web component to this. The other thing that came to mind was like, there’s a VS Code extension called CodeTour that allows you to build these guided tours and code bases. Sourcegraph also has a similar feature. We call them search notebooks–it’s like markdown interspersed with code snippets and search queries.
Max Brunsfeld:
Oh.
Beyang Liu:
But my sense is that you’re not really building this up as a sum of the features–what you’re really after is this overall end-to-end solution that captures a certain mode of working that you’re uniquely well positioned to have insight into because of the way that the three of you built Atom in the early days, the way that you’ve been accustomed to working in the course of your career. This combination async-remote but highly collaborative, highly communicative setting. Sounds like you’re trying to bring that feeling to every developer.
Max Brunsfeld:
Yeah, that’s exactly what we’re trying to do. The way I kind of think about the space of tools that touch on this, like you mentioned–it’s almost like before there was GitHub, if someone said, like, “Oh, we’re building a new thing and you host your Git repos there,” you could kind of be like, “Oh, it’s kind of like Sourceforge or isn’t there already that?”
Beyang Liu:
Yeah. Right.
Max Brunsfeld:
And it’s like, there was, but it turned out that the idea of working together on code was really big and deep and needed. There ended up being all these things that came off of it and it was such a big change to software development to have something like GitHub that it took some real creativity and a lot of people trying different angles on it to propose a workflow in this huge design space of collaboration modes that could exist that clicked with people. And that’s what I want to do, I want to be in this huge design space of taking all this code collaboration into the real-time realm.
I want to execute this vision that we three have that we think will click for us as a team and will click for other people who are searching for more productive ways to work together.
Beyang Liu:
Yeah. Like you said, GitHub was not the first collaborative code host. Before that there was SourceForge and then there was Google Code, which I remember using back in the day. But then they were very different because GitHub came along and it was such an awesome bundling of, I think, multiple really well-built applications. You had a really good issue tracker that had well polished features. You could paste an image in there and it automatically turned that into markdown. That was awesome. You had a good code review tool. It worked well with Git, which was kind of this up-and-coming, new version control system.
And so, just a combination of that, you threw all those together and became almost something new, like a place where developers wanted to go to present their code to others and also find other people’s code. So, now you want to do that for the editor. I guess my question is how close are you to cracking that nut? How many more months do the rest of us have to wait before we get to try what you all have been working on?
Max Brunsfeld:
That’s a great question. We haven’t totally figured out how long it’s going to take us. Right now we have a working editor. Tech-wise we have an editor that we’re building all in Rust and we have a custom UI framework that Nathan built. A lot of that is about doing media mode rendering on the GPU using the graphics card for everything and…
Beyang Liu:
Sounds intense.
Max Brunsfeld:
Yeah. And then we have this CRDT-based editor where you’re always… The cool thing about CRDTs is they actually enable other features too even when you’re not collaborating. This branched undo thing sort of falls out naturally, not branched, but the ability to undo anything even if you didn’t necessarily undo everything after that. And similarly, like, redo something even if you made a change after that you undid.
Beyang Liu:
Oh yeah. I know exactly what you’re talking about. I feel like every developer.
Max Brunsfeld:
You undo undo, better not type anything, otherwise you won’t be able to redo. So we have got this really fast editor that is always a CRDT and so you’re at any time ready to start collaborating. And we have Tree-sitter integrated so right now, we’re just using it for syntax highlighting and code folding, but we’ll be prepared to build nice, subtly syntax-aware variance of any feature that we want to build. We have a basic collaboration system going where we have a server that’s also written in Rust and it’s fully integrated with our editor client. Right now, it’s like a monorepo and they share code so we have this fast ability to collaboratively edit documents.
But there’s a lot of stuff you need to build to make a text editor actually useful to people, stuff that’s just very pretty basic like a tree view of your project. Like just all kinds of things: spellcheck, markdown preview–language integration is a big one.
Beyang Liu:
Yes.
Max Brunsfeld:
We want to ship. Our first ship is going to be… We want to nail the experience of editing Rust in Zed first just because we use Rust every day. And Rust has an amazing language server that has been built, rust-analyzer, and we kind of understand the way that people work when writing Rust, which is often: make a change, 100 compile errors occur, and then you go one by one and fix them all. The compiler tells you–it’s like compiler-driven development. We have ideas about workflows and innovations we want to do about how to make that experience of coding Rust in Zed really delightful.
So, that’s what we’re kind of focused on right now. It’s like we’ve built out a lot of what we think are the high risk pieces, like how the collaboration is going to initially work but what we haven’t built is a lot of the stuff that already exists in many other editors, but it’s just a lot of work to do. So we do need to build all those basic editor features, but then our plan is we’re going to ship an amazing Rust editor/IDE that’s collaborative in this new way first. And as far as when that’s going to happen, that’s tough. But I think next year.
Next year we’ll have something up.
Beyang Liu:
Well, I am super intrigued and I bet a lot of people who listen and watch this show are also super intrigued. Is there a place where people can go to subscribe to news or notifications about this new editor that you’re building?
Max Brunsfeld:
Yeah. Our website is zed.dev. There’s not a ton there right now but you can… Or if not now we were going to have a way to sign up so that if you want beta access when it comes out and to get updates, release notes, things like that so we’ll keep you updated.
Beyang Liu:
All right. Cool.
Max Brunsfeld:
I’ll add you to our list so that you can be on our insider’s list.
Beyang Liu:
Yes, please do. I would love to be an early user. I will give all the feedback that I can, and I think that goes for a lot of people listening in on this too.
Max Brunsfeld:
That’d be great.
Beyang Liu:
Maybe we can get like a sign-up link for podcast listeners later down the road. Cool. Well, Max, thanks so much for taking the time to chat and share your experiences and also what you’re up to lately. It’s been awesome having you on the show and thanks so much.
Max Brunsfeld:
Yeah. Thank you so much. It was great talking to you.
This transcript has been lightly edited for clarity and readability.