Matt: So I think that serverless applications, right, serverless function, Are are one of those places where we, at fermyon, you know, building tools like Spin can really make some vast improvements and say, look what, here's, here's a promising first step for why web assembly is this kind of third wave of cloud computing
Andrew: Hey, before we get started we want to remind you that we have a Patreon. If you want to hear the full episode, you'll have to become a member over there. You can also support us by going to our merch shop and buying one of our shirts.
And with that, let's get onto the episode.
Andrew: Hello, welcome to the Dev tools FM podcast. This is a podcast about developer tools and the people who make 'em. I'm Andrew, and this is my co-host, Justin.
Justin: Hey, everyone. Uh, our guest today is Matt Butcher. Uh, if you've worked with Kubernetes, there's a really good chance that you've used one of Matt's tools. Uh, welcome Matt. So glad to have you on. Um, before we start, would you like to tell our listeners a bit more about yourself?
[00:01:01] Matt's Background
Matt: First of all, I feel like I had a lead with an apology for all the Kubernetes tools, but, uh, , um, uh, I guess I, the, the best way to describe myself as, as a recovering philosopher, I, uh, my original goal was to be a philosopher, like philosophy professor. I have a PhD in philosophy. I actually, Taught philosophy for a while, but somewhere along the line of it turns out you kind of have to pay your way through school and things like that.
And philosophy might not be the highest paying job out there. I mean, it's pr probably close, like top 98%. No, it's nowhere near that. Uh, but, uh, so I started writing code sort of in high school, carried that on in college, worked in like the computer lab at college while I'm studying philosophy. And at some point I was kind of like, man, philosophy moves really slowly and computer science moves really fast and I like this a lot more
So . So I spent, you know, first part of my career really in more like, Systems engineering, then into content management systems. I did Druple for a long time. Druple actually got me into cloud computing. I ended up working at HP Cloud originally to do their Druple sites and kind of fled my way out of the Druple world and into the OpenStack world.
Uh, absolutely fell in love with cloud and have been doing that for the last years, uh, and then, you know, uh, moving close to the present. Uh, several years ago I was at a small startup company in Boulder called Deis. Uh, we were building Kubernetes solutions. Uh, we started the Helm Project and the Illustrated Children's Guide to Kubernetes, uh, along came Microsoft and said, Hey, y'all look like a, a great group of people to join our brand new Kubernetes team.
And so I spent about five years at Microsoft. Really, really enjoyed it. But along the way, you know, something else came along this web assembly thing and I'm like, oh, I gotta, I gotta do this. So a bunch of us left Microsoft and started a startup. So I've been doing that for, since November of 2021.
It's probably the fastest I've told my ever life, uh, ever told my life story.
And it's, it's actually kind of, kind of depressing when you tell it that fast.
Justin: it. It's good though. It's concise. It hits a lot of points.
Andrew: Yeah. Super interesting. I always love hearing about people who didn't start in computer science and then ended up in it, uh, for whatever reason. And it's, it's awesome that you're like aiming to be a PhD and then you're like, ah, nah, nevermind. I'll
Matt: Yeah. .I, I, I got through the part where I had to pay for all of that and then, then it was when I decided I didn't want to do it.
Andrew: oof.
[00:03:31] Boarding Kubernates and Taking the Helm
Matt: So Kubernetes, right? Uh, if, if we talk about Kubernetes in context of what we were trying to solve, right? Uh, we've, we've got this first wave of cloud computing that comes along, right? Uh, virtual machines, and all of a sudden we're all kind of collectively going through that aha moment of going.
Oh, right. So I can take my operating system and I can kind of like package it up and lease somebody else's hardware and they're gonna do the part I hate doing and I'm just gonna keep shipping my, my operating system image up there every time. And we're all kind of, you know, absolutely in love with this idea because it means we don't, we're not slaves to the blinky lights in the data center anymore. And then we start to fall out of love with the thing that we were once passionate about. Right? Uh, for me it was like, I cannot believe I just spent three hours building this image, pushing it up into the registry, and now I realize that I forgot to include curl, so I'm gonna have to do the entire thing again.
And, and, Uh, you know, at the same time we're dealing with orchestration system or, uh, sorry, you know, uh, uh, um, developer tool or, uh, DevOps tooling like chef and Puppet. And again, we're kind of feeling the same way. Hey, this is great. I can write procedural code and instead of writing a whole bunch of commands, I can just kind of enshrine all of this in, in, uh, R Ruby, right?
Or Python. And suddenly it, it feels the same, it feels like the same kind of artifact as my application. And then along comes this next shift, right? And containers become a thing and we start to fall in love with the lighter, you know, the little spelt container that feels so much smaller than the gigantic virtual machine image I was building.
And this, uh, you know, Kubernetes comes along and the orchestration model looks so much more attractive, and suddenly we're talking about declarative. And declarative is the best thing ever because procedural is. So, yesterday I used to have to write this ugly, old ruby code in order to describe what my architecture looks like.
And now, All I do is write this beautiful yaml. Um, and, and you know, we, we kind of go through these funny phases. Um, and uh, again, every time to me it's like, I've found the last tool I'll ever need. This is the best thing I've ever found. You know, this will solve all of the world's problem. Let me tell you why Kubernetes is gonna make you a happier person.
And then a few, few years later, like now, I'm like, if I never see a yaml file again, I will be happy, you know, declarative, ah, so much of a pain to use declarative infrastructure. And, you know, the, the wheel moves on, I guess
Justin: Indeed, indeed, the cycle continues.
Andrew: Yeah. So earlier you mentioned that, uh, you've worked on a bunch of different, uh, Kubernetes tool, uh, pieces of tooling. So like, what are those, and like what do they help you do?
Matt: Uh, kind of the big, big one. So does, so does give a little bit of context. Um, uh, you know, I, I talk about, you know, moving from the virtual machine world into the container world. I started at working at, at Deis, uh, Deis was building a platform as a service. We wanted to build the open source competitor to Heroku.
Uh, and we wanted to do it using containers, uh, which were of course the, the, the newest shiniest, uh, thing in view at the, at the moment. And it was going great. And we were really excited about what we were building. We were using Buildpacks just like Heroku. Uh, and then Kubernetes landed, um, uh, was made public.
And uh, we looked at it and I had come from, from Google prior to that one. So there was a sense in which looking at it was like, oh, hello Borg old friend. Right? Uh, and in a sense it was also like, oh, you know, I understand right. Why this would be a really powerful way to orchestrate our stuff in our platform as a service thing, right?
This is gonna, this is gonna be what sets us apart from Heroku. We'll have containers, we'll orchestrate them. We won't have sort of the startup time problems that Heroku at that time was having. Um, and then we got, we got working on Kubernetes and realized that it was actually kind of hard to install things into Kubernetes.
Uh, you know, you, you ended up having to write a yaml file, uh, for each different artifact you wanted to describe. So if it's a service gateway, you're writing a service. If it's a, if it's a piece of, you know, secret data, you're writing a secret yaml file, you need, you know, replica sets or, or at the, at the time replication controllers and then later on deployments and all this and, and it, we were writing a lot of yaml and we were, and I was writing the same.
yaml-y objects over and over again to try and install the latest version of our platform into Kubernetes. And we had a hackathon, a company hackathon, as part of sort of an all hands meeting. And so I came into that frustrated with not wanting to write the same yaml anymore, and uh, and got together with a couple o other people at Deis who were also similarly, one of 'em was looking at Kubernetes and the other one.
was Uh, was, was kind of working on something vaguely NPM ish at the time and we're kind of chatting together and going, you know, we could build a package manager for Kubernetes. So we built this package manager called Kate's Place. This was our entry for our hackathon project where the prize was a $75 Amazon gift card. And we are like, we are gonna win that gift card. Nothing is gonna stop us. You know, we go out for the team dinner and, you know, we're like, all right, well we finished the main course, do you think we can kind of sneak out and go work on our package manager? So we wrote this package manager called Kate's Place that was sort of approximately modeled on, you know, a little, you know, uh, one teaspoon of npm, a couple tablespoons of, of APT get.
And uh, and, and we got to the show and tell thing and we got there and demoed Kate's place and everybody was like, yeah, that's neat. And we did end up winning the $75 gift card. We split it three ways. I blew mine on coffee and I am not ashamed to admit that. So the, the, the day after that, um, the ceo and so the offsites all done and the, the, the dust has settled and the CEO and CTO call me and I'm like, uhoh and the CEO and CTO call you at the same time you're in trouble for something.
And I'm thinking, did I do anything at the offsite? I, and they're going, so, you know, that thing that that demo you did the little package manager thing? And I'm like, yeah, yeah, yeah. And they're like, we think there's some potential for that. And I'm like, oh, good, good. And they're like, and we think you should start working on that.
Like, stop working on the, the paths so much and, and really kind of pivot over and work on that thing. And I said, oh, good, good. And they said, there's just one thing. We really don't like the Kate's place name. And they're like, it sounds like a coffee shop. And I'm like, I know, right? Doesn't it sound pleasant?
Isn't it like the coffee shop? They're like, no, no. So I called Jack Francis, one of the other two guys that I was working on along with Remus, uh, called Jack Francis. And, and I'm like, I got good news and bad news. The good news is we can work on this Kate's place thing, you know, forever. And the bad news is we have to come up with a new name and they kind of want it to be, you know, Kubernetes ish.
So we sat around with a nautical dictionary just on the phone together reading each other entries out of a nautical dictionary back and forth. It's like mast, Nope, captain. Well, that's actually what Kubernetes means. We're just kind of back and forth. In Jack's version of the story. We said the same word at the same time, but really Jack said it and I probably just repeated it back.
He's like, what about Helm? And I'm like, oh, yeah. So, so we name renamed, renamed the package manager. Helm started working on that. Um, and the project kind of grew by, by leaps and bounds well beyond what we, what we thought it would ever become. Uh, I think at this point it's, uh, probably 70 ish percent of Kubernetes users use home to install, uh, helm charts also from the nautical dictionary.
Um, which were really, you know, a, a great way to solve that problem that I had at the beginning. It, you can encapsulate your YAML files, you can templatize them. Then you don't have to write nearly as much yaml in order to use Kubernetes. And hopefully the world was a slightly better place. Uh,
Justin: There's two lessons here. Uh, good things come from hackathons and pick a theme. It'll make things easier to name
Matt: Uh, worked for, worked for Ruby and, and that whole ecosystem, right? So, yeah.
Justin: Back when I worked at Artsy, uh, it was, it was physics and art terms, so we had like a list of physics things. So we had like, metaphysics was our GraphQL service and force
Matt: oh, I love this already.
Justin: the HTP service and Yeah.
Matt: Oh, oh. But I mean, I gotta say the Java ecosystem as it got going, the, the number of bad coffee metaphors, we, we might have actually passed some kind of threshold after which every new project, it was like, I don't think Frappuccino should be the name of a thing.
Andrew: Yeah, I, I think coffee script killed all of that.
Matt: we over caffeinated at that point. Yeah. Oh, well, I guess we had Java then. Java script, then coffee. Yeah. Yeah. We've really, we've really leaned hard into the caffeinated beverage thing for a while. Um,
Justin: I think the trick for the, the, the philosophy behind naming things is that the names have to be differentiatable in some meaningful way. So if you can tie loose, logical, you make it a loose, logical analogy to like these two things. But when it's just like a bunch of like nouns for drinks or whatever, it's much harder to say, yeah, this drink kind of represents this like web service, you know?
Matt: Yeah, I mean, it gets to be a stretch really at some point. Um, we used to have an in in inside joke at, at days that we would substitute out the word coffee with the word slur. Uh, and, and it was like one of those things where the, the, the sort of an mono poetic sound behind slurm just sounds so gross that, uh, yeah. Kind of makes you, makes you shiver a little bit. I don't know. It was slim slur from Future Ahma. I think it might have been from future
Justin: I
Matt: even entirely sure what the origin of that was.
[00:13:14] Kubernates Kid's Book
Justin: Um, so I, I want us to transition to talk about spin a little bit, but before we do that, um, so, so Kubernetes is like infamous for both being like incredibly, incredibly helpful, but also pretty hard to wrap your head around and like all of its facets and really get into the weeds.
Something that I love is that you helped co-author, uh, like illustrated children's guide to Kubernetes, and I'm like super curious, like how did that come about? Like who's like, you know what? We need to make a children's book out of this. That'll help
Matt: I mean, does, it's often the solution to life's problems. Uh, I, I, the, I am not kidding you when I say this, it, the, the Illustrated children's guide to Kubernetes happened the same day that Helm did. uh,
Justin: Wow,
Matt: in this whole, in this whole, uh, offsite that we were having, where, where we had the hackathon, uh, we, uh, the CTO and I, and a few others had been evaluating Kubernetes as the platform to pivot to.
And, and, and yet there was still a, a huge contingency at the company that had never heard of Kubernetes. And of course, I mean, even the word itself is so clumsy that, that, that in order to get people kind of engaged in this, it's, it's a difficulty, right? So, so, uh, Gabe, who's the cto, comes up to me and says, yeah, yeah, I'd really like you to introduce Kubernetes to everyone, uh, during the offsite.
And I said, okay, like all of engineering. And he says, oh, no, no. When I say everyone, I mean like, Marketing's gonna be in the room, finance is gonna be there. I mean, just kind of introduce it to everybody. And I said, oh, okay.
And he said, but don't worry, I got you covered. I I'm gonna give you the slot right after lunch.
And I'm like, the slot when everybody's falling asleep right after they've had a really big meal. That'll be, that'll be great, Gabe. That'll be great. So, uh, so the night before, uh, I had to give this presentation. I'm just kinda like trying to build some kind of presentation to articulate the, even what Kubernetes was, let alone the value of Kubernetes in this kind of generic way. and, uh, you know, I, I have three daughters, um, and, and I, I don't know what the, what the moment was, but I'm like, uh, it would be really funny if I took a couple of the kids' stuffed animals and took pictures of them and made up a story about a little PHP program called FPI the Giraffe who was trying to get deployed on Kubernetes.
And so I, I took a couple of pictures. I wrote at this really, really bad PowerPoint presentation, but it was essentially the text of the Illustrated Children's Guide of Kubernetes. And, uh, and, and at during lunch, I grabbed a big old poofy chair from another room in the hotel where we were doing the conference, dragged it across the lobby into the room, and put it up in the front and sat in the front, like I was reading a story and projected this slideshow behind me and gave the first version of the Illustrated Children's Guide of Kubernetes to Kubernetes.
Now, marketing was in. Yeah. Yeah. So marketing is in the room watching this presentation and Karen Chu is, is watching the presentation, kind of nodding along and, uh, I didn't know if I'd kind of reached, I, I don't know. I had no idea whether I had done a good job. It was, it was a really rinky-dink presentation right.
There really was nothing special about this. Couple of weeks later, Karen calls me and says, Hey, you know that, that, that slide deck you showed at the offsite, I have this wild idea. What if we really made a kid's book out of. and what if we, you know, use this as marketing material? We could give it away at Con, which the very first Cuban was coming up only a couple of months away.
You know, this would be a great way to get people enthusiastic about Kubernetes. We could mail it to people and, and she's like, I've already lined up an artist. I'm working on some storyboards. Thankfully, she's much better at all of these things than I am. Uh, because, because otherwise you'd be looking at this kind of funny looking giraffe and, uh, Karen's actually, Karen is brilliant.
Uh, before, before asking the illustrator to design the giraffe, we're dais, we're on a shoestring budget. Uh, we, we can't afford a lot of stuff, but she really wanted to have a giveaway to go along with the book. So she goes and finds all the places that can source little squishy toys in the shape of.
Giraffes and owls and the characters in the book, and find some that look, you know, appropriately squishy and kind of cartoonish, and then sends our illustrator those and says, can you, can you just kind of riff on these so that they're, they'd be close enough to your illustrations that we could give these out.
And so we kind of launched the book as a little bootstrap startup with, uh, you know, an illustrated book, uh, you know, illustrated book along with, you know, a line of toys to go with it. And we gave those away at the first con, and it just kind of took off from there. I think at this point there were several books when, when Microsoft acquired Deus, um, CNCF reached out and said, you know, would you be willing to donate fpi?
And that that whole kind of illustrated children's guide to CNCF so that we could use it as more like a mascot. Uh, and, and Karen and I were very enthusiastic about that, and Microsoft actually was very enthusiastic about that too, because they had, you know, really started investing in cncf in the Kubernetes space.
Uh, and so gradually we got to kind of turn things over to C N C F, which in and of itself was really exciting. And now there are all these other, uh, you know, illustrated children's guide, uh, books, you know, illustrated by different people, written by different people. And it's always fun to kind of see this and go.
Yep. That, that silly little after lunch presentation, it just took the right person in the audience to see that and go, we could, we could really do a lot with that. Uh, this has been kind of a wild ride and it's like, okay, so I, like, I'm proud of the code for Helm, right? And, and, and I feel like I should view the Illustrated Children's guide as a thing to be just a little bit embarrassed about.
But I'm not, I think I actually am more proud of the Illustrated children's guide at this point than, than any piece of code I've ever written in my life.
Andrew: Well, you can look back on the children's book and go, oh, that was fun. You can look back on your old code and go, oh God, what was I
Matt: Yeah. you are not wrong. I mean, that is totally why that's the case. It's like the, the stuffed animals are still cuddly. Uh, the, the code. Yeah. It's just prickly all the way at this point. It's like, oh, oh my goodness. I wonder why we, why we have the, that feeling about our code. I mean, every engineer I've ever talked to is like, I get embarrassed about the code I wrote back then. Do we really learn so fast that the code from only six months or a year ago, is that embarrassing? It really seems to be the case. In which case, you know, that says something about human beings in our ability to improve, or at least human beings in our ability to cast shade on things we've already done
Justin: I think it's in part maybe that our taste over time changes, not that something is objectively good or bad because you, you're generally, as you look at code from another person, you might say, oh, this isn't how I would write it, but like, it's not my code, so it doesn't
Matt: Yeah.
Justin: But when you look back on your old code, you're like, I have this taste now and I wrote it that way and I would never write it that way now.
And like, why did I do that?
Matt: I don't think we acknowledge how judgemental we are about ourselves. past me was such an idiot, but present me I'm great
[00:20:20] What's WASM?
Andrew: Speaking of present u let's, uh, stop talking about what, what you have done. Let's talk about what you are doing. So, uh, the newest open source project you've been heading and which is the like underlying co, uh, tool for fermyon is spin. So what is spin and what prompted you and your team to build it?
Matt: So Spin is a developer tool for building serverless applications using web assembly. And I'll just kind of unpack all of that, but really with the story of why we decided this was even remotely a good idea to try and do this kind of thing. I, I mean, web assembly. So let's start with web assembly, right?
Just as a technology, uh, we've had the browser for quite a while now. Uh, one language has done a really has, it's really kind of amassed. You might have noticed there was a language that amassed kind of a following in the browser world and kind of broke out from there. Java script, something like that. I can't remember the name of it.
You know, Um, but others have failed, right? Java applets which were really kind of the first other language to be embedded inside of the browser. Uh, came and went. Uh, VB script had a brief stint in the browser. Silver light, you know, flash. We, we, we tried all of these different techniques for extending the way that we wrote software inside the browser.
And it's seemingly time and again. Uh, we, these technologies failed, right? For one reason or another, each of these technologies failed. Yet we still like the idea of actually being able to run something else inside of the browser. Uh, in must have been around 2015. Uh, the team at Mozilla, uh, kind of introduced their version of what would be a really interesting way of extending the browser to run something other than JavaScript.
And rather than try and tackle the sort of soup to nuts story that Applets and Silver Light and all of these did where it was really all about building code and then, you know, owning a piece of the UI and, and being able to paint widgetry of your preferred kind of, there web assembly really started more modestly.
Um, web assembly really started more modestly by saying, uh, you know, let's just figure the developers said, let's just figure out a way to. Execute different languages in the browser, which means really we need to build sort of a language style virtual machine of this sort that you get in Java or.net. But the, whatever the characteristics of this runtime are, it needs to, it needs to be able to do a couple of things really well. And the first one is gonna be, it's gotta be able to interact with JavaScript. It's gotta be able to interact with its environment so that we can call in and out of these binaries from JavaScript.
Uh, the second one. Uh, was that it had to have a really excellent security sandbox model because as we all know from using the, the, the internet, it is not entirely a safe place. And people try and do dastardly things with your, with your computer. And so they really wanted an even stronger security sandbox model than what you get in JavaScript.
And then they wanted kind of a binary format that was gonna be lightweight and easy to move around, fast to load, fast to start execution because we are very, very attention deprived people who wait about a hundred milliseconds before we start getting grouchy that we're looking at a white screen. And so these kind of characteristics, uh, were the driving force behind the, the original web assembly specification.
Now, Mozilla did a couple of things that were really smart here. First one is, in my opinion, they didn't try and tackle the UI layer. Right. They just said, look, we'll make it interact with JavaScript. JavaScript can do, do all the dom goodness, and, and we won't have to invent another, uh, you know, abstract windowing toolkit like Java had, or, or the whole kind of flash thing.
The second one was that they decided that this road was best, best, uh, traversed with friends. And so they enlisted the Chrome team, the safari team, the IE team at the time, and got everybody sort of in on the idea that they would standardize it under W three, just like htm, L and C Ss, and then everybody could collectively work in lockstep to, to introduce support into the browser. so that was web assembly. So let's just put web assembly on a shelf for a moment and go back to Kubernetes. Um, cause that, that's where we all wanna talk more about more yaml. Uh, so we're working in Microsoft now after Daes has been acquired, uh, working inside of the container team in Azure. And
[00:24:33] Spinning Up Something New
Matt: I had what I swear is the best job at Microsoft.
Uh, my job was to build open source software that exhibited how containers and Kubernetes were gonna solve your problem problems. And so along with Helm, we just kind of kept dropping one after another. A bunch of interesting, what we thought were interesting open source projects, things like brigade that were really more like ETL pipeline kind of thing, and cna, which is a way to package stuff.
And we would contribute these up to Linux Foundation or a C N C F, uh, whatever, whatever sort of made sense. Uh, and, and the, the way we got inspired was to go out there and talk to people and say, you know, what do you wish you could do with containers, uh, you know, at at coupons? Go out or go out to customers of Microsoft's or chat with other teams inside of Microsoft.
when you ask questions like that, you're gonna get some good ideas, but sometimes you're not gonna get good ideas. You're just gonna have problems you can't figure out how to solve. And we started to amass a number of those problems. Um, one of 'em was that we really wanted to figure out a way to sorta instantly scale up and down.
So when you think about Kubernetes, right? I deploy my little application into Kubernetes, and in order to make sure that this application is always gonna be available, I wanna have, you know, three or five or seven or nine replicas running somewhere in my Kubernetes cluster so that if a not or two or seven die, I still have a version of my application or so that if, you know, suddenly we trend on Hacker News and we get tens of thousands of requests coming in all at once, I don't have to start denying service to some of them, right?
Things are just pre scaled so that I can handle the load. Uh, and this solves really kind of two big problems, right? The outage problem and the scalability problem. Uh, but it's wasteful. And that's a problem for me as a consumer because it means I'm paying for three or five or seven or nine instances of my app running instead of paying for one instance of my app running.
Or at two in the morning when nobody's visiting my app. I'm still paying for three or five or seven or nine of these things, even when nobody is looking at or using my app. Uh, but so, so what we wanted to be able to do was figure out a way where we could scale down, uh, ideally to zero. Uh, when the app is not being used, nothing's running.
When traffic start to starts to come in, I start up an instance. If a node dies, I start up another instance somewhere else, but I'm not running all of these things all of the time. So we tried various ways to do this with containers and with virtual machines, and we were just kind of drawing a blank. Uh, we just couldn't find a way to make containers start fast enough.
That they could do the work we wanted them to do without making the sacrifice that in order to start a container that fast, there was nothing worth having inside of the container. Right? So really, we were looking at seconds to start a container when, as we talked about a couple of minutes ago, right?
Your users got an attention span of a hundred milliseconds. So they're not gonna wait for five seconds or 12 seconds or whatever for your container to start up when they come in and, and you're, and you're in the scale up motion. So that was kind of the first catalyst, and there were several others. We were frustrated by the fact that docker containers were not cross architecture and cross platform, and especially as arm processors got more popular in the data center, were like, here we are at Microsoft, telling people, okay, well if you want Windows container support and Linux container support and intel support and ARM support, then here's what you do.
You build yourself a matrix, right? You're compiling and building these things for all. Oh, and you're gonna need different docker files because. Armed libraries are slightly different than Intel libraries on Linux. And, and, and, and, you know, windows obviously different. And this was a really frustrating story to try and tell, right?
It was obviously frustrating to everybody, right? When you think about it in the con, in the contemporary platform engineering world, right? Uh, platform engineers are going, well, if only we had an armed version of this, we could run it a lot cheaper. Well, that means we have to throw it back over the wall to the developers who are like, are you kidding me?
Now I have to build a container on an architecture I don't even have anywhere in. And, and you're just introducing friction all along, both the development pipeline and the social pipeline inside of an organization. So we're looking at problems like that scale to zero, uh, cross platform, cross architecture, uh, you know, performance, some of these kinds of things, and just making no headway in the container ecosystem.
Justin: What was the timeframe on this?
Matt: Oh, this was 2018 was when we were really kind of working hard on these problems. Um, and we all got together for, we worked as a distributed team, um, that we were DAAs Labs when we were at Microsoft. It was about half of the Beas engineers became Dais Labs, and then we just kind of kept pulling in people that we liked to work with and, and growing the team that way.
Um, so in 2018 we were up in Vancouver, up on Vancouver Island, in fact, uh, to, for an offsite meeting. And we had finished a day of planning and we'd finished dinner and we were just kind of sitting around chilling after this, venting about. Huh The many issues we were gonna have to solve in helm. Right?
Things like that. Um, and somehow the conversation sort of turned to these problems that were, that were feeling intractable and we're in. You know, you get that moment where you click into kind of the creative mode and you're not thinking about problem solving in the sense of applying existing patterns.
But you're, you're, you're starting, you start going, well, what if, you know, just throwing out an idea here, what if there were some other cloud runtime in addition to virtual machines and containers that we could draw from, what would that runtime have to look like? Well, it would have to have a really good security model, cuz that's just table stakes for the cloud.
You isolation is what makes compute the kind of thing it is. It'd also have to be really high performance cuz we know we need to be able to start things up in under a hundred milliseconds. It al also have to, you know, be able to load really quickly. It'd have to be cross-platform, cross architecture and, you know, this is, this is how we'd wanna be able to use it.
And we're kind of talking through that. And then, Started going, you know, we should look into this. Maybe there's some technology out. Maybe we don't have to invent this. Maybe there's a technology out there that'll accomplish this for us. We had research Friday time, which was basically each person could kind of do their own research on Friday.
Uh, but several of us started looking into technologies, web assembly being really one of the first ones that we looked at. And I'm going, well, do any of these fit that Bill? You know, high performance cross platform scale scalability. And over and over, you know, we tried all these wacky experiments with web assembly and it, each time it seemed to be working out and we, we did some things that were, were false starts and went, okay, well that's not the right way to use this technology.
But we really realized that we could kind of pluck web assembly out of the browser, drop it into a cloud environment, and all those virtues that web assembly had in the browser, the strong security sandbox, the easy ability to call in and out of the binary, the compact binary format that was portable across architectures and operating systems.
Those were exactly the things that we needed to make at cloud runtime work for us. So, uh, and we discovered that there were indeed some. engineers at Mozilla, some engineers at Fastly who were all working on kind of similar stuff, and we pooled resources together. Um, they were building a runtime called Wam Time.
That was a, a great, you know, standards compliant, really kind of cutting edge, staying along with the standards as they emerged. Um, and it was a great web assembly of runtime for us to start with. And we began kind of building experiments. Um, at this point we started going, all right, um, you know, we're, we're well out of our, our Microsoft mandate of building cool container technologies for Kubernetes.
Um, and, and we decided it was time to kind of pack up, try the startup thing, and 10 of us on the same day left Microsoft started fermyon and said, okay, we've got the kernel of an idea. Web assembly can be, uh, the technology for this kind of next wave of cloud computing that'll sit adjacent to containers and virtual machine. and we started fresh and said, all right, how do we envision this working? Uh, and we started building, uh, the first thing we decided to do was, uh, as one does say, all right, we, we wanna start by proving that we can build something production grade in Kubernetes, I'm sorry, Kubernetes in web assembly. That trauma's gonna live with me for a long time.
Uh, build something in web assembly that we can run in production on the server and really start to exhibit this. So we built kind of with toothpicks and, and marshmallows kind of built our first version of the runtime. Uh, and then we built, because this is what we do, we built a content management system in Rust that compiled a web assembly that could run in our little first version of this runtime.
And it worked. And in February of 2022, we launched ion.com running in a web assembly based cms. On the, on the cloud. We use Nomad instead of Kubernetes nomads and Orchestrator from, uh, from Hashi Corp that we just really like. It's a process-based orchestrator. It's really great to use. Um, And then that that first kind of toothpicks and marshmallow runtime became spin a couple of months later.
And spin was our attempt to say, okay, we can build a generic developer tool, uh, that, that will make it easy for, for the developer. Our, our core user story, all of 2022, our core user story was, as a developer, I should be able to go from a blinking cursor to a deployed application into a, a deployed serverless web assembly application in two minutes or less, right?
We just wanted to build a tool that would make it so easy for people to try out writing these kind of web assembly on the server side applications that at best they had to invest, you know, a a few minutes of their time. I mean, realistically when you're getting started, everything's a little bit harder and it probably does take around a half hour, but it was like, uh, actually you can get the hello world running by, downloading, spin, starting everything up, and you can actually have it running in two minutes or less.
Um, and. That for us was just a really important thing to do. Spin is open source and we, and, and so we started layering on additional things. Microsoft, uh, the, the same people that we kind of, you know, left behind Dun came back and said, oh, we'll help with this. And, and we started getting more external contributors.
Uh, the GitHub star count started going up and we're like, whoa, what's going on here? Right. Uh, we, we didn't think anybody would even notice us yet. Um, and then, you know, over the last, well it's been just about a year now, we've seen spin really take off and grow very rapidly. Um, and, and we've added from, I think our first languages we've supported were just rust and go, if I remember correctly.
And we figured out a way to. Do first class support for, you know, kind of top shelf support with SDKs and everything. For Rust Go Go type script. JavaScript. Now Python, we're working on java.net. the.net team really did most of the heavy lifting for.net support. And then we're just kind of working our way through Red Monk's top 20 languages and moving as many over.
But we also figured by using something that's basically CGI made new again, uh, cgi. For those of you who are much younger than I am, was at, was, was the world's first attempt at building, you know, dynamic website backends, right? The way I wrote it was a pearl script and the web server, you know, would get a request and it would shell out to a pearl script and execute that pearl script.
It was definitely as safe as it sounds. Uh, and then whatever the Pearl script spit back out, it would pipe right back to the web browser. And we were looking at web assembly and going, actually, if we just support that very basic model, then, then most of the web assembly languages, any of them that supported the, the, the standard called the web assembly system interface, most of those languages would be instantly supportable when side of spin.
So from that, we kind of instantly got, uh, support for, uh, for Swift and for now like prologue and all, you know, sort of some, uh, exotic languages like that, new languages like grain, that's a web assembly specific language. Uh, and so it's been exciting to kind of see that we can give. Um, you know, a, a, a decent experience for a really broad swath of languages, and then kind of a top shelf experience for the languages where we have the, where we know there are enough developers that we can build a good SDK and, and give them, you know, start building in things like database support and key value storage support and, and things like that.
So it's been, it's been a really fun year as we've kind of seen this little project go from the, from, from, from a glint in the eye to something that people are using and, and finding enjoyment in.
[00:36:37] Making WASM Work
Justin: Yeah, that's, that's a both awesome context and really interesting story. So, my understanding, I, I haven't really kept as up to date on WASM as I probably should. Um, the sort of first iterations of that didn't have a good story around like garbage collection, like garbage collected languages. So you really had to use like, You know, compiled languages without a garbage collector doing man, manual memory management.
Is that solved within Wasm or do you have to like sort of like build a garbage collector for the languages that need it and sort of like layer that in,
Matt: Yeah, this
Justin: or is my understanding of that incorrect?
Matt: I need, I need like the, uh, verbal equivalent of an asterisk, um, oh, it's solved asterisk. Um, so this, this has been one of the fascinating issues of trying to write a general purpose, um, language run time, right? Where anything from rust to Python to BF to JavaScript should be able to execute.
Inside of this thing is that, uh, languages aren't really built typically with the intent that they will fit a generic model, right? It's going back to the beginning of the conversation, right? We're talking about how. Languages are really like a reflection of the way we think about the world. And that's true when we're talking about, you know, using a functional programming language versus an object-oriented one.
But the pe, the very people who design these languages are designing a new language because they want to express a different way, the way they think is, is the best way of describing these kinds of abstract systems. And so, of course, memory management in Rust, which is, is, is, uh, I, I know you had Steve Klabnik on a while back.
Right? The Rust Memory Manager is amazing and very, very, very different from the Java Memory Manager, which is different in turn from the way JavaScript does memory management and Python and, um, . So the early, the, the first version of the web assembly specification just didn't deal with it, right? Said what every host language runtime will need to provide its own collection or memory management.
And of course, rust does this very elegantly and was one of the first languages that could easily be implemented this way. Uh, this sort of YOLO style of sea was made it such that c was fairly easy to support. Um, but then you start to hit more complicated languages, uh, more complicated compiled ones like go.
Uh, and, and you know, if you use Tiny Go, which is a great, uh, it was a project originally built for embedded Go. Uh, but they really have embraced the kind of web assembly ecosystem, but they actually allow you to choose different memory management techniques, including shutoff memory management altogether.
And you can compile all these into the web assembly runtime and have a good time. Um, scripting languages are a whole d. Ball wax though. And so you get really kind of, and, and so are the Java languages, right? So you really have three classes of languages, those that compile all the way down to a binary format and have a kind of slim run time.
Those where the run time is the scripting interpreter. And then you just feed it in the script files and it executes. And those that are kind of the hybrid bite code ones like Java or like the, anything that runs in the CLR and.net and each of those kind of brings its unique challenges when you're trying to do the one size fits all onto the web assembly, run time.
So along the way, uh, a lot of the web assembly standards, um, uh, contributors began working on a. Standard memory management extension to web assembly that would make it so that make it just a little bit easier so that instead of each language having to implement their memory management inside of web assembly, the runtime itself could do the memory management and language could hook in.
That specification is not finished, nor is it really entirely broadly supported yet. Uh, but it al already seems to be picking up traction. I know the Kotlin team in particular wants to do a Kotlin directly to web assembly, compile, run, and they've been, they've really been pushing the new, uh, memory management spec, uh, but still like, uh, within the last few months, um, Python, Ruby, uh, java.net.
All of those have had web assembly implementations, none of which use the garbage collector specification, each of which, which again, is not widely implemented, each of which kind of brings its own memory manager. And as it's turning out, uh, you know, we're all kind of waiting to see which, which technique actually turns out to be more efficient.
Because when you use a generic memory manager, then there's some amount of contorting your language to fit the, the abstraction that that provides you in. Maybe that'll be hyper efficient or maybe it'll be clumsy. We're, we're not entirely sure. We haven't seen a whole lot of languages support it yet. On the flip side of, you know, if you're just compiling your runtime into web assembly and you've already got.
memory management strategy in the runtime then, you know, I don't, to my knowledge, the python, neither the Python nor the Ruby teams did any major alteration of their memory management, uh, in order to compile the runtime into WebAssembly. So we'll kind of see, and I think that's, uh, that's sort of the, um, the microcosm of the bigger story with web assembly in that, uh, you know, we've got a generic run time that seems to be really well designed and, and reasonably flexible.
We're just now starting to see the, the momentum build to the point where, uh, some of us are going are, are kind of, you know, sitting there with fingers crossed going, I hope all these design assumptions prove true. Because I got a lot invested in this now, and you know, so far it's been, it's been, I I wouldn't say smooth sailing.
It's been good sailing, right? Uh, challenges come up, they get solved. Uh, rough drafts get replaced by, second, drafts get replaced by production quality code, and things keep looking better and better. Uh, it's an exciting time, and maybe this is why we get myopic about how, how enthusiastic we are about today's technology, while less enthusiastic about yesterday's technology, it's exciting to watch this stuff sort of unfurl in front of your eyes, right?
And to know that the thing that you, you bang your head against the keyboard. when you ran into it a month ago, is now totally well supported and, and, and easy to do and trivial. And how come it wasn't this easy all the time? And then the things, I'm banging my head a bit against the keyboard today. I'm like, I really have to wait another month or two.
we're so impatient. Uh, but it's been fun. It's, it's a lot of fun to, to watch this ecosystem develop, develop rapidly. Uh, one of the cool things I think about web assembly, As as a general technology observation, right? We see technologies that are invented for a niche and then they, and they stay in the niche for their entire life cycle.
We see some that are generic languages that then later find their niche. I think Ruby is a good example of that, that was invented to be sort of like a system scripting language. And I think it was, must have been five, 10 years old before Rails came along and suddenly Ruby had a reason for existence and then everybody was doing Rails.
And then there, there are instances like the web itself, right? Or, or, or Java or um, uh, html, which was originally built to transmit physics papers, right? Where suddenly we, we look at a technology that was designed to do a very small thing and realize that it's general, that, that it's solved for, for a generic enough use case or for a generic enough scenario that there are actually a whole bunch of use cases.
And I feel like web assembly. in the last maybe 12 to 18 months, is experiencing that kind of collective realization as we all look at this and go, oh, you know, Fermyon figured out a way to pull this out of the browser and drop it in the cloud. Oh, somebody else fi you know, single store, figured out how to pull this out of the browser and drop it into their database so that they can run stored procedures that are compiled to web assembly next to the data instead of selecting it all out munging it, and then shoving it back in.
And we're all kind of discovering these interesting applications where web assembly, uh, gives us capabilities we didn't have before. And, uh, and, and yet at the same time, we're all kind of looking at this going. Uh, you know, Fermyon, we're going, we think we have the rails moment for web assembly. We'd love this serverless application model that we've developed.
And meanwhile, you know, single stores going, we've found the aha moment for web assembly. It's gonna be functions in the database. And, you know, uh, EXM is going, we've, we've, we're writing the last plug-in system you'll ever need to use, and it's written in web assembly. So we're all kind of experimenting, meeting with success here and there, hitting challenges, but also going, what does the future of web assembly really look like?
Is this gonna be a really broad technology? Is it going to find one niche? And is that niche gonna surprisingly for us all, not be anything to do with the browser? Uh, so it's fun because things are happening at a quick pace, but it's also fun because we're watching this sort of, uh, conceptual explosion of all the different things that can be done with this technology.
And we're watching everybody try and find out is this the right fit to solve this problem that I've had for a long time?
Justin: It's interesting. One of the topics we've talked a lot about on the, the podcast is, is incidental complexity when we're like developing applications and you know, I think like Kubernetes was this answer for this like, Complex problem of like orchestrating and deploying workloads. But people have like, wanted even more simplicity is like, look, I, I, I don't want to think about this as much.
I just want to think about code. And I think the, the sort of like fun functions as a service sort of platform movement at aws, Lambda, CloudFlare workers, that sort of thing, really honed in people's, like, we could have an, an even simpler developer experience where we sort of like commoditize these resources even further.
And I think that it's really interesting that wa Wasm is this more sort of general purpose target at this. So it's, it's a, it's cool to see it evolve. It's really interesting.
Matt: And when you, when you bring up serverless functions that, uh, you know, like Lambda or Azure functions or you know, CloudFlare workers or things like that, that is, that is a case where I think web assembly has a real sweet spot that meshes well there. Um, and, and part of that is because, you know, serverless is an idea, right?
Talk about serverless functions is an idea, right? It's, it's a way that we introduce elegance to an area where previously there was complexity. Instead of Maya having to write, uh, a full HTTP server with TLS and process management and multi-threading and all of that, in order to handle HTTP requests, I just write a function and tell the, the environment how to call my function and execute it.
And then when the call comes in, my function starts up is given the request, processes it to completion, spits back out of response, and shuts down. Right. Now, earlier I was talking about how one of the things we were after was scale to zero and web assembly provided scale to zero. uh, functions of service is a, is a quintessential scale to zero architecture, ideally, right in the abstract.
It should be a scale to zero architecture. Uh, the, the way that we got to the first wave of serverless functions, though, was a little bit clumsy because we didn't have, you know, Lambo was built without web assembly. Lambda's built on top of a virtual machine architecture, which as we talked about earlier, is really the slowest of the cloud compute, uh, primitives, right?
It's the, it's the one that, uh, a decent sized virtual machine can take minutes to stand up. And so there's all this optimization behind the scenes on platforms like Azure functions and, and, and AWS Lambda and Google functions that are designed to sort of prewarm things and get them all queued up and be ready.
And every time you start using words, Queuing things up and pre-war, uh, we hear that. We hear in the back of our heads, oh, waste, waste. Right? More, more electricity is being consumed, computers are sitting idle when they could be doing something, uh, you know, somebody's engineers are having to run those things and the rest of us are having to pay for those things. The typical lambda function, still, even, even with all these optimizations, right, still takes around 200 to 500 milliseconds to coldstart for most of the tier one, uh, aw AWS languages, right? As we already established, 100 milliseconds is when your user starts losing interest, right? So already we're outside of the bounds of where the user of the user's natural tolerance.
And so for that reason, I think Lambda hasn't really sort of, or any of these serverless functions. Things haven't really quite hit stride with being able to be full on web backend platforms, right? We've managed to couple together things such that it's satisfactory, but cobbling doesn't feel nearly as nice as, you know, kind of.
Uh, finally crafting, right? And I think Web Assembly is providing a very interesting avenue for us to start building beyond that first version of serverless functions into one. That's, that our, our web assembly, uh, startup times right now are about, uh, two milliseconds and under really, our average one is under one millisecond, uh, to cold start before it's executing user supplied instructions.
And that's the kind of profile you want. If you wanna have a page that serves out and gets a 99 to a hundred percent on the Google page speed rank, right, and scores well on seo. And, and that's exciting. Uh, you know, web Assembly's got something to offer here that we can do in addition to, you know, these other, the security, the, the, the package size, um, and that kind of thing.
So I think that serverless applications, right, serverless function, Are are one of those places where we, at fermyon, you know, building tools like Spin can really make some vast improvements and say, look what, here's, here's a promising first step for why web assembly is this kind of third wave of cloud computing that's gonna be able to speed up the, the performance, speed up the development process, support a wide variety of languages.
And then as web, web assembly itself matures and we get things like the component model, uh, we're gonna be able to build very interesting and sophisticated applications using, uh, using web assembly for all it's worth. Right. And one of the things that's exciting to us is right now you can build a web assembly serverless application using whatever languages you want.
[00:50:25] Fermyon: The Platform
Andrew: that's a great example of the technology. Um, before we move on to tooltip, uh, let's talk about fermyon a little bit. Uh, talking about hosted platforms, you built a hosted platform, so, uh, why would you use Fermyon and what is it?
Matt: I mean, doesn't everybody just build a hosted platform? Isn't that what we're doing until, until the world is full of No, we, so we wanted to build, you know, SPIN is this developer tool to show people how to build web assembly applications. We, we really wanted to build a place that could execute these and show, and, and we wanted to do it in such a way that it would be.
Trivially easy for people to try this out on their own. So we've got an open source, you know, full packaged up, nomad console, vault, uh, spin, all of that kind of stuff where you can install it into whatever your cloud provider or bare metal is. But that's, that's a time consuming process. It's gonna take you a while to get it set up, then you're gonna have to maintain it thereafter.
We really wanted to provide developers with that experience where they could try it out nearly instantly. Right. Again, going back to that core user story, we wanted to go from blinking cursor to deployed application in two minutes or less. And so, uh, 66 seconds or less is what we are now down to. Um, so you know, you type in a command spin new, it scaffolds out your project.
You can do spin build. It builds your web assembly module. Then you do spin deploy and it pushes it up into fermyon cloud. and then you, you get your domain name back and you can start playing around with it. Uh, you know, and that's essentially, that's where we are now, right? It's a free platform. It's open beta eventually.
We'll, and, and it's kind of rate limited at this point because, you know, when you get a thousand, 2000 applications on there, you do kind of have to watch the budget. But I think we'll kind of evolve this into, uh, into the kind of thing where a mion platform is scratching the itch you have then, you know, for a few bucks a month, you can upgrade your plan, uh, run all the, all the finicky whiskers and slats games that you want to build whatever you want to as a serverless application.
And if it's really more your thing that you wanna manage your own platform, there's still, you know, spin in fermyon platform, all open source that you can kind of do your own thing with.
Andrew: Yeah, I think it's commendable that you have all of the, the, the tech essentially open source. So like if you want to go host this yourself, you can. Uh, I think that's su super cool.
Matt: I think that was actually one of the things we learned to do well at Microsoft of all places. Uh, you know, when you, when you think about Helm, right? We never attempted to monetize Helm at Microsoft. Uh, the, the, the only goal of Helm ever was to make it easier for people to get the job done that they cared about, which in Helm's, you know, purview was packaging up that yaml and making it possible to reuse the same packages to install your software on as many different Kubernetes instances as you ever wanted to.
But what we observed there was when you give people a tool like that, that makes it easier for them to build the things they care about, then they build more stuff using your platform. And, you know, AK s was the fastest growing, uh, service offering in all AKs being Azure Kubernetes service, right? Fastest offering in all of the Azure catalog, right?
There are hundreds of different application types and services and Azure and AKs was, was one of the fastest growing. And the reason why I, I'm. Utterly convinced is because we as an ecosystem provided a whole bunch of open source tools that made it compelling. It was easy to move. First, my little application, then my databases, and then the monitoring stack and all of that into Kubernetes using tools like Helm.
Definitely not exclusive to Helm, but it was one of the enabling tools. So as we looked at fermyon, we said, you know, we we're all a bunch of open source developers anyway, but we really like to apply that learning where we really want ION Cloud to be the easiest, best place for people to run spin applications. but they're not all gonna find their needs met by that, right? Some people, for regulatory reasons, need to run things on their own hardware. Some people will wanna run it on a skiff or on-prem or uh, or in digital ocean instead of us, and we want them to be able to do it. And, and spin is, we are successful at fermyon.
If we can really articulate that vision and people say, oh yeah, this vision is great. This vision is really compelling, I love it. Right? Um, and, and you know, we do want to be able to pay everybody's paychecks eventually. Uh, but we understand that, you know, in a sense, all of us as software developers are actually in it for something a little bit bigger than that, right?
We really want to go in all the way back to the beginning, right? We're passionate about trying to show people, look, I've, I've constructed this mental model of that. I think if you use it, it's gonna make your life easier. It's gonna make, uh, your day a little less stressful. Your weekends a little less full of pager interruptions.
Um, And, and that, you know, that is really the kind of driving thing behind us as software engineers at at fermyon . We want to give people that experience. What we found that we love about Web Assembly, we want other people to be able to try it and then use it the way they want. And we hope that Fermyon Cloud ends up being the kind of thing that people want to pay for.
Um, but if they don't, there's spin. There's Fermyon platform and they're part of this big community that's gonna keep pushing this technology forward.
Andrew: Great interview so far. It's been a pleasure having you on. Uh, yeah, . Um, with that, uh, we'll move on to Tooltip.
Want to hear the full interview in all of our tooltip. Head over to Patreon and subscribe. Awesome. Well that wraps it up for Tooltips. Uh, Matt, thanks for coming on. Uh, neither Justin or I are the biggest backend or Kubernetes guys, but I think you explained everything in a very concise and very interesting way. Uh, I'm, I'm really excited for this episode to come out. It was a lot of fun talking to you.
Matt: I feel like, again, I should apologize for everything I've, for all the bad software I've created, but also, you know, as we talked about, maybe I'm just being hard on myself.
Justin: Yeah, I think it.
Matt: on past me.
Justin: It's one of those things where we, we have to have people to pave the path to the better future. And that, that creates, you know, you just gotta build a lot of tools to get there. And, you know, we both appreciate the work that you've done. Uh, I have experienced the joys of Helm personally, so, you know, like not having to rebuild a lot of things.
I appreciate the work that you've done. Uh, and yeah, this is, this is such a fun episode. Uh, really appreciate you coming on.
Matt: me. Yeah, this was a lot of fun.