#116 August 12, 2020

Independent Open Source, with Alex Ellis

Hosts: Craig Box, Adam Glick

Alex Ellis created serverless framework OpenFaaS while working a day job. It’s used by some big companies, but he’s resisted the temptation to join one. Instead, he’s offering consulting and seeking sponsorships, building a business from the ground up. He explains the pros and cons of independence to Craig and Adam.

Do you have something cool to share? Some questions? Let us know:

News of the week

CRAIG BOX: Hi, and welcome to the Kubernetes Podcast from Google. I'm Craig Box.

ADAM GLICK: And I'm Adam Glick.

[THEME MUSIC]

CRAIG BOX: Here at the "Kubernetes Podcast," we like to ask the important questions, and I've been asked to ask you this week, why did the chicken cross the road?

ADAM GLICK: [LAUGHS] I think I found the answer to that this week. I was out for a walk with the family, and here in Seattle, there are a number of people that actually keep chickens as pets. Roosters are not allowed, but chickens are. And we came across a group of chickens that were actually crossing the road on our walk. And apparently, the answer is, because the grass in the neighbor's yard appears to be tastier.

CRAIG BOX: Greener as well, I'm sure.

ADAM GLICK: [LAUGHS] Certainly longer. Because I think it took them a while to get to it. But they came over, they nibbled on it, and then they went back over to their coop. It was neat to see.

CRAIG BOX: Did they stop traffic?

ADAM GLICK: It was not a very busy street, so there was no traffic to be stopped. Otherwise, if I would have been faster on my phone, I would have taken a picture of them just to, actually, accent this discussion a little bit more.

CRAIG BOX: Many times, on major New Zealand highways, have I been held up by farmers moving cattle or sheep between paddocks.

ADAM GLICK: Speaking of tasty food things, I've been continuing my work on trying to find fun and new and exciting things to make at home to eat during COVID times. And this week, we went for loaded nachos and created a giant tray full of nachos with cheese and ground beef. And it was almost like I'm back in a bar doing trivia. It was fantastic.

CRAIG BOX: There is one thing that has come to my attention that I do quite miss recently, and that's other people making food for me.

ADAM GLICK: [LAUGHS]

CRAIG BOX: There's just something to it. It's a thing. We used to have it. We may have it again one day.

ADAM GLICK: Shall we get to the news?

CRAIG BOX: Let's get to the news.

[MUSIC PLAYING]

ADAM GLICK: Microsoft looks to be trying to capitalize on some of the recent noise in the service mesh space by launching Open Service Mesh, or OSM. OSM embraces and extends the Service Mesh Interface spec that Microsoft released last year, while simultaneously suggesting that SMI itself might not be enough by offering an escape hatch to the Envoy XDS APIs.

Most of Microsoft's statements with the announcement are aspirational. They had developed the project behind closed doors but are hoping it will be community-led, effortless for operators, and immediately accepted into the CNCF. The day-old project did spark controversy after some eagle-eyed observers noticed that they had appeared to have copied some of the source code from linkerd, a competing service mesh project, without providing any attribution as required in linkerd's license. Microsoft has responded that they are removing the code in question.

CRAIG BOX: Because there is clearly no such thing as too many service meshes, Kong has announced Kong Mesh, an enterprise version of their Kuma open source project. The clear winner in all of this is Envoy, which can truly lay claim to being the universal data plane, as the engine used by effectively all but one service mesh product.

ADAM GLICK: VMware has released version 2.10 of their Tanzu application service as generally available. The service was formerly known as Pivotal Application Service and brings a long list of updates, including a simplified CLI, better auto-scaling, updates to the Spring Cloud Gateway for API routing, a new operations manager that supports multiple versions, which is designed to simplify upgrades, easier certificate rotation, and a faster web UI.

CRAIG BOX: Kubernetic has released KubeCarrier, with a K and a C, in that order. The new project is designed to help automate the full lifecycle of anything that is API-driven using Kubernetes operators. It creates a central repository for managing operators by running a multi-cloud, multi-cluster hub for service management. If you want to know more about Kubernetic, check out episode 109 with the CEO, Sebastian Scherer.

ADAM GLICK: Bay Area company NestyBox, a startup that has been working on containers within containers, has open-sourced sysbox, the engine behind their software. Sysbox is an OCI runtime that lets you run entire virtual servers that can themselves run Docker or other application containers. They pitch these system containers as lightweight VM equivalents, more secure than a privileged container, but more convenient than a VM.

CRAIG BOX: At the Virtual Blackhat Security Conference last week, researchers at Palo Alto Networks' Unit 42-- named for the answer to life, the universe, and everything, and not for the Belgian Netflix show-- demonstrated a Kata container escape. The team showed multiple ways of escaping the Kata VM and compromising the host machine. Remediation for these techniques have been implemented with the help of the Kata community. So if you or your provider are using Kata, be sure to update to the latest version.

ADAM GLICK: Jenkins X has gotten a new open source UI implemented as a plugin for VMWare's Octant. The new UI allows users to view applications, environments, pipelines, and repositories, as well as quickly navigate to pod, log, pull request, or preview environments. For each step, you can view the details or log and the various jobs and pipelines used to operate Jenkins X itself. Over time, the creators plan to add management UI capabilities for installing, upgrading, and administering Jenkins X. If you're curious to see the Octant JX UI in action, the link in the show notes has an embedded demo video.

CRAIG BOX: Banzai Cloud has announced that the Backyard Service Mesh of their hosted Istio platform is now FIPS-compliant. To achieve this, they had to recompile Istio and Envoy with certified SSL libraries and Go versions and configure it to reject insecure encryption cipher suites. The blog post announcing it talks through what had to happen and links to an upstream Istio issue. Backyards joins Anthos Service Mesh, Google's hosted Istio-based mesh, in the FIPS club.

ADAM GLICK: OpenStack hosts an edge computing platform inexplicably named StarlingX, or, as I will read it, "Starlinx." This project, as with so many projects, is now a managed Kubernetes platform, and has reached version 4.0 this week. Support for the newly fixed Kata containers is a headline feature, as are other security enhancements. Additionally StarlingX was recently promoted to a top-level project in the OpenStack foundation.

CRAIG BOX: Sounds a bit like Space Golf. Two new features in Azure Kubernetes Service this week. Your containers are stateless, so your Node OS may as well be too. You can set your nodes to use ephemeral disks, which are slightly faster by way of not replicating the boot volume to Azure storage. Azure Portal also now provides a Kubernetes object resource view similar to the one available in the Google Cloud console for GKE.

ADAM GLICK: The CNCF has announced that etcd 3.4 has passed its third-party security audit. No major issues were found with the core components of etcd, though one high severity issue was found in the etcd gateway. This issue has been fixed, as well as having been back-ported to version 3.3.

The maintainers are now aiming to graduate the project in the CNCF, as passing the audit is a requirement for entering the graduated phase. For those interested to read the audit report themselves, a copy of it has been posted into the etcd community repo. If you want to know more about etcd itself, check out episode 95 with the project founder and maintainer, Xiang Li.

CRAIG BOX: Finally, a project the size of Kubernetes needs a team looking out for its code of conduct, and such a team was bootstrapped two years ago. Three members have finished their two-year term, and the Kubernetes Steering Committee elected three new members-- Celeste Horgan, Karen Chu, and Tim Pepper. Thank you to the members rolling off for getting the Code of Conduct Committee underway, and congratulations to the newly elected.

ADAM GLICK: And that's the news.

[MUSIC PLAYING]

CRAIG BOX: Alex Ellis is a consultant on serverless and cloud-native computing. He founded OpenFaaS, an open source serverless project, and has built a community via writing, speaking, and extensive personal engagement. Welcome to the show, Alex.

ALEX ELLIS: Thanks. It's great to be here.

CRAIG BOX: For someone who's now predominately working as an independent consultant, your career started with almost 10 years of what I would call, quote, "big software," working for a payroll company, ADP. What was your experience like? How did that come about?

ALEX ELLIS: I went to university in London and graduated around 2006, then thought I'd get a job near Peterborough where I lived and where I'd grown up. And actually, my first job ended up being back in London again, commuting into a small consultancy-- doing lots of C# and .NET, working on websites for companies building homes, things like that.

And then they relocated to Leamington Spa, and I needed to find another job. And having looked around, someone said, have you heard of ADP? They've got an office in Peterborough. And I applied, got the job, and started working as an application developer for them on their payroll product.

ADAM GLICK: How did you get involved with Docker?

ALEX ELLIS: Well, during my time at ADP, one thing that I noticed-- well, as it was quite early my career, maybe two, three years in, is that sometimes, what you're told in an interview isn't actually what happens when you get your feet under a desk.

CRAIG BOX: No!

ALEX ELLIS: And say it ain't true! The first job that I'd had, we'd hired an agile coach who came in and taught everyone to do TDD and index cards. And back then, it was Extreme Programming instead of Scrum. And I got in there, and they're like, yeah, we do all that. And within about two weeks, I was reprimanded by HR for writing unit tests for the code I was writing. And I just absolutely couldn't believe it. And that was my first experience of enterprise.

ADAM GLICK: [LAUGHS]

ALEX ELLIS: Now, by the time I left-- and there was a tweet about this recently. Why do people stay for 10 years-plus? And I reply, it's a trap. You think you can improve things, and you'll get made promises, and you think you can change and improve.

You think you sort of have something, an ax to grind, to some extent. And I guess it was true. But also, I was worried about whether it would work out if I left.

ADAM GLICK: Mm-hmm.

ALEX ELLIS: And I know people there-- one of my friends who is still there 15 years later, he would do so well at a company like a Google or a small startup, no doubt. But he just feels so petrified of making that move. And for me, I just put it off, and off, and off.

And actually, that gave me the time to really go and do my hobbies. Like, I just loved cycling I'd go home on my bike and ride 40 miles home, go the long way. And then, in evenings, I would look into technology. And I'd heard about Docker. I didn't know what it was. I didn't know what the difference was between that and a virtual machine.

And I tried it out. And then I started to think, well, how can I get this, like I'd done with the test-driven development and the unit testing? I had that sort of thing to prove. I felt like the underdog, and I wanted people to see the value in this technology.

And so I tried to relate it to the business problems.

ADAM GLICK: What did HR have to say about that?

ALEX ELLIS: Well, by that point, I had settled down. And I was toeing the line, and stood my place, and one thing and another, and kept my nose clean. But when I was a little bit older-- obviously, 10 years is a long time, and I think it can be very formative years when you join a company-- and so yeah, I'd learned a lot about fitting in and working in an enterprise company, about going through all the processes and politics to make sure that you get buy-in for things.

And actually, by the time I left, I'd got to the point where I could have a open conversation with the VP of development about an idea I had. If he saw something in it, I might put together a five-, 10-minute presentation for him, then go off and do a prototype on it, and get other people involved in it. And that's something that I'd seen and done in open source and brought in there.

One of the tools we created generated Jenkins jobs, which was very tedious, and we couldn't use any of the plugins. So I used some Go and some YAML, and got someone else involved on it early on, and this is something that you just wouldn't have seen anyone else doing there. We built the project out, we got one or two teams onboarded on it, and then other teams started coming to us and asking to use it.

Then I set up a workshop where we had 20, 30 people at a time come in to use it, and try it, and understand it. And those sorts of skills I'd had in the open source world had upgraded ADP Alex to deliver way more value.

CRAIG BOX: You were still working nights and weekends, effectively, on open source and Docker. Tell us about how you became a Docker Captain.

ALEX ELLIS: With Docker, I remember speaking to someone, one of the directors of automation testing. And they had this problem with managing all the computers you needed for a testing grid. And I'd been learning about Docker. I'd showed him about it. And he said, you should write a course about this. And I said, don't be ridiculous. You know how hard that is? That would take so much time. And he said, well, it might help people understand the value of it.

CRAIG BOX: You should have written it in Rails. It would have taken no time at all.

ALEX ELLIS: [LAUGHS] Yeah. And so I wrote the course. It was called Learn New Docker, or something like that. And the Docker team saw it, someone in their outreach program. They reached out and said, this is better than stuff we've been writing. Can we use it? Would you like to join this program we're doing?

And from there, I started to get a better understanding of almost like what you could be as a developer relations in a voluntary capacity. And so I start to understand these concepts of submitting a paper to a conference, writing your ideas in a persuasive way, and then using Docker a little bit as a springboard, helping them out when they did stuff like launch Swarm.

And so that had a real impact to me. I started going off and doing conference speaking and putting together talks, developing my public speaking skills. And then I remember, at one point, I met a lord, Lord Birt. And I was just walking in the countryside, quite like my hiking. And he said, what do you do?

And I started telling him about this conference I was going to go to. And he said, well, why are you working two jobs?

ADAM GLICK: Hmm.

ALEX ELLIS: And I just couldn't understand what he said. So, you're working a job for free. Why don't you just go and get that job that you'd rather have and stop giving these people all your time?

And I'd never really seen it that way before. And I said, but I get all these other things from it. They give me a free pass to a conference. Sometimes they pay for my travel. All I have to do is spend 20 hours a week writing content for them.

And it was at that point that I realized that these attractive brands that draw us in are just companies that need to make money. And we had a pretty good deal, one of the Docker captains. But at the end of the day, it really helped me to see things in that perspective. And I think from that point onwards, I've always respected having some perspective from people that had gone before that have got that business experience.

I think Lord Birt chaired the BBC and PayPal in Europe at one point. And having these people come in, even if they're non-technical, and give advice, and listen to their wisdom.

CRAIG BOX: Surely, Lord Birt had two jobs as well, as a member of the government, and should maybe have taken his own advice?

ALEX ELLIS: [LAUGHS]

ADAM GLICK: You had this kind of epiphany moment speaking with him. What impact will that have on ADP, Alex?

ALEX ELLIS: I started to balance what I was doing a little bit more. And I went to a Docker call, and I did a little presentation there. And then I came back, and I'd seen a guy, Ben Firshman. He was part of the Docker Compose acquisition. And he was talking about serverless. He'd made a little demo program. And I got back and I thought, right, I'm going to find out about serverless.

I was googling it, and I couldn't find out the difference between serverless and FaaS. And it's like, well, which do I want? What was Ben doing? I found his projects, and I thought, well, this doesn't suit my needs. It's got all these deficiencies.

But it's a nice idea. And I had put together this program for FaaS. It's called Funker Dispatch, because it used his Project Funker. Wrote it in JavaScript, and then, at the end, threw it away because there were so many questions that I couldn't answer, and I didn't really know if it was a good design.

Then, a little bit later, I met up with Justin Cormack at Docker's office in Cambridge. And we were explaining it. And he was saying, well, I'm thinking of rewriting Funker in C. I said, well, that's going to be an awful developer experience. Think about type safety and contributions. And in the end, we came up with this idea of having watchdog.

Back then, the project was just called FaaS. Had a go with that. Very late December, 2016, it worked out to be quite usable, quite popular. But there was no tooling around it, no CLIs, nothing like that.

I then tried to bring that back to work and get people interested about it, and get them using it. The most that I could really find as a fit was a Webhook receiver for Jenkins jobs. And so it was always the thing that I was excited about but didn't really have a good fit in the company or for people there.

But yeah, the Docker Captain program, the impact it had was I think, eventually, it just got me more and more aware of what could be done even what other people were being paid. I remember finding out that in the US, developers at a similar level were being paid $150,000. And at the time, I was like, that's a lot more money than we get paid in the UK.

CRAIG BOX: But I bet none of those American developers have ever met a lord while they were out for a walk.

ALEX ELLIS: Right. Thinking about it now, that seems like a small amount of money, comparatively. But it just got my eyes opened. It got me thinking, and it got me out of Peterborough. I almost got hired by a number of Silicon Valley companies, and sort of stayed the course, carried on working on this FaaS project.

And then I went and applied for this sort of contest at DockerCon they call Hacks. And I put FaaS in for it. And they said, think again. And then I put it in again, and they said, think again. You need to push harder. And I think eventually, I just was so persistent, they said, all right, then.

And they made OpenFaaS one of the winning projects. And I got to go out there and do a closing keynote, the first one I'd ever done, 10 minutes. Very exciting. Real highlight of my career. And when I got back, ADP were like, yeah, maybe we should talk about a small pay rise for you, and I got a bit of a promotion at the time.

CRAIG BOX: What was the decision process that eventually led to you leaving ADP?

ALEX ELLIS: I remember very clearly being at the DockerCon in Europe. And it was afterparty with the Docker Captains. And Docker Captains were a group of consultants, of, maybe, misfit technologists, developer relations from different companies, a few people from Docker. And one of them, again, was Justin Cormack, who I mentioned earlier.

And he said, look, just, you need to quit your job. It's pretty obvious that this OpenFaaS thing is taking off. You need to have the time to focus on it, which, obviously, you don't have, and you'd be appreciated more. And I went home and spoke to my wife. And I said, look, Justin says I should quit my job and work on OpenFaaS. And she said, yeah, I think that will make you happy.

But there was that key thing, which was, how do you do that? How do you finance it? And I'd spoken to some VCs, and I just didn't feel like I wanted to create a venture-backed company to continue the work. A couple of companies offered me multiples of seven figures just to take OpenFaaS and have it be their thing. And then I got speaking to a team at VMWare, and they said, you could come here, keep it independent, you retain the IP of what you've built already and the brand, and we'll give you a team to work on it.

And at the time, that seemed like the best option, because from a developer's point of view, I just wanted to build the code, build the community. I wanted to see the project do well. And there was a company here that was using it as a basis of one of their enterprise products who were willing to finance it.

ADAM GLICK: How did that work? That they hired you, but they didn't own it, and you continued to work on it?

ALEX ELLIS: Well, this is something that they helped me understand. And I don't mean this in the way that they took advantage of me. They just actually helped me understand something that I didn't really understand properly. And I think this is the case with a lot of people doing open source.

I didn't own OpenFaaS. I owned the commits that I had copyright for that I'd written-- maybe 80% or 90% of it. But all of that 10% or 20% that had come from other people I didn't own. They owned it. This is distributed copyright. It's the way that open source software works.

Now, I was licensed to use it however I wanted, within reason, within the MIT license, for 100% of the code. But I didn't actually own it. And I think that was one of the problems, as well, is that there was no commercial entity other than me that had copyright in the code.

And so if I went back and I thought about this again-- and a friend had actually said this to me-- he said, well, why don't you set up a company and start allocating the IP to the company? And I said, well, that's not needed. I'll just do developer sign-off instead.

So the commits that I made whilst I was employed at VMware were copyright to VMWare. They still are, just like the commits that are made by Netflix, or the commits that are made by LivePerson or by BT. They are owned by those companies, and they always will be.

ADAM GLICK: Where within VMware were you employed? Were you part of a particular business unit? Or how did an outside open source project fit within their structure?

ALEX ELLIS: One of the questions I want to ask in the future when I look at something like this is, have you done this before? And I even ask that to some of my clients now as they reach out and say, we really need some help with community. We don't have one and we want your help. We see that you're good at that. And I say, have you ever hired anyone to help you before that wasn't a full-time employee?

And if they say no, then it might be time for me to pass on that. Because I don't want to be the first one that they learn with. We don't know how to do that. We don't know how to buy swag for a project that we don't own. We don't know how to allocate a budget and a headcount to you because it's not our thing.

And I won't go into too much detail, but think if the same, you go in, and nobody knows how anything works. You're the first time for everything. There's a lot of corporate policy and process. And you can imagine how that might be, being a company as large as Google yourselves.

ADAM GLICK: It does seem to be that they've followed that path a little bit, though, if you take a look at bringing in Heptio and some of the other pieces that they've continued down the acquisition of open source.

ALEX ELLIS: Right, because that was an acquisition of a company rather than an-- almost like a acquihire of a person that brought stuff that didn't become theirs. And so yeah, I think it's very easy for companies to understand what happens with an acquisition. You get rid of a few people. You keep the people you want. You set a budget. You find a place for them to fit in the organization.

When it comes to something that is independent of the company and will remain independent forever after, then that is just a very different thing for people to get their heads around. But some of the things that I really enjoyed at VMWare was working with some really bright people. Some of the best friends I made were in sales and in customer success, because they were the kind of people that would go, I really like what you're doing. It's inspirational. I can talk about this to customers. Joe Baguley said that he had a meeting with a very large customer in London, and he said, the reason I had that is because of what you're doing with OpenFaaS.

And so I always struggled to understand why VMware wanted to hire me and what value I was bringing. And on the surface, my basic understanding was, well, they're building a product on OpenFaaS, and it de-risks it. Also, if you think about it, it does bring street cred. This is pre-Heptio, so pre-- we own two Kubernetes founders.

And so I think every little helps. And the unit that I was in wasn't a traditional business unit. It was one that didn't, effectively, have the same kind of profit and loss. Its equity was more, what kind of impact could it create? How could it show that VMWare had goodwill, that it was a good open source citizen? How could it help the internal teams consumer open source products and software in a way that made sense? How could it de-risk?

I also learned about things like how dangerous it can be to take a particular software project off the shelf, due to its license, and use it in your product-- what implications that might have for your source code. And those are things that I just didn't know about before I joined. And so it was an enlightenment.

I learned a lot of things. I made some really good friends. And then, in the end, I think there were rumors about a service product Google was going to launch, and maybe it caused a bit of uncertainty in the whole space. The internal team that was using OpenFaaS got wound down, and then, I think, I was probably kept on just long enough to hit my cliff as a goodwill gesture.

And if it hadn't been for that, they probably would've shuttered my team earlier as well when they did the Dispatch one. But none of this is official. I don't actually know if that's the real way things went. But it seems to make sense.

And then, VMWare are very silent on the whole serverless thing. When Joe joined, he didn't really have any opinions. And then, around November, I guess, last year, VMWare went and hired the Knative team and brought them in. And that's, basically, their vision for serverless now, as far as I understand it.

But there is a product that's being built called VEBA and that is a automation platform for vSphere, the big cash cow for VMware. And that one runs on OpenFaaS, is built all around it. So IT engineers are able to write some PowerShell and have that run when a VM is provisioned. It will then go and tag a VM fod you, so you can track it in your inventory, for instance, or create a JIRA ticket.

CRAIG BOX: Just proves that anything worth doing is done by multiple different divisions of a big company in multiple different ways.

ADAM GLICK: [LAUGHS]

ALEX ELLIS: that's one of the other things that I saw, as well. There were countless projects where people were doing it in different ways, or they were all building the same thing. There were three serverless projects while I was at VMWare. And I think that will continue to be the way.

CRAIG BOX: After leaving VMWare, you started OpenFaaS, Limited. Does that company owned the trademark, for example? Does it owns the logo to OpenFaaS? What does the company have, versus the contributions you mentioned earlier?

ALEX ELLIS: This is, again, one of the things that I encountered when I was talking to companies about M&A and talking to VMWare about their options way back two or three years ago, is, what is a legal entity? And a project isn't one. But a company is, and a person is. And so between OpenFaaS Limited and myself, yes, those things are owned.

However, as I've said, most of this code is contributed either by me on my own, me at VMWare, or 290 other people and companies. So really, when it comes to ownership of an open source project, there isn't really such a thing as that, if that makes sense. And actually, I'm thinking of changing the name of OpenFaaS, Limited to something else, because it doesn't really reflect the work that the company is doing on the whole.

CRAIG BOX: I was going to ask that. And this is the vehicle that you use for your consulting, presumably, on things other than OpenFaaS, and OpenFaaS exists predominantly as this open source project. You're now going to make a change to that to bring it more in line with the company being you?

ALEX ELLIS: The company isn't me. The company is starting small and bootstrapped. My vision isn't for me to have a shell that I can put money through. My vision is to create a software company that has a great developer focus, creates tools that solve real problems, but also has a way of sustaining itself financially. And part of that, the way I see it, is, if you're going to bootstrap something, is you need to get a certain level of revenue, a certain level of success and branding out there, before you can hire people and put them on a contract.

I've subcontracted people from the community, and giving them opportunities, and we're doing a live project with a company out in Norway last year. We had a pretty big budget for it until we hit COVID. And so that's my vision, is finding people that I trust, that I think do a good job, bringing them in, and being able to do that together.

CRAIG BOX: What has OpenFaaS, Limited achieved since it started trading?

ALEX ELLIS: One of the things that was really nice was to be able to send the first invoice. It wasn't for a lot of money, but it was for something. And the client was local to the UK, and they wanted to build a managed OpenFaaS service. They'd been reaching out to me. I'd known them for about a year or so. And I went down to see them, charged them for a day, and I said, well, look, I don't think you should do that.

And I think they must have been really confused. I said, don't think you should build a managed service on the project that I've built. I think you should build one on Kubernetes, and then, if there's interest and if your customers want it, build one on OpenFaaS using that. Because by that point, OpenFaaS was effectively a wrapper on Kubernetes.

It went pretty well. They ended up using K3S and had the first managed K3S service. And I did some other content for them and helped them build the community around that and think about their strategy. I think they're doing a really good job of it.

And that was the first thing. And there's been some other firsts as well, like the first year-end, the first company party-- which was myself, the only employee at the moment, and then my plus-one.

ADAM GLICK: Picturing you with the little hat and the cake, and just like standing there. Toot!

ALEX ELLIS: So I've seen that in other companies as well I've seen people like Kinvolk that effectively started as a couple of guys. They've now gone on to be able to hire people like Vincent from-- well, his last job was at RedHat, which, I imagine, would have been a pretty big salary or equity slice to get him on board. And yet, they started small. They started with a couple of people. They did stuff around CoreOS, consulting. That's probably not what they thought they'd do with this new company. And it eventually led to what they are doing now.

Container Solutions, again, if you think about Jamie Dobson and Adrian Mowat, the two of them sort of came together, the great business mind and this technical guy. And again, they've grown a reasonably sized company. A few dozen people work for them now. But they're very well respected when it comes to consultancy. And I look at that, and I think that, actually, this is something that I want to do.

CRAIG BOX: You've said that you enjoy more the process of building something than, perhaps, being an employee of a capital-E Enterprise. Do you think that you want to do the company creation things and the things that are involved in building a company like that? Or do you think that might take you away from the things you enjoy day-to-day.

ALEX ELLIS: I think this is also related to the question of why don't you just go and get-- I think we put 250,000. It'd probably be a little bit more than that, actually-- per year, right? That is a very good question. Because when you think about things like taking out a 30-year mortgage on a house worth a significant amount of money, that's going to create a lot of pressure, a lot of potential stress.

And so it is a good question, and it's one that I'll come back to. But it's not one that I feel is right for where I am now. Like Justin said, quit your job. I was amongst a group of people that were very much like me that were advocating for open source technology, that were very much engaged by learning technology, finding out what it's good for, and helping others understand that-- so this advocacy side of things.

I don't think you can get that when you go and just take a job with one company, and you're in one team. You're very limited as to what you can work on and what your scope is. I also think that having spent the last 18 months building this company up is entrepreneurs are traditionally looked as these are people that make money. But they're not. They're people that create value and can capture it. It's better if you can capture money as well, but I think that's one of things that I've learned.

I've gone on and created three more open source projects over, I think, the last year that have all gained a significant amount of traction. And so I feel like that's a big part of what I want to be, and I don't think that I'd be able to do that whilst full-time employed. And it is my worry, when I have these conversations, it's we have a dev role, or we have a this role, we have a that role. And I don't think that would necessarily align with those things I want to build and the kind of problems I'd like to work on.

ADAM GLICK: What are some of those other projects that you've started up?

ALEX ELLIS: I'm very reluctant to create an open source project now, because I've come full circle, and each of these projects that are being created solve a problem that I had and that I thought other people would have. Inlet is one of the newer ones, and that is a tunnel. You could compare it to ngrok, but it's not quite the same, because it's almost like comparing Lambda to OpenFaaS. It's something that you can self-host, and it's also something that's been modernized. So it works more deeply integrated into Kubernetes. It will be a load balancer for your Kubernetes services.

You get public IP addresses. You can serve TCP traffic. Whilst I was at VMware, one of the things we were doing was handling a lot of web hooks and building CI pipelines. And we just couldn't do that. Ngrok was banned by domain name. You couldn't even use SSH to go to a remote host.

And so I looked into it, and almost reluctantly, I created this project, because I knew there wasn't going to be a budget for the team to use. I thought, I'll just build it myself. It's a WebSocket. It's got the TLS on it. And it acts like a router. You run a cheap server, and you connect to it. And proxies don't block it. Corporate VPNs will never know about it, because it's always got a different IP. And that's been really popular.

ADAM GLICK: It's shadow IT as a project, basically? [LAUGHS]

ALEX ELLIS: Yeah. But actually, creating it as a business, I helped some of the functionality back. So I've got stage one out the door. Worked great. Big community around it. Stage two, I wanted the server capability, TCP, expose the database, replace the VPN, et cetera, et cetera.

But I don't want to give it away for free, and I'd learned my lesson. If something is free, a company will never pay for it, ever. Just my experience. So I kept it back, and it's proprietary. You can pay for it. And so far, that has worked. People have said, I want that. How do I get it? And I say, pay for it. And they say, OK, how much?

That's something that never happens with OpenFaaS, right? And that's why, in its current state, it's very difficult to balance.

CRAIG BOX: You've kind-of rediscovered shareware.

ALEX ELLIS: [CHUCKLES] It's open core, right, end of the day. Now, Inlets Pro is, hopefully, going to be one of the things that will help run the business. We talked about a case of these various streams of income. I think Inlets Pro could be one of them.

I don't know if it's going to work out, but as an entrepreneur, I want to do experiments that don't cost me a lot of money to find out if it's going to stick. And I think it's doing enough for them, being OK about the failures, being OK about the rejections, just like submitting CFPs to a conference. Knock after knock, eventually something will stick. It might not be what you expected it to. And you may do all right for yourself.

CRAIG BOX: How do you get K3sup to "ketchup?"

ALEX ELLIS: I think this is one that my friend John McCabe found. I'd said, this program is going to be K3sup. He said, like ketchup, and a ketchup bottle? And it kind of-- if you think about "kates-up", you say it fast enough, it's kind of "ketchup," and people like it. So it's more fun than, I don't know, k3sadm, or whatever else we could have called it.

CRAIG BOX: You live in a country that doesn't consume ketchup!

ALEX ELLIS: [CHUCKLES] I don't know. What do we call it?

CRAIG BOX: Brown sauce.

ADAM GLICK: Brown sauce?

[LAUGHING]

CRAIG BOX: HP Sauce. They don't eat ketchup in the UK. It's mayonnaise with fries, and brown sauce with everything else. But I digress.

ADAM GLICK: The branding side of me agrees with your decision to call it "ketchup." [LAUGHS]

ALEX ELLIS: [CHUCKLES] Yeah, so one of the things that I love is that if you get to a certain point, you can call a project whatever you want, and people will take it seriously. And this is one of those.

I came to another project, and I just didn't know what to call it. I wanted to call it something like "agora," meaning "the market" in Greek. Or-- I spent a lot of time thinking about these different words. And then, OK, well, what about a gallery, where you go in and you've got all these things laid out, different shops. What's another word for gallery? Arcade. And so Arkade came along.

CRAIG BOX: With a K?

ALEX ELLIS: With a K, right, to make it more Kubernetes-y. And that's a CLI and app installer. So if you have Docker installed on your computer, you can then use Arkade to download 'kind' that can then run Kubernetes. You can download Kubectl. You can download Helm. You can then go arcade install kubernetes-dashboard.

And rather than you having to go through 10 different blog posts that are out of date, that have got the wrong 404 manifest and all the rest of it, it will just always work, and when it breaks, we'll fix it immediately. So it's making it much easier for developers just to get different apps. Mineo, Postgres, OpenFaaS, Inlets Operator, whatever you want.

On your computer, try it out. Learn Kubernetes. If you want to run it in a production pipeline, that's up to you. If it uses Helm internally, you might as well.

CRAIG BOX: Arkade actually got spun out from K3sup.

ALEX ELLIS: It did. And again, this is where-- maybe, Adam, if you've ever had this, you call something something, and it just doesn't work. And you have to give it a new name to make it work.

ADAM GLICK: Mm-hmm.

ALEX ELLIS: 'k3sup app install', the things I heard were, well, you can only use it if K3s. It's like, no, any Kubernetes. Can only use it with Raspberry Pi. Why would you say that?

And this was a conversation I had time, and time, and time again. And I just thought, OK, you know what I'm going to do? I'm going to make a Kubernetes plug-in. And then I changed my idea, and I made Arkade, span the code out.

And again, begrudgingly, because I hate making new open source projects now, because I have to maintain them, and create community and a brand around them, and get all the GitHub pings. So I do as much as I can not to do that now.

ADAM GLICK: You showed us behind you, you have quite the array of Raspberry Pis as well and have been fairly involved in that world.

ALEX ELLIS: Yeah, Raspberry Pi, for me, I bought one when they first came out. I bought the B, the better one. I tried to run the desktop on it. I thought, what is this? It was so slow and laggy. And I put it away, right?

And I think it was about a year or two later, I got it out again because I'd seen a tutorial about turning an LED on on a breadboard. And I did it late one night, was fascinated by it, and really got the bug for all sorts of temperature sensors, LEDs. Started building little robots and competing with them at local events and then writing a blog posts about it.

The short story is I have around 100 or maybe more tutorials on Raspberry Pi. Pretty well-known for doing clustering with them. And it's been five years now that I've been mucking about with them, getting Docker running on them. And it was a really nice little highlight of the year, was to get a post on raspberrypi.com all about that.

CRAIG BOX: You also have a subscription, a newsletter that people can subscribe to, to hear what you think about what's happening in cloud-native. It that a weekly newsletter?

ALEX ELLIS: Yeah, that's a weekly newsletter. And actually, the way that came about was that I was trying to see if I could get companies and users to fund OpenFaaS. And it turns out that you can't. I managed to get up about $700 a month, for one month, which then dropped down to $300 a month. And it's not going to pay your mortgage.

Then switched over to GitHub sponsors. It did a little bit better initially. And then I just thought, well, maybe this is the wrong way around. Maybe I should be creating value for other people rather than asking them for donations.

And that is how I brand GitHub sponsors now. It's, effectively, it's a way for people to get value. Find out about my experience like we're talking about now, how to, maybe, improve their career, how to use different open source technology, and also, at the same time, I provide updates on all these open source projects I'm using. Yes, the changelogs for 20 projects on GitHub are out there. Enjoy going reading them all. Or, in one place, get them condensed every week.

ADAM GLICK: That's a great identification of the fact that sometimes, the value you bring is actually sifting through a lot of the information and putting it in a very consumable way

CRAIG BOX: Like the news section of this podcast.

ADAM GLICK: Or a teacher.

ALEX ELLIS: Yeah, and as I said, I want to have this sort of hat on of being an entrepreneur. And I thought, well, how can I get more value for people? There would be people that would join-- and GitHub doesn't have the feature yet where you can get past editions of the newsletter.

And they're not all about what was added to OpenFaaS. There's one that's called, "How to Make PRs and Influence Maintainers," which is my 10 notes and a bunch more blurb about how to get a PR accepted, how to be you know a good first contributor. That's in a treasure trove.

So take the nautical theme. You log in with GitHub sponsors. You have to be on a certain tier. And then you get access to all of those updates. It's written in a little function, runs on a Raspberry Pi, exposed by Inlets. But that doesn't matter. You can get access to all of that writing. You just go to the right tier. And again, that's worked reasonably well.

And this is what you have to do. If you want to be an entrepreneur, you want to pay your own way, you have to be willing to make experiments. If GitHub had this feature, they'd effectively take that revenue away. And I don't mind, because it's about how can I capture the value? How can I create more value for them?

ADAM GLICK: Traditionally, people have thought of open source as kind of a nights and weekends, hobbyist kind of undertaking. At least, that was a broad perception that was out there. But now, we've talked about some of the largest companies in the world are contributing to open source and focusing on it. What do you think has changed to make such a shift from the proprietary world and this being a hobby to this really being the engine behind how most things are built today?

ALEX ELLIS: One of the things that I guess you and I, and maybe Craig, were growing up was, yeah, open source was something that people did evenings and weekends. They spent their time hacking on stuff because it was fun.

Now, it can change your career. You can get on a SIG, or on a board, or part of a Kubernetes project, get a lot of high-profile conference talks in, and get hired by, I don't know, Apple. That's something this happened to a lot of people recently. Or maybe Packet.

Open source has changed a lot. I think some of the reasons why people contribute to OpenFaaS is because they want to be seen to be contributing to a bigger project, and it reflects well on them. Sometimes people contribute because, like when I was at ADP, I was a frustrated developer. I wanted to play with technology, but we had to do what was right for the business, understandably.

And so we have people that come because they can't write Go at work, or they can't use Docker or Kubernetes, and they want to. I think when you see people that are hired full-time to work on a project, like VMware, Microsoft, Red Hat, Google, et cetera, it has to be viewed through the lens of, it is your job to work on that project and to be in that team, but you're not hired to work on open source. You're hired to work on a project or product that happens to be open source. And it might as well be proprietary for what it matters, right? That's your job.

The way that I am enabling myself to work on open source is by not being hired. Because if I was, I would be, as you said earlier, almost pigeonholed into working on only one thing, and having interests and obligations to the company.

CRAIG BOX: One question about OpenFaaS, which is one of the bigger projects that you've created, is that it sits in the serverless space. And there are a number of other projects in that space-- thinking about things like KEDA. You mentioned Knative. OpenWhisk is other one I've seen you speak about. How did these relate to each other?

ALEX ELLIS: If you look at the serverless landscape now, you could be forgiven for thinking there's a lot of projects. Because in 2017, when the biggest part of OpenFaaS was written, there were way, way more projects, and there was a lot more interest from bigger companies in them. As we said earlier, VMWare had their Dispatch project. There was OpenWhisk. There was Kubeless from Bitnami. There was FN from Oracle. I mean, there were so many.

Some of them are still around today. There's also Fission being developed at Platform9.

CRAIG BOX: Riff, from the other half of VMWare?

ALEX ELLIS: Right, and Riff, that's gone through a few transitions now as well. Today. I think most of the time, you're going to hear people talk about Knative or OpenFaaS. Sometimes they talk about KEDA if they're particularly interested in eventing.

OpenWhisk is also coming up at times. There's a small company built around that to develop it further. But I think, if it's serverless, it was maybe just a bit too early. Maybe like Service Mesh is now, there's so much hype about it, and yet the demand for it, in terms of actual real use cases, maybe a bit more limited. And I think the same was of serverless back then.

And what both Knative and OpenFaaS have done is said, well, actually, maybe we can think about this more as providing a developer experience, where we'll abstract away the code. We'll just say, here is a basic, de facto definition, port 8080 for HTTP traffic. We'll serve on that. It's stateless. And we'll just basically give it a name, some environmental variables, and that is the contract.

What that means is that you can write code for Knative and run it OpenFaaS. You can write code for OpenFaaS and run it on Knative. They'll both scale to 0. They'll both scale to n. They both have event integrations. But they have different flavor.

And when these think about, I don't know, a Linux distribution, some people use KDE. Some people use GNOME. Some people really love Arch Linux where you build everything from scratch, and you have complete control and configuration of everything. Some people get the Ubuntu ISO.

And there's some of that with OpenFaaS, and with Knative and KEDA to some extent, as well. You can mix and match. I think we're going in parallel swim lanes with different approaches, different philosophies.

CRAIG BOX: Having created OpenFaaS, do you feel a duty of care for it?

ALEX ELLIS: Yes, I do. I'm really proud of the community. We have some really talented people there that care about it too. 290 people have contributed code to the base so far, and then there's a smaller group of eight people that contribute on a really regular basis. We have member slash team calls together. There's a certain number of things that they get as perks from that.

But I do feel like working with volunteers is very different than working with full-time employees. And I wrote about that in the "Five Pressures of Leadership" blog post in open source. And it does go back to, how can you fund a project? How can you make it sustainable?

And it's four or five options. And one of them is, effectively, you carry that. And you do consulting. And then you, for lack of a better word, are doing unpaid work to keep the project going and bring in other people.

If there was a team of 20 people working on OpenFaaS day and night, with, maybe, something like Knative, we maybe wouldn't need so much focus on community. But it's almost part of the DNA now. Being very heavily relied on from the beginning, it's the only way the project can run and be sustained. And that is really on a very shoestring budget there.

CRAIG BOX: Are a number of other places people might have seen in your regular blogging. This is something that, as a consultant, you have to do in order to keep the wheels of business turning over, as much as anything. Is this something that you enjoy, or is it just something that's necessary to be out there?

ALEX ELLIS: I think just like when I met Lord Birt--

CRAIG BOX: I do hope there's a Lord Ernie as well.

ADAM GLICK: [LAUGHS]

ALEX ELLIS: --in the backwaters of Peterborough that time, I absolutely loved writing about Docker. Anything I could get, day and night, that's all I'd do. I'd actually reserve time every weekend to go and write about stuff. Prometheus was one of the topics I covered one week, and it's still the most popular post I've got.

So yeah, I think there's a balance. I feel like I'm pretty good at creating a blog post, time-boxing it, getting it out there, understanding a good format for it. But it has become more of my work now. And so yeah, I have 10, 20 things I could be doing, and I just need to decide what is going to be the best use of the time.

CRAIG BOX: You and I had a chat once, and then, three hours later, we had a blog post about it. I should say you had a blog post about it. I had nothing to do with it at that point.

ALEX ELLIS: Yeah, I mean, that is something that I do get sometimes. I get, like, an earworm or something, where I just really want to see if something can be done. Again, one of the posts that I didn't expect to be that popular was about multi-stage Docker builds. And Tõnis at Docker just released it that night. And I stayed up till about, I don't know, 12 o'clock at night writing a blog post about it. And then Docker used it in their documentation, and it's one of the most popular posts I have on my site.

And that's just something about me. If I see something cool that can be done-- like I really want to create the TCP capability for Inlets-- I'll have it in my blood. I'll try and do it. I'll come at the problem try and solve it. And I don't know, maybe that's the engineering side of me.

ADAM GLICK: Do you still get out and do the cycling and hiking? Or is that time whittled away as you do more of these projects?

ALEX ELLIS: COVID makes some of these things harder. But one of the things that I have learnt is it's important for me to get time away from the computer. It's tempting to feel like I'm always connected, and always have a duty to man my Gmail, and reply to GitHub issues, and merge pull requests. But really, this last six months, I've started taking more time out at the weekend, for instance, and not feeling pressured to write another blog post.

One thing I really love is my cycling, really just getting out and spending a couple hours in the countryside on a bike. And then, hiking is something I do with my wife a lot. We like just go walking, taking in the fresh air, appreciating nature, seeing what wildlife we can spot. I think it's important to have that-- have something that you can use to unwind and something that isn't computers all the time.

ADAM GLICK: I see that you're going to be participating in a fun charity event for Black Girls Code coming up, playing Cards Against Containers with Ahmet Alp Balkan, one of the developer advocates at Google. I'm curious how you're preparing for that competition.

ALEX ELLIS: I have the pack of cards right here, and I need to learn the rules for this.

ADAM GLICK: [LAUGHS]

ALEX ELLIS: But what I did manage to do was to make some very funny sentences. But yeah, I'm really looking forward to it. I think it's all going to a good cause, and it's going to be a lot of fun.

CRAIG BOX: Thank you very much for taking the time to join us today, Alex.

ALEX ELLIS: Thank you both.

CRAIG BOX: You can find Alex on Twitter at @alexellisuk, and on the web at alexellis.io. You can find OpenFaaS at openfaas.com.

[MUSIC PLAYING]

CRAIG BOX: Thanks for listening. As always, if you've enjoyed the show, please help us spread the word, and tell a friend. If you have any feedback for us, you can find us on Twitter @kubernetespod, or reach us by email at kubernetespodcast@google.com. Or come up to our booth at KubeCon next week-- just kidding.

ADAM GLICK: [LAUGHS] You can also check out our website at kubernetespodcast.com, where you'll find transcripts and show notes, as well as a link to subscribe. Until next time, take care.

CRAIG BOX: See you at KubeCon!

[THEME MUSIC]