CRAIG BOX: Hi, and welcome to the Kubernetes Podcast from Google KubeCon Extravaganza. I'm Craig Box.

ADAM GLICK: And I'm Adam Glick.

[MUSIC PLAYING]

CRAIG BOX: Thank you, everyone, for joining us. If you're wondering why this podcast is a day late, well, we put it out on Wednesday this week instead of the usual Tuesday so that we could give you an update on all the news from KubeCon. And boy, is there a lot.

ADAM GLICK: Indeed. But first, the important update of what happened with Bird of the Year?

CRAIG BOX: Well, after a tense race over the past few weeks with some serious electioneering, and amid reports of ballot stuffing and other voter fraud, it turns out New Zealand's Bird of the Year is the kakapo, the world's fattest parrot, probably due to the fact it can't fly.

Any fans of BBC television may have seen one trying to have it off with Stephen Fry's head in an episode of a nature program a few years back. The kakapo is so rare that they only have, I want to say, 206 of them? I'm not sure if I got the number exactly right. But they know exactly how many there are. They keep them on an island for the purpose of conservation.

ADAM GLICK: Congratulations to the kakapo on its victory there.

One other note of housekeeping, we'll be taking some time off coming up. In particular, we're taking two weeks around the American Thanksgiving holiday and another couple of weeks at the end of the year. So our next episode after this will be on December 8. We hope you all get a chance to take some time off and relax as well.

CRAIG BOX: Shall we get to the news?

ADAM GLICK: Let's get to the news.

[MUSIC PLAYING]

CRAIG BOX: Congratulations to our friends at Banzai Cloud, our guests on Episode 59. Cisco has just announced their intent to acquire Banzai and merge the company into their Emerging Technology and Incubation Team, which is where their Portshift acquisition also landed. Cisco said that they intend for the team to continue to innovate in the areas of networking, security, and edge computing. Terms of the acquisition have not been released.

ADAM GLICK: The CNCF have released the results of their 2020 Cloud Native Survey. Highlights include that 92% of respondents use containers in production, up from 84% last year.

CRAIG BOX: 91% use Kubernetes, and 83% uses in production, up from 78% last year.

ADAM GLICK: Weekly app releases are most common, done by 26% of respondents. But 29% of respondents released daily or multiple times a day.

CRAIG BOX: 82% use CI/CD, with Jenkins leading the pack at 53%.

ADAM GLICK: 30% are using serverless in production, with 60% of those using a hosted platform.

CRAIG BOX: 27% use a service mission production, with 47% running Istio. 37% use a service proxy, with a majority using NGINX.

ADAM GLICK: And finally, 55% have containers with stateful applications running in production.

CRAIG BOX: Red Hat has made a number of announcements this week. Updates to OpenShift include a new edge computing feature for remote worker nodes, which are managed by a control plane back at your HQ. To show what you might do with your edge, the Red Hat team published a blueprint for machine inference-based anomaly detection on time series sensor data. OpenShift is now available in the government clouds run by AWS and Microsoft.

Additionally, a Red Hat/AWS jointly managed version of OpenShift similar to the Azure offering has been announced. Knative Eventing is now supported by OpenShift Serverless. And the cloud native Java stack provided by Quarkus.js is now available as part of an OpenShift subscription.

ADAM GLICK: The CNCF end user community has published its third Tech Radar, this time focusing on database storage. The radar suggests adoption of MySQL, Postgres, Elasticsearch, Kafka, Memcached, and Redis. The "trial" tier brings in Cassandra, Console, and MongoDB, as well as the cloud-hosted options of BigQuery, Google Cloud SQL, Aurora, and DynamoDB. The final category of "assess" contains CockroachDB, MariaDB, and Vitess.

The community is also bumping up its training benefits, with end user members receiving a 15-seat starter pack with unlimited access to e-learning resources and one certification exam each. End user supporters will receive five codes for 100% off e-learning or certification. For the broader community of users who have passed the certified Kubernetes administrator exam, the certified Kubernetes security specialist exam and certification are now generally available.

CRAIG BOX: Congratulations to the Envoy Mobile Project, which joins its older sibling in the CNCF. Envoy Mobile brings the benefits of the Envoy xDS APIs to mobile applications and developers running on Android and iOS. It has bindings for Swift, Kotlin, Objective C, and Java, and powers Lyft's production mobile apps.

Seven sandbox projects were also approved. cert-manager, the popular tool from JetStack, covered in episode 75--

ADAM GLICK: Amazon's cloud development kit for Kubernetes, cdk8s--

CRAIG BOX: The Kyverno Kubernetes native policy engine--

ADAM GLICK: OpenKruise, with a K, a Kubernetes automation suite for workload management--

CRAIG BOX: Pravega, a storage abstraction based on streams--

ADAM GLICK: SchemaHero, an operator for database schema migration--

CRAIG BOX: And Tinkerbell, a bare metal provisioning engine, and the hardest of all these projects to Google for.

ADAM GLICK: DataStax, our guest on episode 98, has released K8ssandra, an open-source distribution of Apache Cassandra running on Kubernetes with elastic scale and auto-healing functionality built in. The new distribution is designed to help DBAs and SREs easily manage a cloud native database and integrates with Prometheus and Grafana for monitoring and observability.

CRAIG BOX: K8s was compiled down to one binary in the form of Rancher's K3s. If that sounds like three too many for you, check out K0s from Mirantis. K0s promises zero-friction Kubernetes with zero dependencies and zero cost. The project is still at zero-point-7, but it's derived from the production Mirantis Kubernetes engine, and one-point-zero is promised for early next year.

ADAM GLICK: solo.io, our guest on episode 55 and makers of the Gloo API gateway, as well as a bunch of other things, has rebranded their entire portfolio around the Gloo brand. Service Mesh Hub becomes Gloo Mesh, with a new Gloo Mesh Enterprise offering production support for Istio.

CRAIG BOX: VMware dreams of a world where a single authentication process of your choice can be used to log into all of your clusters, cloud and on-prem. To that end, they launched Project Pinniped which delivers a consistent user authentication experience in Kubernetes. Using Pinniped, you're able to log in once to safely access mini clusters, using standards-based protocols and login flows and integrating with external identity providers.

Pinniped is the Latin for seal, perhaps a tenuous marine connection to Kubernetes, but the project hopes to get your seal of approval-- their pun, not ours.

ADAM GLICK: Shipa, an application management startup which launched in October, has open-sourced its engine. Ketch, a two-masted sailboat, is the name of a new application delivery framework for Kubernetes. It allows you to deploy and manage applications with a command line interface and no YAML.

CRAIG BOX: Want a Kubernetes dashboard and not impressed with Lens, Octant, Infra.app, the dashboard from your vendor, or the upstream Kubernetes dashboard? In the time-honored tradition of open source, Kinvolk found fault with all of those and built their own, which they call Headlamp. Headlamp is extensible, role-aware, and scratches their exact itch. If you think it might scratch yours too, check it out.

Kinvolk also announced new versions of their Flatcar Linux distribution, including an LTS version for stability and compliance; a pro version optimized for cloud use; and a free version, available in the marketplace of the three major cloud vendors.

ADAM GLICK: The SPIFFE project, building a secure production identity framework for everyone, has published a book explaining how to establish trust in your infrastructure. Solving the bottom turtle refers to the famous theory that the world rests on a turtle's back, raising the question of what that turtle is standing on. SPIFFE aims to put an end to the infinite pile of reptiles, and this book explains how and why. You could also listen to episode 45.

CRAIG BOX: If you want to get a feeling for what it is like to develop for Google's Anthos Platform, now all you need is a Google account-- no credit card required. The Anthos developer sandbox packages up the Cloud Code IDE and container build tools into Google's Cloud Shell, giving you everything you need to iterate on, debug, and test your applications before deploying to Anthos. Google also announced new Ingress features to GKE with SSL policies and custom health checks now generally available, and HTTP to HTTPS redirects now in beta.

ADAM GLICK: Ambassador Labs, formerly Datawire, closed an 18 million US dollar Series B financing round this week to continue to grow their cloud native API gateway and finally allow them to grow the team in Canada they have always wanted. The first order of business was releasing version 1.9 of their API gateway and Edge Stack, which adds IP allow-deny lists and customizable user-friendly error messages. They also announced version 1.0 of the K8s Initializer, a tool that enables developers to configure a full platform stack on Kubernetes with some of their favorite tools. No prizes for guessing which Ingress it installs.

Ambassador also preannounced version two of Telepresence, their CNCF sandbox two-way proxy for fast code testing and deployment. The new version will be rewritten in Go to better align with the contributor community while also increasing speed and performance.

CRAIG BOX: VMware has sprinkled some Tanzu over the Pivotal Postgres operator and released VMware Tanzu SQL, now generally available. The operator allows self-service and automated deployment of Postgres on any certified Kubernetes runtime with official support available running on VMware's products or GKE.

ADAM GLICK: And now it's time for the famous Kubernetes Podcast from Google KubeCon lightning round.

[MUSIC]

In the traditional alphabetical order--

CRAIG BOX: Accurics a cloud-based cybersecurity company, has announced that the open-source static code analyzer, Terrascan, has been extended to support Helm and Customize.

ADAM GLICK: Amazon added container support to their Lightsail virtual servers.

CRAIG BOX: Arrikto, with a K, announced a $10 million Series A lead by Unusual Ventures, to build GitOps for machine learning.

ADAM GLICK: Brobridge released a microservices data caching and provisioning platform called Gravity.

CRAIG BOX: CircleCI announced the GA of their runner service to build and test code remotely for teams that want to deploy on-prem.

ADAM GLICK: Cloud 66 announced a new version of Maestro designed for web development agencies, Skycap version two with several new features designed for easy multi-environment deployments, and multi database support for both platforms.

CRAIG BOX: Cloudflare released a search manager plugins for the Origin Certificate Authority they use for their CDN customers.

ADAM GLICK: Cloudical announced the soft launch of their GDPR-focused on vanilla cloud, a hosted Kubernetes service located in Germany and available in multi-tenant and dedicated varieties.

CRAIG BOX: Cloudify announced version 5.1 of their open-source, multi-cloud orchestration platform that uses layers of abstraction to enable easy migration between different Kubernetes environments.

ADAM GLICK: Codefresh has released a number of new observability and logic features in their CI/CD tools as part of an effort to brand what they are calling GitOps 2.0.

CRAIG BOX: Commvault announced Metallic VM and Kubernetes backup solution, a backup-as-a-service offering for container applications.

ADAM GLICK: Diamanti has added support for AWS in their Spektra 3.1 Ultima services. They also released a new customer portal called Diamanti Central.

CRAIG BOX: Dynatrace preannounced the release of PurePath 4, coming in the next quarter. The latest iteration of the distributed tracing and code path analysis tool will add support for OpenTelemetry and W3C trace context, service mesh, and serverless functions from all three major clouds.

ADAM GLICK: Elastisys released their open source compliant Kubernetes as GA.

CRAIG BOX: Fairwinds introduced a Kubernetes maturity model for identifying which of seven phases your organization is in and how to get to the next.

ADAM GLICK: Garden has come out of stealth mode with 3.1 million euros in funding for their unified development testing and continuous integration platform for cloud native applications.

CRAIG BOX: Gremlin, our guest on episode 82, has announced new soundproofing features to its Chaos testing tools to help protect applications from so-called noisy neighbors in the shared host environment.

ADAM GLICK: Humio announced the availability of an operator to deploy their open-source Kubernetes configuration management and deployment tool.

CRAIG BOX: Instana announced that their application performance management and observability tools are now deployable on top of Kubernetes.

ADAM GLICK: Intuit announced that they are now running the majority of TurboTax critical services on Kubernetes.

CRAIG BOX: Kioxia has announced a new storage offering to provide block, file, and object-storage simultaneously to Kubernetes via high performance in VME flash storage.

ADAM GLICK: Kubecost, our guests on episode 124, have added features to look at cloud costs outside of the Kubernetes cluster.

CRAIG BOX: KubeMQ has released an update to their message queuing system that does automatic network creation to simplify scaling up bridges, connectors, parameters, and config files.

ADAM GLICK: Kubermatic released KubeOne version 1.1, the latest version of their open source cluster lifecycle management tool for single Kubernetes clusters. The new release adds a cluster autoscaler and can mirror Docker images into a private registry.

CRAIG BOX: Kubernative has announced a change in strategy to focus on SINA, their software hub used to install software and its dependencies directly into your cluster.

ADAM GLICK: Kublr has released version 1.19, which includes a new user interface, full support for external clusters, AWS configurability improvements, and chaos engineering integration with Litmus Chaos.

CRAIG BOX: Lablup has announced a new release candidate of version 20.09 of their Backend.ai open-source, machine-learning platform to manage computing resources and software.

ADAM GLICK: Magalix launched KubeAdvisor 2.0, which now provides dozens of built-in policies for a 360 view of Kubernetes security, governance, and operational visibility. Additionally, reports identifying CVEs at the OS, orchestrator, network, and other critical entities are now generated by the tool.

CRAIG BOX: Mayadata, our guest in episode 56, has announced the commercial availability of Kubera Propel and Kubera Chaos. Propel is based on OpenEBS and Chaos is based on Mayadata's chaos engineering project, Litmus.

ADAM GLICK: Mirantis, our guest on episode 110, have announced an extensions API for their Lens Kubernetes IDE, allowing for developers and partners to create add-ons for visibility, security, API gateway access, and more.

CRAIG BOX: Puppet Labs is integrating their event-driven automation platform Relay with their Puppet Enterprise product. They additionally announced Puppet Connect, which is designed to connect people with self-service tasks and agentless orchestration across infrastructures.

ADAM GLICK: Reblaze Technologies has launched Curiefense, an open-source Envoy extension for securing web applications, sites, and APIs. The tool includes bot management, a web application firewall, application layer DDoS protection, session profiling, advanced rate limiting, and more.

CRAIG BOX: Replicated has announced that they are making their troubleshooting tools available as a separate open source project called Troubleshoot. The tool will help ease the challenge of installing applications on remote and air-gapped systems. It has two parts, a preflight tool to validate an environment before install and a support bundle tool to record logs from the installation.

ADAM GLICK: Styra, our guests on episode 101 and the creators of the Open Policy Agent, have released several new editions of their Declarative Authorization Service, giving them a free, pro, and enterprise edition.

CRAIG BOX: Sysdig added network visibility and segmentation with the announcement of zero trust network security for Kubernetes with the unfortunately unpronounceable acronym of ZTNSK. They also expanded partnership with IBM by adding Sysdig Secure to the existing monitor product to enable broader security scanning of CI/CD pipelines running in Big Blue's Cloud.

ADAM GLICK: Trilio announced TrillioVault for Kubernetes 2.0 to do backups of Kubernetes, including in multi cloud deployments.

CRAIG BOX: Zerto announced the beta of Zerto for Kubernetes, an extension to their existing data recovery platform to support cloud native applications.

ADAM GLICK: And that's the Lightning Round.

[MUSIC]

CRAIG BOX: Finally, if you're not sated for Kubernetes content by the end of this week, Google Cloud is hosting a free online event as part of their Open Source Live series. The event features Kubernetes maintainers talking about API requests, Admission Webhooks, Service Side Apply, and etcd. Live Q&A will be available throughout, and there will be a wrap-up where speakers and attendees can connect. The event takes place on December the 3rd from 9:00 AM to 11:00 Pacific, and you can find a registration link, as well as links to everything else we've mentioned today, in the show notes.

ADAM GLICK: And that, my friends, is the news.

[MUSIC PLAYING]

CRAIG BOX: Stephen Augustus is the senior open source engineer on the VMware Tanzu team. He is an upstream leader in the Kubernetes community, currently serving as chair of SIG Release, and with a hand in many other contributing groups. He is the co-chair of this week's virtual KubeCon North America event. Welcome to the show, Stephen.

STEPHEN AUGUSTUS: Hello, hello. Good to see you all. Hear you all.

CRAIG BOX: How are you doing? It's been a tough time. I'm sure it's been a very busy few weeks for you.

STEPHEN AUGUSTUS: Yeah, it's been crazy, is the nicest way to put it. Lots of things going on. Exciting nonetheless. I think what we put together is the culmination of a lot of the work that we've done across the community for the year, so it's always fun.

CRAIG BOX: And you're in New York City. I imagine that's probably been an interesting place to be the last few weeks as well.

STEPHEN AUGUSTUS: Yeah, New York always keeps it interesting. I think what's been going on with the overall situation with COVID as well as election results coming out, I think the city has been a little bit more alive than I would expect it to be right now.

CRAIG BOX: Let's first of all on this week's KubeCon event. We spoke with your co-chair, Constance Caramanolis, in episode 117. She said that for the event that you're working on together, the Europe event was planned to be in person and went virtual, but this is the first event that was actually virtual from the beginning. Her suggestions were more content, more whiteboards, and more puppies.

How did you manage to bring that to life?

STEPHEN AUGUSTUS: Ultimately, what I love about these conferences is the hallway track. Really I spend more time in the hallway track than I do attending talks. It's the opportunity for a lot of us to get together for the every four months that we have it. This time around, definitely felt like we wanted to do more around making it feel like we were in the hallway. Seeing some of the work on the Twitch stream that's now live for CNCF, as well as some of the activities that we're planning for-- the happy hours, all of the Slack sessions and breakout rooms that we've got going on in the conference, as well as the community days. Ultimately, the goal is to make people feel like they're in the room.

CRAIG BOX: How do you bring something like SIG Late Night online?

STEPHEN AUGUSTUS: Very, very hard to replicate, especially given the fact that it's usually not timebound. If you've heard of SIG Friday, SIG Friday I think has been our best approximation of what SIG Late Night was. Every Friday we hang out and talk about life, talk about work, talk about not work, talk about learning. That has given me some of the vibes of Late Night. It's also gotten people more accustomed to just hanging out with friends on a video call as opposed to in person.

CRAIG BOX: There are a lot of occasions at the conference where you see large groups of people who obviously have worked together a lot, and there are a lot of people for whom that's the first time attending a conference. It can be hard for them to get in with those kind of groups and maybe it can be even harder to do that online. What would you recommend to someone who's perhaps seeing names of people that they've heard on this podcast or they've seen in the community, and they want to reach out to them? What would you recommend they do in the virtual sense?

STEPHEN AUGUSTUS: I would say just do it. It's very easy to say that, after getting to know some of these people for a few years. But we all started as baby contributors, poking around the cloud native space and trying to learn a little more. Really one of the biggest suggestions I could give is attend the 101 Track talks. The 101 Track was specifically designed to help people onboard into the community and give them an idea of the resources that they should be looking for and the people that they should be contacting.

So do that. Hang out on Slack, get on both the Cloud Native Foundation Slack as well as the Kubernetes Slack, if Kubernetes is your thing. Lots of channels to hang out. Everyone is incredibly friendly and happy to help when they're available.

CRAIG BOX: Make sure you check out the Kubernetes Podcast Slack channel. Place to be. You say, there, everyone starts off as a baby contributor, and your experience with cloud native started at a startup. That was in the early days of Kubernetes. Tell us a little bit about how you got involved with containers and cloud native.

STEPHEN AUGUSTUS: My career is I think going on 15 years long at this point. I've done a variety of different things-- the classic corporate IT, desktop helpdesk, support, sysadmin stuff, security engineering, network engineering. Eventually I started looking at what's next? What's the next move in the career.

And it was looking like operations. This is right around the time that the DevOps was getting really buzz-wordy. And I was like, that's the thing that I want to do. That seems interesting.

CRAIG BOX: I'll do the second half of that, please.

STEPHEN AUGUSTUS: [CHUCKLES] Only the second half.

CRAIG BOX: Just the Ops.

STEPHEN AUGUSTUS: So I think given the opportunity to kind of leverage everything that you've learned from the Op space and start to learn a little bit more about the Dev space, start to learn a little bit more about cloud computing in general gave me an opportunity to reach a little deeper into this community. It started at a startup where we were doing some investigations of containerized services. We had our production services classic three tier application on AWS, and a lot of works in the background for a staging environment that was running on Docker Swarm. We had another DevOps engineer on that team looking into Rancher.

When I came onto the team, there seemed to be this gap in the middle that needed to be filled in terms of discovery. For me, that ended up being Kubernetes. Kubernetes ended up being a lot of the technology is down the CoreOS stack. So looking at service discovery and etcd, Fleet and Flannel for network overlay, as well as overall service management. And that ultimately led me to Kubernetes.

And my rallying cry for the company was, Google runs on the order of two billion containers a week with a technology that looks a lot like this. I think we don't have those goals. I think this technology can more than meet what we're trying to achieve.

I spent quite a bit of time digging up issues in our ice box, right, our backlog, and playing around with etcd, doing different things, standing up clusters. And we started investigating Tektonic at the time, and reaching out to some folks over at CoreOS. At the time, the sales team that they sent to us was Alex Polvi and Redbeard and Jeff Gray, who is their VP of biz-dev. They sent out the heavy hitters.

CRAIG BOX: That's the CEO and CTO and so on?

STEPHEN AUGUSTUS: Yeah.

CRAIG BOX: I guess that is a small company. They wear many hats.

STEPHEN AUGUSTUS: Yeah. Very enthused with everything that they were working on and their general vision for cloud native technologies. Ultimately, that led me to CoreOS. I think I was the only customer to employ conversion at CoreOS, and that was part of my bio when I came onboard. Really working through that and getting to work together with people who basically influenced part of my growing as a DevOps SRE production engineer type.

CRAIG BOX: You've gone through a couple of big acquisitions in the cloud native space. First of all, let's talk about the acquisition of CoreOS by Red Hat. How was that process?

STEPHEN AUGUSTUS: I remember that vividly. We were prepping for an all-hands, or I was flying out to San Francisco for what was supposed to be the sales kickoff for that quarter. I was going through security with the all-hands on my headphones, and it cut out a little bit in the airport. And I was pinging people on the Slack channel, and I was like what did I just hear? I said something, something, acquired by blank.

They're like, yeah, we just got acquired by Red Hat. I was like, oh, wow. Flying out to sales kickoff became like flying out to our acquisition celebration. Really, really great experience at the beginning. And moving forward, I was on the field engineering team at CoreOS.

I said to myself we know this technology cold. We implement it for customers every day. They need to know who does this. They need to know who had a hand in putting together these solutions. And I decided to move into the OpenShift Tigera team that's one of the specialist solutions architecture teams for Red Hat. If we're going to sell our technology, our baby, I want to make sure that someone who worked on it is in the front of that.

That was my goal within the acquisition. We spent a lot of time interfacing with customers, thinking through their migration strategies for Quay, as well as migration strategies for older customers moving from Tektonic into OpenShift, as well for OpenShift customers who were moving from three-somewhere-- 3.9 to 3.11 really-- into OpenShift 4.x, which was that integration.

CRAIG BOX: OpenShift 95.

STEPHEN AUGUSTUS: OpenShift 3.1.

[LAUGHTER]

Which was really the integration with all of the Tektonic components. That journey was fun. We got a lot out of it and built something better as a result.

CRAIG BOX: Was it around that time that you started getting involved with the upstream Kubernetes release process?

STEPHEN AUGUSTUS: Yeah. Kubernetes was dragging me in for a bit the wider community. At CoreOS, I was working with Starbucks, and they were interested in Tektonic on Azure. Tektonic on Azure was pre-alpha at the time, so I wrote a good chunk of what was Tektonic on Azure.

And to answer some of those questions, some of the stack was Terraform-based, so going into the Terraform community and figuring out what module didn't work with Azure and hacking up some solutions. So spending a little time working with the Terraform folks.

And then also realizing that not all of the answers existed in those Terraform modules and that maybe I had to go further upstream. I spent some time in SIG Azure, eventually chairing that, then also got more interested in the release process, some in documentation, and it kind of took off from there.

CRAIG BOX: How many people did it take to release Kubernetes when you first got involved, and how many people does it take today?

STEPHEN AUGUSTUS: I would say it doesn't take that many previously. It's about key people. When I was just getting involved in the release process, it was very Google-centric. The people who had the keys to do releases were heavily Google employees.

We started shifting that process outward into the community. It's a great success, I would say. There's the release engineering components of the process but also the people components of the process. So between the release engineering sub-project and the release team sub-project. The release team is this team of amazingly equipped cat herders which focus on various areas.

In total, the release team at any one time could be between 25 and 40 people. And they focus on various areas, whether it be bug triage, issue triage, CI Signal, communications components of it, the enhancements components of it. Each of those sub-teams has a lead. Each of those leads has a set of shadows.

The goal is to better understand the process, become an effective leader, be able to lead your sub-group, and then maybe eventually lead the release. My rallying cry sometimes is, we're going to delete the release team. One day less cat herding will be needed. The process just flows, and the requirement for the team goes down. So right now, a lot. Soon, maybe less.

CRAIG BOX: So you're employed now by VMware, and I understand you got there through the Heptio acquisition

STEPHEN AUGUSTUS: I did. Acquisitions. I was actually looking at Heptio right as they were getting acquired. And I thought to myself, we've got some time on this. They're probably being targeted for acquisition at this point, but we've probably got a year.

CRAIG BOX: And you were at the airport on your way to the sales kick off?

[LAUGHTER]

STEPHEN AUGUSTUS: During my interview process actually was when they made the announcement. When I got my offer letter, my offer letter was on VMware paper.

CRAIG BOX: So did you ever officially work for Heptio? Is there an internal group for the old timers? Did they let you into it?

STEPHEN AUGUSTUS: There's a Heptio OG Slack.

CRAIG BOX: I'm glad that you're able to get onto that.

STEPHEN AUGUSTUS: Yeah. I had to beg and plead. No, the relationship with all of the CoreOS folks-- a lot of the field engineering team, solutions architecture groups moved to Heptio during and soon after the acquisition. It was really an opportunity to work with my old boss, amazing person, as well as my old team. It felt really natural to go back to that team vibe.

CRAIG BOX: Was the upstream work that you'd done part of the benefit that you could bring to the company? Was that something that they looked very fondly upon?

STEPHEN AUGUSTUS: Absolutely. At Heptio's core, and definitely across VMware, the need is growing and the recognition that upstream is paramount to the success of the company and outside of finding contributors. What it's really important to do is find people who are willing to provide sustained effort into any one thing so looking for leadership in the community and looking for something that I felt that I was quickly establishing during that time was definitely additional checkbox for coming into the group.

CRAIG BOX: Let's talk about some of those leadership positions that you've had. You were one of the leads of SIG PM, and that was the product, project, or program management SIG, depending on what day of the week it was and how you chose to look at it. That SIG doesn't exist any more. So you might be in a good position to talk about why it started and why it ended.

STEPHEN AUGUSTUS: Yeah, of course. There are some deep, dark history and magic to Kubernetes that I find out a little bit more every day. But initially when SIG PM spun up, the idea is Kubernetes, while it's an open source project, it's something that we as a community depend on deeply, cross vendors and end users, and in that sense Kubernetes is also a product. Whether or not certain people look at it that way, that's what it is.

From the perspective of the way that we organize our governance model, Kubernetes is also program. The way we carve out these different explicit programs to focus on the horizontal and vertical efforts, SIG PM made a lot of sense to try to apply what is classically the PXM methodologies to an open source project. Some of our learnings have been interesting, namely that it's hard to do and maybe it can't be done.

As we grew in SIG PM, we learned some of this. All of the people who have been SIG PM chairs, or involved, have deep, deep experience in product, program, project management in their day-to-day jobs. I was not one of them. I was just an organized engineer at times.

And often when you're organized enough, and you're available, and you're interested in the work, you can do the work. And open source people will enable you and give you that space to learn. My career in SIG PM started as part of when I joined the release team. So I joined the release team as a features shadow under Ihor.

CRAIG BOX: Which release was that?

STEPHEN AUGUSTUS: 1.10? Yeah. Feels like decades ago at this point. I joined and was very interested in the features collection process, what's now the enhancement process and everything that kind goes into that. I realized it was broken. What I hoped for out of any release team member is that they learn, and they grow, and then they leave. They recognize some deeper problem in the community that is related to that area that they may be working in.

So you think, again, with the release team we've got, essentially, these arms out into various areas of the community. So you think communication has an arm out to contributor experience, and CI Signal has an arm out into SIG testing, and enhancements had an arm out into SIG PM, so on and so forth. My hope for a release team member is that they eventually grow into pushing the boundaries of that arm and trying to fix some inherent problem.

I found features collection, the enhancement tracking spreadsheet, and all of the business around the beginning of the release cycle to be a huge burden, both for the release team as well as literally every SIG who buys into the release train. My goal is to try to fix what of it I could.

The nugget that we got out of this was the enhancements process, Kubernetes enhancement proposal that came forth. This is something that was developed based on the thinking of the Python enhancement proposals and also the Rust RFCs. So just being able to describe what we do and make sure that we can commit it to the project's history, just as we would any piece of code was huge. Being able to look back into our history and understand why we made the decisions that we made instead of having them in GitHub issues.

CRAIG BOX: It's also very useful for going forward. So for the people who aren't familiar with the process, when you want to enhance Kubernetes in some way, it might involve dozens of commits or issues. But you actually have one document this says here's what I want to achieve, and then maybe even break that down into documents and how you'll get a feature from inception to alpha to beta to GA and so on.

STEPHEN AUGUSTUS: Exactly. Between the work of some folks like Eric Tune way back in the features collection process, and then also Caleb Miles and Jaice Singer DuMars. I remember when I was getting started and Jaice was like, are you sure that this is the thing that you want to work on? It has a lot of history behind it and there's a big potential for ruffling feathers. I want to make sure that you're comfortable with that before we get into fundamentally changing something that is core to the community now.

It was exciting. And I think that at the time I was the bright-eyed contributor and was like, yeah, let's do it. Just give me that thing. And--

CRAIG BOX: Did you end up ruffling any feathers?

STEPHEN AUGUSTUS: Oh, for sure. For sure. We ruffle feathers to date. What's interesting about the process is that the process is also subject to the process. So when we improve the idea of enhancements, we have to step back into our enhancement proposal, right? We have a meta-KEP which describes improvements to KEPs overall. We're working on what should be a big one for one twenty-one, which I'm excited about.

It's called the receipts process. The receipts process is essentially we want to reverse this push-pull mechanism that we have with the release team reaching out to people and dragging information out, figuring out where documents are attaching, connecting all of the dots between the docs and the Kubernetes, Kubernetes PRs, and the tests and implementation history, and putting that all in one place. And instead do what we might do in an organization, have groups commit to the work that they're going to do. Again, it's going to be a YAML, because we love YAML in cloud native.

CRAIG BOX: Who doesn't?

STEPHEN AUGUSTUS: The idea is that the way we have the enhancements tracking spreadsheet right now that gives you an idea of what's being tracked, what's at risk, what's been kicked out of the release, any exceptions, so on and so forth. We want to put that in YAML. Being able to aggregate all of these, get these receipts, basically run a tool on it, generate a manifest. A manifest that gives you an idea of exactly what's in the release.

Part of that tooling will be applying pre-submits to some of those folders so that your receipt won't merge unless you do these things. It doesn't have a thing attached, doesn't have a docs PR attached, doesn't have the right status on your KEP. Starting to tie the thread is the reason that we created the metadata for the KEP into what is something that people have looked for since the start of the project, which was really a roadmap.

Starting to be able to get at least a clear vision of what's happening in a release with a lot less human toil. And then putting this up on a website, putting this in a way that's consumable, whether it be an end user who just wants to look at a website, or maybe a SIG who wants to ingest this data and get a report of what's at risk for them, or maybe what they're planning for the next few cycles.

CRAIG BOX: Can we hold the next release until Sidecar containers are already?

STEPHEN AUGUSTUS: Sidecar containers have been requested for quite a bit. And I think that, ultimately, when we push forward enhancements, it's always important that we do the right things at the right time and try to make sure that the enhancements that we release are as stable as it can be before releasing them to the community.

Corollary for that is the PRR that has recently popped up, or the Production Readiness Review. That becomes a requirement of the KEP where you basically have to describe how your feature, your enhancement, is going to react in production. Because this is what people really care about. If I run this on my cluster, is everything going to go horribly wrong?

Getting a little stronger validation from enhancement owners about whether or not that's true is definitely part of it. The sidecar containers, I'm not sure when they're coming, but if everyone's following the process, we have a lot more clear idea of what is coming and how it's coming in, maybe getting to the point where people are less scared of consuming a dot-zero Kubernetes release, which I think is a common bit of feedback I've heard.

CRAIG BOX: So you could always go with id Software mantra where it'll be really like a week after the last person asks about it.

STEPHEN AUGUSTUS: Like now, is it time? Is it time? Should we go now?

CRAIG BOX: I just asked again, so I'm sorry. I've reset the clock.

STEPHEN AUGUSTUS: And we're running into a code freeze right now for one twenty, so--

CRAIG BOX: You're the co-chair of SIG Release, along with Tim Pepper, who was our guest of episode 10, when he was working on Release at that time. He's rotating out of the lead role at the moment. Tell me a little bit about that decision and how you're keeping the team growing.

STEPHEN AUGUSTUS: One, Tim is a phenomenal co-chair. I love the fact that we started at the same time. This is something that, again, I kind of follow Jaice's roles. Jaice was a chair for SIG Azure, worked on SIG PM and was one of the chairs when I started off in SIG Release. This is something I poked him about being interested in, and just poked and poked.

And one day Jaice was like it's time, I'm doing it. I'm going to step down and give an opportunity for some new blood to come in. Tim and I were in that meeting, and we're like, hands up immediately. We're the ones, pick us.

We had been doing the work at the time. It was a good opportunity to really step into leadership. SIG Release has taught me so much about how to be a leader. I'm still learning and growing every day. And definitely having someone senior, like Tim, in the cockpit with you, steering this stuff and trying to understand the vagaries of the Release engineering tooling, how we make things better for everyone, balancing those priorities.

Tim and I have very different styles, I would say, a matter of experience. Tim is incredibly methodical, well-written as well, and has deep, deep experience in software architecture. I'm coming from the opposite side. I'm coming from small-mid businesses, where ultimately, if the thing falls down, even if it wasn't your thing, you own it. You own making sure that happens.

I often bias towards action. Yes, we might break the thing, but, as a result, we might learn something. We might learn some nugget that we didn't have before. Tim is the slow and steady. I'm the move fast and break things. That struck a really great balance for the SIG, going into the release team and maybe eventually dissolving that sub-project. Again, this is probably a few years out if we were ever to consider.

CRAIG BOX: Is it your goal just to dissolve every SIG that you're ever involved in? Automate yourself out of a job?

STEPHEN AUGUSTUS: Basically. DevOps for Kubernetes, right? Part of what we did is we started to take some of the technical roles around Release engineering tooling and pull them into their own sub-project. That became the Release engineering sub-project. And that was guided by the branch managers who were on each Release team, as well as the patch Release team members.

Normally the patch Release team members were responsible for one explicit set of Kubernetes minor patches. If you got 1.10, you were on the 1.10 train until 1.10 was deprecated. We decided that that model wasn't necessarily great for sustainability. If a patch Release team member needs to roll off, and they're no longer covering their branch, then we have to find coverage for that.

So we eventually made this team of just Release managers and Release manager associates, where they would be responsible for doing all of the things, whether it's anything around branch management, actually cutting the releases, and they would share the responsibility across all of the in support Release cycles. And then as well as having this kind of associates team, which are mentees to the release managers proper group.

This model is somewhat adapted from the at-the-time product security team, now the product security committee. We were like, this is a nice model to bring people into a role that's incredibly privileged in the community, right? These folks have the ability to essentially blow up Kubernetes if they did the wrong thing. So it was important for us to allow time to learn and grow these tools, as well as make the tools safer to use so that we could bring more people into the process.

And I think that we did a good job doing that. We've done a great job growing the release team, growing its processes, starting this emeritus advisor role, which is essentially maybe an older Release team lead or older Release team sub-group lead back into a future Release. And have them guide the process, not quite from the shadows, but in the background, allowing a role where shadows could reach out for mentorship, problems that are happening within their team, and kind of building the sustainability across time.

So going to your question, I think doing all of that has allowed leaders within our SIG to kind of emerge. When we were thinking about transition, one of the things that we had planned for is this is not something that we're going to do forever. One of the things that we had initially planned for was, first, we brought in SIG technical leads. And it's the first time SIG Release had technical leads as of last year.

We brought in two of our folks who had been doing incredible work on the Release engineering and the Release engineering sub-project, as well as the Release team sub-project, and brought them in to steer this across cycles instead of from this quarterly standpoint. And then from there, we've seen more people grow into these roles. As Tim's rotation happens, we bringing in Sasha, who came in as a technical lead, Jorge, who is technical lead on the Release team's CI Signal side.

Jorge remains a technical lead. Sasha is coming up to become a chair, focused more on the people aspect of the SIG process and administriva. And then Dan Mangum, who has been a phenomenal-- shot through the ranks on the release engineering side has been a phenomenal member of the team. We're bringing him in to essentially replace Sasha as technical lead. We're slowly seeing leaders emerge and present themselves with the work that they're doing, and rewarding them for that by giving them more work, I guess.

[LAUGHTER]

One thing I'd love to point out regarding the leadership overall is one of the best things that we've done for the SIG is, actually dovetailing into the SIG PMiness of it all, is we brought on the community's first program manager. Lauri Apple is the program manager for SIG Release, and she is also one of the enhancement sub-project owners for SIG Architecture.

With all of these folks in the room, now it's starting to feel like something that is truly sustainable. It's not on any one person's shoulders. Continuing to learn and grow through that.

I thank Tim for his work. He has been phenomenal. He's not going away. He doesn't have the hat on. Helping me and the rest of group build something that is more sustainable than it was when we walked in.

CRAIG BOX: After the death of George Floyd in May and the protests that ensued around the US, there were two things that happened to the Kubernetes project that you were involved in that I'd like to hear about. First was a banner message committing that racism was unacceptable in the Kubernetes community. And the second is a program to go through the Kubernetes code base and ensure that the naming of everything was inclusive. Can you tell me a bit about the decisions behind and the execution of those two programs?

STEPHEN AUGUSTUS: First, with the banner, it's a no-brainer. For a lot of communities, it's can we say the words and then apply the action to it? I think a lot of the, "efforts," quote-unquote, that have happened in various areas have been-- are they as deep as people expect them to be or think they actually are?

With the Kubernetes community, it's in the name. We are a community. We are a family. These are people who are my friends. These are people who are my family.

And I think that part of being a community is making sure that we're inclusive. Part of being inclusive is not being afraid to say things that are important to your community members. Saying we support you. We support you in what you're doing. We support you in the base idea of existing. That shouldn't be hard to say.

So that banner was honestly a no-brainer. I chatted with steering about we have to say something, even if it's just a start. We can't not say something. We are the Kubernetes community. We can't not say something.

We hacked up the banner, hacked up the ability to post announcements. This also turned into enhancement for the website, being able to publish announcements in general. Now steering and docs are also working through a policy to determine how announcements go onto the website, how they're approved, how long they stay, just to make sure that this process is, again, sustainable over time. Gives us the opportunity to use our voice as a community to say things that are important.

I don't believe this to be something that is political in any way. It's human. None of what we do in cloud native is about technology. It's really about making life easier for the people that we work with and the people that come to depend on the products that we have. That's the banner.

Wrapped into that was, that can't be it. Just a banner feels hollow. What are we going to do?

If you dig into k/website and kubernetes/community history, you'll see some issues around handling naming overall. If you think about the handling of PetSets. PetSets to StatefulSets, or the naming of minions to nodes. We've had the conversations before. The master conversation has come up in the past, and that issue is incredibly heated.

People not understanding the decision-- project elders. Brian Grant is the one who was like, "this is too heated": lock the issue, and close it out. It wasn't a productive conversation. Any issue that you open like that is essentially a troll magnet. Lots of that happened. This issue kind of kicked back up on the mailing list last year. What year is it?

CRAIG BOX: It's hard to tell these days.

STEPHEN AUGUSTUS: Yeah. It was a thread about the way we name control plant nodes, the way we name components, and kind of having that discussion through where the individual components of Kubernetes clusters are nodes, and you sprinkle some stuff on nodes.

Some of those nodes hold special roles. You think about the control plane of Kubernetes. It's not any one role. It's the set of components that comprise that classification. I'll talk about the naming stuff in a bit, but just in general it's ambiguous. The language is incendiary.

When that thread popped up, we had some fruitful discussion, but no one took action. Some of the hats that I wear-- I'm cautious of putting on too many hats. It's something that I wanted to give the space for someone to step up and start charging forward with. During the George Floyd protests and everything that had been going on with Black Lives Matter, I dug up the issue and said, OK. It's time we did something.

CRAIG BOX: Now is the time.

STEPHEN AUGUSTUS: It's almost a year to the day when we looked at it again. And I was like, all right, let's do something. It sounds like we know what we want to do. What do we do in Kubernetes when we know what we want to do? We form an effort. We form some sort of governance group, sub-project, working group, SIG, what have you. It's time to form a working group.

This seems it is an effort that touches multiple SIGs and that's the definition for a working group. It's cross collaboration across multiple governance groups. And then it's also something that can be considered to be "time bounded," with time bounded in quotes. It depends.

CRAIG BOX: Some number of decades, maybe.

[LAUGHTER]

STEPHEN AUGUSTUS: Yeah, so we went forth, and we knew that we needed to pull together a few different types of people. We needed leaders and active members in SIG Architecture. We needed folks in SIG Docs. We needed folks that are on, in, around steering contributor experience, as well as the Code of Conduct Committee. That's what we put together.

Myself, I'm around. I do stuff. I do a few things in the community. Zack is now an emeritus SIG docs chair. I love any opportunity I get to work with Jaice, and being part of the Code of Conduct Committee as well as in and around steering and being a project elder.

Celeste as well. Celeste is a senior technical writer at CNCF, as well as very active SIG docs member. We thought we put together the right group. You'll hear throughout the conference, as we talk about naming, it's really how can we do this once? We want to build something that is repeatable enough that it can be carried into different open source projects, it can be carried into larger LF and CNCF efforts. It can be carried outside of the cloud native community without hopefully much burden.

What we've recently done is lay down a framework for language evaluation, templates for making recommendations about these changes, as well as really what the workflow should look like. How do we take a recommendation from inception to having that thoughtful discussion about making changes, as well as actual execution? And who is responsible for execution?

What becomes interesting about the group is that you have these parallel efforts happening, where the first part of it for us is making sure that-- again, I say this is sustainability, right?-- making sure that we build a process that is sustainable and is repeatable for the community. And then also making sure that the recommendations can be sticky. Because some of these terms are so contentious, we don't want to have to go through a reversal of any decision, to make the process sticky enough that the justification is there to keep it in place.

The parallel track is we're full of energy, this community, and people are ready to contribute where they need to contribute. People already started making the changes. People already have started renaming git branches and swiveling the search and replaces for various terms. And corralling those efforts and making sure that the active work is starting to fall in line with the active work for developing the framework and working to backfill any of those decisions so it is a recommendation that is sticky.

So that's a lot of what we've been doing so far. But what has been really great to see is the evolution of that work into the various companies that we work with, as well as thinking about the work internally at VMware, and knowing that when they were looking around for references, they found our docs, and asked, are these great docs to use? Have you seen this before?

And it's like, yes, we wrote it. We have a decent amount of confidence in it. Seeing that goal realized of not repeating the work, being able to reuse those frameworks across full areas. And that's evolved into this inclusive naming initiative that, again, the first community meeting for is later in the week.

So it's been really exciting to, one, see these ideas turn into reality. And then also get an opportunity to work with all of these incredible people across companies who are dedicated to open source projects, standards, organizations, as well as the LF and CNCF. That's exciting. And the work is just getting started.

CRAIG BOX: Well, you've definitely got a lot going on, not least with the conference this week. Just to end up, Dan Kohn, as the previous Executive Director of the CNCF, was a constant presence on stage at these events, and was also responsible for selecting the co-chairs for the event. I wondered if you wouldn't mind sharing a few memories of your interactions with Dan, who passed away earlier this month.

STEPHEN AUGUSTUS: Dan, just an amazing guy. And if you have the opportunity, go and check out the memorials CNCF repo to see just all of the stories that people have shared about Dan. One of the common threads across all of those stories is community. Building real community. Giving people opportunities for leadership and giving them the space to realize that. It's definitely part of the story for Constance and myself.

It's interesting to recognize some of the work that you do is maybe bigger than you realize. And this is not just for me, this is for everyone. Don't discount the work that you do. Everyone has a streak of imposter syndrome. But having people consistently come to you and give you that space to learn and grow, Dan is 100% one of those people. Those stories tell everything.

My last conversation was about accepting the role for KubeCon chair. Dan was heading off to China. And Dan and I both live in New York. It's constantly on and off plane. It's ships passing in the night.

If you think about the conferences, in my first few KubeCons I did, I think, six or seven talks apiece, flitting between room to room. Not really recognizing your impact or your potential impact to the community and maybe not feeling like you are important enough to go talk to that person. We were talking about that earlier.

I felt that way a little bit with Dan. He's too important. He doesn't have the time.

I remember a time before the Open Source Leadership Summit, where I was outside, getting some fresh air, detoxing after a chat. And Dan was on the phone, looked like he was in a very serious conversation, and paused the conversation to turn to me like hey, Stephen, how's it going? And just take a few moments to chat. That's the kind of dude he was. Very friendly, very caring about this community, and very much about giving people opportunities.

I did not expect to become a KubeCon chair. I didn't know what the path was to be considered for that, even. To, one, get the offer from Dan and to get the play-by-play of how that happened from Nancy kind of warms your heart to know that the impact that you're having is deep enough to be considered for something like this. More than anything, my goal is to make this one of the best conferences that you ever go to.

CRAIG BOX: Or don't go to, as the case may be.

STEPHEN AUGUSTUS: [CHUCKLES] Or attend virtually. That's the last conversation I had with him. And I think that that would be what he would want.

CRAIG BOX: All right. Thank you very much for sharing that memory with us, and thank you for joining us today.

STEPHEN AUGUSTUS: Thank you so much for having me, Craig.

CRAIG BOX: You can find Stephen on the KubeCon keynote stage. After that, you'll find him on CNCF and Kubernetes Slacks, on Twitter @stephenaugustus, or on the web at just.agst.us. You couldn't get august.us?

STEPHEN AUGUSTUS: [LAUGHS] It was the thing about dropping the U's, right?

[MUSIC PLAYING]

CRAIG BOX: Thanks for listening. Thanks for making it to the end! As always, if you've enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on Twitter @KubernetesPod or reach us by email at kubernetespodcast@google.com.

ADAM GLICK: You can also check out our website at kubernetespodcast.com, where you'll find transcripts and show notes, as well as links to subscribe. If you're listening live, remember, we're taking a two-week break, and we'll see you again in December. Until then, take care.

CRAIG BOX: See you later.

[MUSIC]