#118 August 25, 2020

Kubernetes 1.19, with Taylor Dolezal

Hosts: Craig Box, Adam Glick

Taylor Dolezal is a senior Developer Advocate at Hashicorp and the Kubernetes 1.19 release lead. His desire to give talks and join the CNCF Ambassadors led him to the release team and to his new job. He talks to Adam and Craig about how a TI-83 calculator started him on the path.

Do you have something cool to share? Some questions? Let us know:

Chatter of the week

News of the week

CRAIG BOX: Hi, and welcome to the Kubernetes Podcast from Google. I'm Craig Box.

ADAM GLICK: And I'm Adam Glick.

[MUSIC PLAYING]

CRAIG BOX: How can I be sure it's really you speaking to me right now?

ADAM GLICK: You'll have to take my word for it. But deep fakes are getting better and better.

CRAIG BOX: They are.

ADAM GLICK: One of the things I stumbled across this week was a thing called moondisaster.org, which was a project that basically tried to create a deep fake, which is artificial intelligence creating media that never actually existed. And they created a video of President Nixon giving a speech about Apollo 11 failing as a way to bring the astronauts back, that the astronauts have actually died on the moon. And it's based off of a speech that was actually written but, of course, never delivered, because the astronauts made it back.

And it's just an interesting kind of proof point into how good a fake can be. It was created by some folks at MIT. And also, it's an interesting way to get yourself educated on some of the technology that's out there for creating fake media. Full disclosure, my cousin was involved in the project.

CRAIG BOX: Wow. Did he use the floating head of Nixon in a jar, from "Futurama" as a reference point?

ADAM GLICK: He did not. But if you look in the credits, you'll certainly see a last name similarity that will identify what his role was. But I found it was a really interesting thing, just to educate myself a little bit on deep fakes, as people talk a lot about media that's created today.

CRAIG BOX: Well, I got out over the weekend and went to a lavender farm.

ADAM GLICK: How would you describe the taste of lavender?

CRAIG BOX: The lavender farm had a little cafe. And you might expect everything there just has that little hint of lavender to it. So I got a bunch of different things. And lavender earl grey tea was quite nice. The lavender scone, tolerable, I guess. The lavender vanilla cupcake, quite nice.

Lavender ice cream, however-- on a hot day at a lavender farm, you might think, I'll have some ice cream. I'm like, it just takes like soap. I didn't quite know whether there wasn't enough sugar, or whether it was the bitterness of it, or something. It was just, it wasn't nice. And it's just quite a shame. You think, well, that would've really topped the day off, a nice hot day in the sun, in end of summer, and get a little bit of ice cream. But my advice to you, if you ever find yourself with the option to eat the lavender-flavored ice cream is, don't.

ADAM GLICK: Shall we get to the news?

CRAIG BOX: Let's get to the news.

[MUSIC PLAYING]

Kubernetes 1.19 is out. The release includes over 30 enhancements, with 12 going to Stable. That includes the Ingress API, as foretold in episode 104, the certificate signing request API, and auto-rotation and renewal of kubelet certificates. Other highlights include lengthening the support window from nine to 12 months. You'll have more about the release in today's interview.

ADAM GLICK: Also out this week is Istio 1.7, adding more fit and finish around VM workloads, support for canary upgrades when using the Istio operator, and the ability to ensure the sidecar starts before your workload amongst other features. Upgrading is easy as proven by Banzai Cloud, who have already updated their Backyards mesh to Istio 1.7 as well as adding SRE dashboard features.

CRAIG BOX: The Istio project, this week, announced a new revision to its steering charter, which opens up governance roles to more contributors and community members. The new charter ensures the community around the project will always be able to steer its direction and that no one company has majority voting control over the project. Salesforce joins Google and IBM as members by contribution. And an election has started for four community members from other companies.

ADAM GLICK: Rancher Labs' cut-down Kubernetes fork/distro k3s, or "keys", has joined the CNCF sandbox. This move wasn't without controversy as other similar projects like Minikube and Kind are Kubernetes sub-projects rather than top-level CNCF projects themselves. Either way, the project is popular. And you can learn about its history in episode 57.

CRAIG BOX: A number of networking updates to Google Kubernetes Engine this week-- services using the internal load balancer are generally available as well as features for global access and configurable subnets. You can also now use private IP addresses outside the RFC 1918 ranges for when your network has exhausted 10/8 and is hungry for more.

ADAM GLICK: It's Anthos week at Google Cloud's Next on Air. They announced new hybrid AI capabilities starting with speech to text available on prem, a beta of Anthos running on bare metal, the ability to attach other Kubernetes clusters to the Anthos UI, deployment templates for regulated workloads, and Anthos identity service, which allows you to use open ID connect compliant identity providers to work with your hybrid applications. Migrate for Anthos also added Windows Server support, the ability to migrate from on prem VMs to Anthos running on prem, and integration into the Cloud Console UI.

For developers, there are lots of updates to the Cloud Code plug-in for IntelliJ and VS Code, including support for Cloud Run on Anthos and Google Cloud Build packs, which help define a containerized application without having to write a Docker file.

CRAIG BOX: The serverless framework at the plumb domain name of serverless.com has added support for Knative in partnership with Red Hat. You can use the new serverless framework Knative component to deploy applications written in any language or framework you're familiar with. And their quick-start guide includes templates for Express.js, Go, and Java.

ADAM GLICK: VMware has pre-announced the release of vRealize 8.2, their cloud management software and SaaS offering. Updates related to Kubernetes include the auto-discovery for Tanzu Kubernetes guest clusters, an adapter that will take the data being sent to Prometheus and forward it to vRealize operations, and the rebranding of the management pack for container monitoring to the management pack for Kubernetes. No date for the release was provided.

CRAIG BOX: Kubernetes' SIG architecture has announced the change to the API lifecycle. As of 1.20, beta APIs will now have nine months to move to the stable stage or to ship an updated beta API version. If an API does neither, it will be deprecated. And after another nine months, it will be removed. The goal of this changes to avoid APIs staying in beta for extended periods of time-- we're looking at you, Ingress-- and increase the stability of each release.

ADAM GLICK: MailChannels has released Palinurus, a tool to automate the creation of Helm charts from Kubernetes resource files. The tool is open source. And the GitHub repo helpfully lists a number of caveats from the read me to help you understand if the project can help you with your use case.

CRAIG BOX: Falco, the open source security tool that takes kernel events and provides additional information from the cluster with them, has released version 0.25. Headline improvements include understanding pod security policy, adding tags that map to the Mitre ATT&CK framework, and awareness of recently discovered CVEs. gRPC is now available as the output means or as a listener, which has enabled integrations with other community projects, most notably Prometheus. Finally, the release has a new driver that respects the pod node security boundary and doesn't require host access.

ADAM GLICK: AWS has relaunched AWS controllers for Kubernetes, previously known as the AWS service operator, into developer preview. ACK is like the GCP config connector, a tool for creating resources on their respective clouds using the Kubernetes API.

CRAIG BOX: Also relaunched, a VMware project that used to be known as k14s for Kubernetes tools, it is now known as Carvel. Carvel, a technique for boat building, includes three tools, ytt, a templating tool, kapp, for managing Kubernetes resources, and kbuild, a container image building orchestrator.

ADAM GLICK: Red Hat's Operator SDK has reached version 1.0. Changes include the inclusion of Kubebuilder into the SDK for Golang operators, a new project layout, and CLI switches based on Kubebuilder's Makefile-based approach.

CRAIG BOX: Two tools for scaling Prometheus have become incubation-level projects in the CNCF. Thanos, which aims to centralize and scale Prometheus, was created at Improbable, and is sick of hearing your Avengers jokes. Cortex, which is designed to provide horizontal scalability and multi-tenancy with long-term storage for Prometheus, was created at Weaveworks, and has a mind of its own.

ADAM GLICK: Finally, Farhan Hasin Chowdhury from Free Code Camp has released an online book called "The Kubernetes Handbook." As the name suggests, it's a beginner's guide to Kubernetes. If you're new to the community and want to dive in, this is a nice free way to get started.

CRAIG BOX: And that's the news.

[MUSIC PLAYING]

CRAIG BOX: Taylor Dolezal is a senior developer advocate with HashiCorp and the release lead for Kubernetes 1.19. Welcome to the show, Taylor.

TAYLOR DOLEZAL: Howdy. Great to be here.

CRAIG BOX: I've heard that your tech career can be blamed on the TI-83 silver edition calculator?

TAYLOR DOLEZAL: That is absolutely true. Who have you been talking to? [LAUGHS] I think it was around seventh grade. I think it was algebra. And my teacher said, OK, we've got a surprise for you today.

CRAIG BOX: It was unlikely to have been physical education.

TAYLOR DOLEZAL: [CHUCKLES] Oh, absolutely. Yeah, that would have been a strange class. Like, wait, we don't have to run around laps? And they brought them out, started showing us how to graph on the calculator. Very quickly, someone else said, oh, I know these. I've played games on these before. At that moment, that was the moment. And then found out that you could actually use Basic to program them, and then several years later, got into Java, and Python, and .NET. And I think Visual Basic 6 was probably the first one that I really started learning on, the label click and how to handle those events.

ADAM GLICK: You and I both come from the same part of the Midwest in the United States. And I'm curious, growing up there, what was your experience with the tech scene in Cleveland?

TAYLOR DOLEZAL: Tech scene in Cleveland was really interesting. I'd say it was mostly .NET or Java. There were a whole bunch of enterprises that really enjoyed those skills. But there was a point in time where I had talked with a recruiter. And they had said, those are the only two options that you have, and told me, you know, get in line when I asked about if there's anything like Python or anything like that available as far as a career.

And lucky enough, I found American Greetings, which was-- I'd say, honestly, still to this day, there are two people, Mike Prenaut and Dave Forgac, that really just are huge inspirations for me. They're still very active in the community, mostly around Python. And it was just one of the coolest places I've worked. So it exists. Open source was definitely a theme there.

ADAM GLICK: Yeah, the Northwest Python community, I believe, is the largest community and has the largest gathering of Python programmers in the world each year. That is a surprisingly large community in a place that people don't normally think of as the hub for tech in Python in specific.

TAYLOR DOLEZAL: It really surprises me, even seeing some of the projects that people make out of Cleveland. Even at Disney, I've met a few people like Elizabeth Moy, who works at Twilio, also born-- so I was born in Columbus. So that's where she had spent most of her time, and yet another Python developer.

CRAIG BOX: Everything I know about getting to Hollywood I learned from "The Muppet Movie." I'm sure you had a slightly different experience to Kermit the Frog. So how did you get from Cleveland to Hollywood?

TAYLOR DOLEZAL: That was a very interesting experience. I was working for the Cleveland Clinic beforehand in the medical industry, obviously. And we were focused on making consumer-facing products for Parkinson's, MS, and just other neurological functions, and basically, the end goal being, let's see if this medication or these treatments are working for you, and if you're getting better, staying the same-- you know, just getting that feedback. All that was previously being done on paper, which, as you can imagine, is quite a lot to sift through, literally and figuratively.

After we got a few products nailed down and working, those projects got more or less into a maintenance mode. And so after that, really just started looking. I had talked with my wife-- my fiancee’ at the time-- and we were like, do we want to stay in Cleveland? My wife said, you know what, we're going to be out in California in the next nine months. And I said, pff, OK, sure. In six months, we were out in California after kind of looking at jobs and seeing what existed out on the market. So really just kind of blew me away that that was even a possibility. And then it really just happened at breakneck speed. It was wild.

ADAM GLICK: You have a strong background in operations in the organizations you worked for, and probably most notably as an SRE for Disney. What was that like?

TAYLOR DOLEZAL: Incredible, honestly. It was a lot to take in day one, walking onto the studio lot. And we call that pixie dusting, when you would get trained on the first couple of days, which was a lot of fun. Having friends say, hey, are you free this weekend? And then saying, no, I need to be on call to support movie X that is coming out, or, oop, "Star Wars"-- sorry, I'm busy was a fun excuse, and completely tolerable by most friends and family. But that was really interesting. I worked mostly on the business systems that supported all of these movies.

So theaters typically get shipped a digital asset. They'll either get beamed the movie information, or they'll get it shipped so that it will actually be on a hard drive. That'll be encrypted. And it will come locked. And so movie theaters will buy an unlock code from Disney that'll be good for about a week. That's one of the big systems, as an example. That would ship out the unlock code. That movie theater would then use that to unlock their movie and then be good for seven days-- so keep getting that going. So that was pretty cool.

CRAIG BOX: Can we see you in any background shots of any movies we might have seen?

TAYLOR DOLEZAL: No, I tried. I actually had a meeting on the studio lot. So I worked in a satellite building at the time. But when I had meetings on the main studio lot quite a bit to meet with our stakeholders, things like that, there was actually some filming going on for-- I forget which TV show. And I almost walked into the shot. And they were like, whoa, whoa, whoa, whoa. So almost-- I tried, but no success.

CRAIG BOX: You went from a senior systems engineer role to a lead site reliability engineer role. I'm curious to know whether this was a change in what you did day to day. Did the team change focus? Or was this just Disney saying, all right, we're going to apply this title to the same role?

TAYLOR DOLEZAL: I had worked a while with that theatrical team, which was a lot to take in and understand in terms of-- distributed systems is really cool, obviously, going through that with college, and with some other firms, mom and pop shops, small, medium businesses. But at a Disney level of size, that becomes international. What are the concerns? What are the constraints?

So doing that for about two years, it was fun. It was a lot. And so I was talking with my manager at the time and said, hey, this is a lot of fun. I would like to kind of delve more into the SRE space. And I've really understood it at this level, but Disney doesn't just do that. Even within studios, there are other concerns like video encoding and residuals, making sure the actors get paid. And so with all of that broad focus, really wanted to jump on to the operations team and then move it more from a ticket-based type workflow to something that's a little bit more proactive, and something that we can actually start to measure, and work with people, not like, hey, my Wi-Fi doesn't work. Oh, sorry, we don't do that. That's not our team. Please go over here.

CRAIG BOX: Have you tried turning it off and on again?

ADAM GLICK: I was talking to someone else at Disney. And they told me that they give their developers really broad authority to choose what kind of tools, services they use and really kind of allow them as much freedom as possible to unlock their creativity in what they want to build. What does that mean for you as an SRE leader in having to support and keep those systems up and running?

TAYLOR DOLEZAL: That was my first gut response. I was like, yeah, just more to support-- that's no biggie. Each of the different sections have their own CTO. So Disney Animation, for example, has their own CTO. Disney Studios has their own, Pixar, Marvel, et cetera. And so there are a lot of good discussions that happen around that is that it allows the teams to pick what works best for them. Of course, there are things like at a corporate level where, here are a couple monitoring and metric solutions for us. And then they will provide that to the greater whole of Disney.

But it was nice to be able to talk through, we have these concerns and these constraints. What actually works for us, and be able to choose that. So that free reign, more or less, was great to have. Because it allowed us to solve problems a lot more proactively than we were previously able to.

ADAM GLICK: You must have seen a lot of different systems and ways of doing things. What did being an SRE there teach you about building great software?

TAYLOR DOLEZAL: Always be thankful it's not raining. It could always be worse. Getting to see things break, again, at that scale, was really enlightening. And what do you focus on? That kind of comes to mind and troubleshooting a lot of these things. So I'd be lying if I said we had zero SEV1 issues. And that's definitely where I learned the most was, all right, let's try this. OK, this doesn't work. Hold up, let's not change anything.

The teams are very good about-- they were working fast but slow at the same time in that, let's make sure this is actually causing this problem. Let's not just-- all right, just kick it. Distribute it. Just delete this queue. And I know this will make it work, so let's just do this, and then hope it never happens again. It was actually a little bit more diligent. And then getting good RCA process together was part of what I had worked on over there. So I really liked that, and getting that full download of the issue, and then talking with teams after. Like, hey, where can we actually make some changes that this won't happen again, or we'll be a lot more informed before it happens again?

CRAIG BOX: Do you think that the cloud-native stack that you were using at Disney, presumably at very large scale, is applicable to smaller shops as well? Or do you think the decisions you were making there were influenced by the scale that you were operating at?

TAYLOR DOLEZAL: That reminds me a lot of one of my favorite Twitter posts of all time. And it's this giant flatbed truck and a very small box that's strapped onto it. And it says, hey, I deployed Wordpress to Kubernetes.

[LAUGHTER]

CRAIG BOX: There's a little toy truck inside that box.

TAYLOR DOLEZAL: And inside that, tons of /wpadmin issues. So with the stacks that we'd used, Disney is a huge place. So again, lots of technologies were used. But I would say that most of what we focused on, a lot of where we got an advantage, was kind of uniting on, what are the things that we can agree upon? What are our primitives? Looking at working within Kubernetes was definitely one. There are some on-prem concerns and workloads that's like, we're going to support this until we don't need it anymore.

And then other things, we aligned on Kubernetes and kind of made that decision, I'd say, earlier on. Around maybe 2016, 2017 is like, this is the way we're going to move forward. This will allow us to talk with teams and people and agree that, hey, look, we all need storage. And we all need a place to run these applications. Here's a unified and a standardized way that we can actually talk about the same thing. And we don't have to worry about anything getting missed. And we can actually start to mature as a technical organization.

CRAIG BOX: Now you've recently moved on from Disney to a new role as a senior Developer Advocate at HashiCorp. First of all, that's a title that means many things to many different people. How do you define what you do?

TAYLOR DOLEZAL: I absolutely love my job. Because like you said, it means a lot of things to different people. And do we put developer advocacy in marketing? Or does it go over here? You know, where does that fit? It means a lot of things, same with SRE to a lot of different companies.

What it means at HashiCorp is sitting down with practitioners, and talking through the workflows, and finding out all of the pain points, and what it means to be an operator or be a practitioner. I love that I don't have to sit down and sell you on a solution. That's not our end goal. Our end goal is to really just figure out, we can't walk a mile in shoes as easily anymore. But let me sit in your chair for a day or two and understand what it's like to be you. Let's talk through your problems and figure out what would make that easier. How can we provide some better solutions for you?

So I'd say that's really what it means for me is getting to sit with people, and talk with them, and kind of work through things that might be difficult, as well as just answer any questions I can with them.

CRAIG BOX: I have the same title. And one of the things that is complicated, I guess, is that while the advocacy piece follows through, a lot of the people we're dealing with are, as you say, practitioners. They are perhaps operators. They're not necessarily people who associate themselves with capital D development. Do you find that in the tools that you're working on? And how do you talk to those people with that title?

TAYLOR DOLEZAL: Definitely. I'm mostly focused on Terraform. We have a setup at HashiCorp where it's like a T-based setup, where you have a good understanding of all the tools, but a very deep understanding of one. Obviously, Terraform made a lot of sense for me. It was a very good fit for me in terms of what I wanted to focus on. And so it is interesting to kind of see that convergence of operations and SRE and what that means for everyone.

Because we never really used to write code to spin up infrastructure. It was, install the agent, system D, this and that, configure this, set this up. Chef made a big difference for us. Ansible made a big difference for us overall. But even still, that's a huge question I get asked is, hey Taylor, behavior-driven development-- how do we do that? Test-driven development-- how do you do that with infrastructure? And I'm glad that there's things like Open Policy Agent and things like that you can use with Terraform.

I have a good colleague, Rosemary Wang, who actually gave a great talk about that at Velocity and kind of delved into each of those concerns. But it's just very fascinating to me in that we haven't solved that yet, as there's no just, oh, yep, we have a silver bullet over here, works every time. We don't have that yet. And I find that a fun problem to talk about, and even more fun to solve, work towards.

CRAIG BOX: What drew you to advocacy work? What do you like the most about it?

TAYLOR DOLEZAL: So at Disney, there were a few contexts where there was the opportunity to give a talk. And so I'm strange. I really enjoy vacuuming my house and family room. I love public speaking. So clearly, there's something wrong with me. I don't know what it is yet. But I'd say that was a big part of it, and then just really getting comfortable in sharing these ideas and thoughts, and just really taking the stance of, here's what I've learned.

Talking to me isn't something like, hey, look what I've learned. This is how you have to do it. It's about having a discussion. It's about creating a feedback loop and creating even more discussion. It's not a closed type of experience. And so I really loved doing that within Disney.

There were a whole bunch of internal talks. And took a lot of opportunities to get involved when I could for those. And then there were a couple chances submitting CFPs where I wasn't able to jump on those, unfortunately. It didn't work out, timing and whatnot. But being able to do that not only just within one company, but across several and in public was a huge one for me.

So that's what really drew me to advocacy, that and then sharing in that moment of the aha when people figure that topic out. You see that sparkle in their eyes where it all dawns on them that-- oh, I get it now. To me, there's nothing greater than that. That's why I do it.

ADAM GLICK: Along with your advocacy work, you're also the release team lead for 1.19. How did you first decide you wanted to start training to be a release lead for Kubernetes?

TAYLOR DOLEZAL: That came around the time of when we started using Kubernetes at Disney. And it was a lot of fun using that. I was in one of the first groups that ended up spinning that up and launching workloads onto it. And so it's fun to work with the new and shiny tool at the time. But after a while, once you start to get more competence with that and more comfortable with it, I took a look at that and said, you know what, this could actually be a lot of fun to work on. I had always wanted to break into open source. I knew that was something that I wanted to do, but never really knew what the right project was, kind of always felt uncomfortable around, like, where do I dedicate my time? What's the right thing to really invest in?

And Kubernetes was-- spoiler, it was a great choice. And so I took a look at that. It all started with wanting to be a CNCF ambassador. Because at Disney, it's a great company, but it's very difficult to talk publicly. You can't talk on behalf of the company. It completely makes sense.

But more times than not, if you go to ask to give a talk, they're going to say no, because that just makes the most business sense. And it just takes a lot of time and approvals. And by the time you get a talk approved, it might not be as relevant than if you gave it within a week or a month.

And so I was looking at the CNCF ambassador position as a good way to start talking with people and do so publicly. So I tried with that. It is very difficult to get in. They only accept about two people per month. And then there's obviously a very big backlog. And so unfortunately, I was not able to get that position for a while.

And then, I think, the 1.14 release for Kubernetes, those leads had emailed out of the shadow application survey. And I saw it. And I was like, oh, this isn't an end of the road for me situation. I can still get involved. This is a great way to do it. The release team? Oh, that sounds like a lot of work. But maybe this will work out.

And so submitted to that, got accepted to that. And then that was gangbusters after that. It was a lot of fun working with the release team. I started in the communications role for the release team sub-project and did that up until 1.16, took a break and 1.17, and then 1.18, got pulled in for the release lead shadow position. So that was kind of a slightly unabridged version.

ADAM GLICK: What's the process of leaving a release like, for those that haven't done it?

TAYLOR DOLEZAL: It's a lot. Get ready. It's really involved. But it's more and less than you would think at the same time. The community is really just so fantastic. This is open source. So these are people volunteering their time. These are people that want to do this.

And so these people that work on the releases are people that are passionate about seeing this happen and working on these things. So that's not something that I've found that you are guaranteed in a job. A lot of people have a job to get paid, or I'm doing it wrong.

CRAIG BOX: It's always a nice side effect.

TAYLOR DOLEZAL: When it comes open source, you have people that genuinely want to be there. And so there's that want to succeed and a want to do things the right way and have that community focus. So that made it really easy.

What makes it difficult, really, is just because it's open source, you aren't guaranteed that nine to five time slot with a lot of people. This is an international team. So communication is very difficult. It's always DNS. And that's very much the case here. Communication is difficult, but especially in a distributed way. But I think that we're getting a lot better with that as we go through each of these releases.

CRAIG BOX: What changed the most from the experience that you had as a shadow on the previous release to this one as a lead?

TAYLOR DOLEZAL: Really, the biggest thing that changed for me is visibility into what was going on, I think. When I worked with Jorge on 1.18, he was really good at showing how everything was done and all of the processes. But as much as you can involve people in shadows, even if you are really good at delegating, you still just don't get that level of visibility about people reaching out. Hey, where do I find this? Hey, has the release schedule changed? Hey, what's going on-- and stepping forward and helping out those people, and then just getting engaged.

At the end of the release, I'd say, is really when that happens. At the beginning, it's not as crazy. But at the end, it's really wild with all the things that are trying to get shored up and finished.

CRAIG BOX: Was there anything that you would say was unexpected?

TAYLOR DOLEZAL: Oh, always, always. I think at the end of the release cycle, typically, the go version gets changed and bumped up. And so that's usually, I think, the biggest wild circumstance that has happened so far. Obviously, 1.19 has been substantially different from any other release, mostly around COVID and just other socioeconomic things that are happening around the world.

And really, the focus with this one has been, let's not stress these people out. This is open source, like I said before. This is something where we would need to give people time. And this is the community-driven thing. If we stress out the community, if we put a lot on them and just don't treat them how we should as people, then it's really going to hurt the community. So let's give that time back. Let's focus on making sure everyone is in a good state, physical, mental, and what have you.

And so that, I'd say, was really a big change here, was that focus. And then that brought a whole set of new problems along, and solutions as well.

CRAIG BOX: Well, it was kind of you to give everybody KubeCon week off.

TAYLOR DOLEZAL: Absolutely. So many good videos to check out. And everyone really, honestly, needed the breather. It's a bummer for those which had to cancel their summer vacation plans and things like that. But hoping that they saw some good talks at KubeCon.

ADAM GLICK: What do you see as the major improvements that come in this release?

TAYLOR DOLEZAL: Some of the things I'm excited about for the 1.19 release are-- my personal favorites have to be we finally see Ingress go to stable. It's like, that's literally been a long time coming. So I'm very excited about that. And then the change of the support model to a whole year-- nine months is a pretty quick time frame for a lot of people to constantly keep upgrading Kubernetes. Working with large-scale systems and teams, I feel that, I absolutely do. I hear you. I've lived that. And it's nice to have an extra three months to adjust and get everything set up.

CRAIG BOX: The nine-month release support cycle was in large part because the tooling was able to deal with one release ahead and one release behind. And so there was sort of a natural three-month release cycle, and three of them. Has the tooling changed in order to enable this? Or is this just a decision that was made about how long the project needed to support each release for the users?

TAYLOR DOLEZAL: A lot of it was based around that same concern of time, and just like, does this make sense for the community? And a lot of end users were saying, hey, this is great. New and shiny is always fantastic. But we just don't have the time to be able to support things like this. And we really, again, heard the community. And that's where that KEP, the Kubernetes Enhancement Program, had come from.

CRAIG BOX: Why not, for example, 18 months, or go for the long-term release?

TAYLOR DOLEZAL: That was a big thing that we had talked about as well. And that was kind of fun, to read through all of the comments on GitHub around that. There was a lot of vacillation on what is the right time, even with the Kubernetes release. And I think that my favorite answer is that just is the one that felt most right was the three-month cycle for each of the releases for the year.

But really, it's an iterative process. And so that was the one that made the most sense in terms of being a year. And then obviously, if there's any troubles or pushback in the future, that's something that we want to hear about.

ADAM GLICK: Are there any hidden gem features or fixes that you think people should know about but aren't a part of the headline for the release?

TAYLOR DOLEZAL: That question is one I've had in my mind for quite a bit. Even this past week, I took a look at the features and tried to see, is there anything? And honestly-- promise this is not a cop out answer-- but it's really just about the process. The things that you don't see, there's been a lot of improvements on. So I'm really happy that we had Laurie Apple join us. So she became program manager for SIG release.

And so that's something we had not had before is someone who-- there were a lot of people that will assign themselves tickets or things like that. But we really didn't have any oversight, or any singular person or group helping us out with managing those. And so Laurie has been fantastic in calling people together. Let's talk about these things. What makes sense? Let's bump this up in priority-- and elevating all that we do. And so that's been a huge thing that, obviously, you won't see that in the release notes. But just the overall process improvements on how we actually make the release happen, and adjustments to documentation, and role books, and all the manual-based-- manual meaning the one you read-- based things around our repositories and process. So process, plus, plus is the TLDR.

CRAIG BOX: For many releases now, we've been tracking the status of a feature called sidecar containers, where you can specify that a container is a sidecar and should start before. I understand it's not in this release. What can you tell us about it?

TAYLOR DOLEZAL: That one has been a long road coming. That one caught my eye as well. I think it was last release that it had been scheduled to go out in the 1.18 release. And then there was the realization that there was a lot more work that needed to be done. I know it doesn't have a milestone right now in that specific issue just yet. But that's one I'm really excited to see come out as soon as it does too. Because there's a lot that we can do with that architecture pattern and that level of support.

But not too much I can say on it right now just because it's still kind of brewing and being formed. But I'm just like you. I'm really excited for that to come out as soon as it does.

ADAM GLICK: I saw on your Twitter that you announced that Jeremy Rickard will be the release lead for 1.20. What advice have you shared with him about how to be a great release lead?

TAYLOR DOLEZAL: The things I've talked with him shared with them so far are I made a few notable mistakes in this release, which I'm more than happy to talk about. Helped elevate everyone there. But really learning how to use Google Groups effectively-- I think, a couple times, I had a double signature in there. There was one time I forgot the subject line, and so got some playful comments by my team on that one. But again, iteration-- we can always get better. But I think that was a big one, really just that and a lot of the usual administrivia, being mindful of that, like scheduling meetings and making sure it works for everyone.

But I'm really excited to see Jeremy work on 1.20. He worked mostly on enhancements and things of that nature in previous releases. And so he just comes with a wealth of knowledge on how to run 1.20. And he was just a fantastic shadow to have. I had Mr. Bobby Tables and Jeremy as [shadow] release leads in 1.19, and they really helped with shouldering a lot of the duties and delegating.

It really felt like a team, honestly. I was the lead, but they really, really helped me out and made all of this possible. When I had something come up with COVID happening and different things being asked of me with changing jobs, just all of the things that happened, both of them were just more than ready to help out, jump in, run a meeting, talk through things, ask people questions. And I think that that's a big thing that I want to pass on to Jeremy too is that it is all on you, but it's not all on you. You can call on the team, and they're always more than happy to help.

CRAIG BOX: Now that you've moved to a job with a technology that is sort of adjacent to Kubernetes but not directly in the Kubernetes ecosystem, do you still see yourself being involved with the release process going forward?

TAYLOR DOLEZAL: Definitely, I'd say more than ever now. So being honest with you, that was a huge concern for me and something I was really thinking on was that there's Nomad at HashiCorp. There's Kubernetes. Are these things that are pitted to fight against each other? Or are these things that can complement and work with one another? And I've seen a lot of the latter, honestly. And the teams have been really enthusiastic about that as well.

There's never a one size fits all solution, especially with systems. And so getting to talk with teams about that-- HashiCorp joined the CNCF before I had started interviewing there. So that was a huge nod in the right direction in my opinion. And so I thought that was just really cool to see.

And I've seen more and more of that working at HashiCorp and seeing that people really do want to focus on that, and again, focus on the operators and the practitioners. And you can't avoid Kubernetes these days when it comes to infrastructure. So not fighting that battle and saying, no, you have to go over here, and really focusing on, like, how do we get this job done are the conversations that are happening. And so that's what I really like to see happen.

CRAIG BOX: You can keep wearing your CNCF ambassador cape.

TAYLOR DOLEZAL: That I'm really excited about. And I think they just gave us the flashing crowns as well. I've put that on once this week. And then I think-- sadly, I think the batteries have run out. So I need to replace those.

ADAM GLICK: That's what they get for using incandescent light bulbs. Finally, I hear that you're an avid reader. Any recommendations you'd like to make for people looking for a good book?

TAYLOR DOLEZAL: Oh my goodness, yes. So Stripe Press has been just so fantastic within the past couple months. So they have had-- the Working In Public book has been one of my personal favorites, talking about just the open source community and a lot of observations there. How can we do more with less? And just, how can we enable people developing software to continue doing that and focus less on, again, meeting invites, and calendar updates, and things like that? You know, how can we automate that? So that was a good one. It's called "Working in Public-- The Making and Maintenance of Open Source Software."

Stripe Press had another great one released last year called "An Elegant Puzzle-- Systems of Engineering Management." And then finally, one that came out around my birthday this year was "The Art of Doing Science and Engineering-- Learning to Learn." So all those have been really, really fun reads. If you're not looking for something in the technology space, I think "Defending Jacob" was the last non-technical book that I read. Because it had come out on Apple Plus. But yeah, that was also something where the book and the film were a little bit different. But you know, I can appreciate both endings.

CRAIG BOX: Well, some great recommendations there. And thank you very much for joining us today, Taylor.

TAYLOR DOLEZAL: Thanks for having me.

CRAIG BOX: You can find Taylor Dolezal on Twitter at @onlydole.

[MUSIC PLAYING]

CRAIG BOX: Thanks for listening. As always, if you've enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on Twitter @KubernetesPod, or you can reach us by email at kubernetespodcast@google.com.

ADAM GLICK: You can also check out our website at kubernetespodcast.com, where you'll find transcripts and show notes as well as links to subscribe. Until next time, take care.

CRAIG BOX: See you next week.

[MUSIC PLAYING]