KASLIN FIELDS: Hello, and welcome to the "Kubernetes Podcast" from Google. I'm your host, Kaslin Fields.

ABDEL SGHIOUAR: And I am Abdel Sghiouar.

[UPBEAT MUSIC]

KASLIN FIELDS: In this episode, we interview the Kubernetes 1.27 release lead Xander Grzywinski, we'll explore some of the highlights of the release, and discuss a bit about what it's like to work with the release team. But first, let's get to the news.

[UPBEAT JINGLE]

ABDEL SGHIOUAR: SIG Security within the Kubernetes community said that the Auto Refresh in official CVE feed is in beta now. The programmatically-accessible feed allows users to subscribe to the list of CVEs announced by the Kubernetes security response committee. The new version introduced some improvements to the JSON and RSS feeds, in addition to freshness metadata, and ordering.

KASLIN FIELDS: VMware announced the general availability of their Tanzu Mission Control for Amazon Elastic Kubernetes Services. The product allows users to create, update, and scale EKS clusters using the native Amazon API directly from Tanzu.

ABDEL SGHIOUAR: Red Hat announced the general availability of OpenShift virtualisation. This add-on is available on OpenShift 4.12, and allows users to manage virtual machines from OpenShift using KubeVirt in a declarative manner.

KASLIN FIELDS: Azure announced long-term support for Kubernetes on AKS, or Azure Kubernetes Services. Starting with Kubernetes 1.27, Microsoft will offer two years worth of support and security updates for AKS clusters, instead of the usual one year offered by the project.

ABDEL SGHIOUAR: Google Cloud turned profit for the first time in its history. On the earnings call for the first quarter of 2023, Google turned around $190 million of profits for Cloud Services. The total revenue increase for this quarter was 28% higher than the same quarter of last year.

KASLIN FIELDS: The CNCF announced a new CFP platform for upcoming events. The migration from smapply, or SM-apply, to sessionize was motivated by the record number of submissions received for KubeCon EU 2023, which was about 1.7 thousand, in addition to the 16 community days organized in 2022. The new platform, according to the CNCF, should enable them to better handle the proposals for future events.

ABDEL SGHIOUAR: The CNCF, in partnership with Chainguard, performed a software supply chain security assessment for two graduated projects-- Argo and Prometheus. The assessment was based on the supply chain levels of software architect, or SLSA, version 0.1. They concluded that Argo meet SLSA Level 3 for audibility of source and integrity of the provenance, while Prometheus only meets Level 3 for the same framework for source and build, but Level 0 for provenance. Reports for both projects can be found in the show notes.

KASLIN FIELDS: The next edition of KubeCon EU will take place in Paris from March 19 to 22, in 2024. The CNCF also announced that KubeCon is returning to APAC for the first year since the pandemic on September 25 to 27, in 2023, in Shanghai, China. The CFP for both KubeCon NA and KubeCon Shanghai are open, with both closing June 18.

ABDEL SGHIOUAR: Traffic Labs adds API hub for Kubernetes. The new API hub makes it simpler to publish secure and managed APIs on Kubernetes. This SaaS offering is available in closed beta, but the company aims to make it GA on May 16.

KASLIN FIELDS: The CNCF announced the Spring 2023 cohort of the CNCF Ambassadors program. The revamped edition of the program saw 155 ambassadors accepted, representing 37 countries, and 125 companies. And that's the news.

[ELECTRONIC JINGLE]

I'm Kaslin Fields with the Kubernetes Podcast from Google, and I am live on site with Xander Grzywinski. Xander, would you like to introduce yourself?

XANDER GRZYWINSKI: Yeah, I'm Xander. I was the 1.27 release lead, and have been involved in the project, and SIG Release, for a while now. And outside of that, I am a senior open source PM with Microsoft.

KASLIN FIELDS: So you do quite a few things.

[LAUGHTER]

Though today, we're mainly going to be focusing on the 1.27 release, which has the theme of chill vibes. Definitely a fan of this. I like that-- someone pointed out this week, while we've been at KubeCon, that it's titled Chill Vibes. And I think a lot of folks are feeling that here in Amsterdam this week in April.

XANDER GRZYWINSKI: Yeah. Yeah, it's a fun theme. I think I started thinking about it after we did the enhancements freeze, which for those of you that don't know, is a milestone that we have in the release process. It occurs, like, 3 to 4 weeks into the release, in which we kind of lock down the features that are going to be in the release. And generally, as part of that, there are a number of features that typically fall behind on the criteria needed for that lock-in, and then there's an exception process.

I've been on the team for over three years now, and I've spoken to folks on there longer. This is the first release anyone can remember where we didn't get a single exception request. Like, every single feature that put themselves up for the release met all the criteria, and things just kind of flowed really smoothly from there.

KASLIN FIELDS: Impressive. I wouldn't expect that to work out.

XANDER GRZYWINSKI: Yeah.

[LAUGHTER]

KASLIN FIELDS: I think a lot of folks listening are probably familiar with similar processes of having to cut a release, essentially.

XANDER GRZYWINSKI: Yeah, and we actually had more features in this release than is typical as well. So not only see the increase in number, but then not have a single exception request come through, like, it was pretty spectacular.

KASLIN FIELDS: And also, speaking from an open source perspective, my work and contributor experience, it's always a tough time to get a hold of the heads of the SIGs.

[LAUGHTER]

XANDER GRZYWINSKI: Yeah.

KASLIN FIELDS: Stuff like that.

XANDER GRZYWINSKI: Yeah--

KASLIN FIELDS: People are always busy.

XANDER GRZYWINSKI: --things going on or anything. You know, they're so busy. They're doing so much. And I always kind of feel like a dweeb. Like, hey, can you update your KEP? We just need this little PR for documentation, because they're doing so much. And like, it's truly astounding work.

KASLIN FIELDS: Yeah. So, what are some of the new features that are in this release?

XANDER GRZYWINSKI: There's really a lot. And I think I saw this on Twitter. I saw someone refer to it as like the first big, boring release. Because it's like, it's full of features, but also it's like-- it's all stuff that just is kind of making things more stable, and building on the existing functionality. And I think that's just kind of where we are in the Kubernetes project right now.

But there's a couple that I think-- the vertical in-place pod auto-scaling entering alpha, which has been talked about for a long time, but finally reached the alpha state with this release is the one that I think I'm personally most excited about. I think that's going to be cool for a lot of people.

And then there's some cool Storage stuff, too. Like, that's an area, I know, that's had a lot of spirited debates in the past, on whether you should put state on Kubernetes. But I think SIG Storage is just full of so many brilliant engineers that are doing amazing things to take those storage primitives, like, further and further, and make them more stable. And we're seeing some of that with this release, too.

KASLIN FIELDS: I'm really excited about those areas as well.

XANDER GRZYWINSKI: Yeah.

KASLIN FIELDS: I've explored the vertical pod auto-scaler a little bit on my own, and it's definitely been a bit confusing.

XANDER GRZYWINSKI: Yeah.

[LAUGHTER]

KASLIN FIELDS: So I'm excited to see what this new release holds for that. And also, I was just giving a talk about Stateful, and I'm trying to get more into that space. So I'm curious to hear about that more as well.

XANDER GRZYWINSKI: There's some cool stuff there. And like I said, that SIG is just brilliant. Like, it's a good exercise to just go and read some of their KEPs and get a good idea of what they're working on. It's all really cool.

KASLIN FIELDS: So that's a couple of the new things. Are there any depredations or removals in 1.27?

XANDER GRZYWINSKI: Yeah, there actually-- there's a whole list of stuff that kind of meets that criteria. And we started publishing a Deprecation and Removals blog ahead of the typical Release blog, which is a good thing to go check out. We were trying to just communicate that stuff earlier.

And then the one that everyone has absolutely heard about, and is almost certainly sick of hearing about, and yet I'm going to keep talking about it, is the registry change.

KASLIN FIELDS: We've all got to keep harping on that one.

XANDER GRZYWINSKI: Yeah, the k8s.gcr.io cut over to registry.k8s.io. So that's the one that we really tried to overcommunicate throughout release, to ensure that users are not impacted by that change in a negative way.

KASLIN FIELDS: Yeah, and it has a lot of opportunity to also help users. It provides performance enhancements, theoretically-- should be for a lot of users.

XANDER GRZYWINSKI: Yeah, yeah. In the past, images were being pulled from a central repository. And now we've got a more geo-distributed setup, and it's also a chance to kind of distribute the cost of running the infrastructure for the project across multiple cloud providers. And it really is, like, a way for the community to come together, and support the project as a whole.

KASLIN FIELDS: Yeah. If you haven't checked out any of the information about that change, or how it was achieved, definitely very interesting stuff. But coming back to deprecations and removals, are there any specific ones that you want to call out?

XANDER GRZYWINSKI: Beyond the registry change, no. I don't actually know off the top of my head which ones are on there. I think-- I don't remember where Pod Security Policies are in the pipeline at the moment, but that's the big one--

KASLIN FIELDS: Right, they're kind of awkward. [LAUGHTER]

XANDER GRZYWINSKI: --that I know would bite people if they don't keep an eye on migrating over to Pod Security Admission, and-- yeah.

KASLIN FIELDS: Yes. So if you've been keeping an eye on that space, or if you haven't, Pod Security Policies were deprecated a couple of releases ago. Yeah,

XANDER GRZYWINSKI: I want to say like three releases ago, four releases ago-- somewhere in there. I've been on the team since, like, 2019, so it really does all kind of blend together at this point.

KASLIN FIELDS: Yeah. And to clarify the terminology-- I didn't explain that earlier, but deprecated means that it's going to be removed at some time in the future. Removed means that it's not there anymore for you to use.

XANDER GRZYWINSKI: Right.

KASLIN FIELDS: So Pod Security Policies are still in a deprecated state, I think. Though I have to check.

XANDER GRZYWINSKI: Yeah. I guess I would have to check, too. I know if they're not yet removed, it would be soon. Yeah.

KASLIN FIELDS: We'll have to include a link to that blog post in the show notes.

XANDER GRZYWINSKI: Yeah, that'd be great.

KASLIN FIELDS: Yes. And then they're being replaced by the new Pod Security Admission?

XANDER GRZYWINSKI: Pod Security Admission, yeah, is the new thing to replace. And then there's also-- I think a lot of folks end up moving to things like Open Policy, and Gatekeeper, and Kyverno, those, like, dedicated policy tools, too, some people tend to rely on.

KASLIN FIELDS: Yeah, policy is a very interesting area that folks are working on right now. I feel like we're talking about a lot of interesting areas. We're talking about auto-scaling-- [CHUCKLE] and , actually when we were talking about the vertical pod auto-scaler, we didn't explain that much about what it does, did we?

XANDER GRZYWINSKI: I don't think so.

KASLIN FIELDS: So maybe we should do that.

XANDER GRZYWINSKI: Yeah, it's pretty cool. So when you deploy a pod, you set the resource requirements of the pod. And this is a feature that's going to allow you to auto-scale that without actually having to completely remove the pod, and like edit it, and redeploy. So in the same way that you have the typical scale out horizontally through replicas, now you will be able to have auto-scaling for the resources of a single pod.

KASLIN FIELDS: And that's something I'm always harping on, set your resource requests and limits.

XANDER GRZYWINSKI: Yeah, yeah. It is a little confusing how those actually work under the hood, but it is worth like taking the time to dig into that documentation, and really get that right.

KASLIN FIELDS: So you can run things on Kubernetes without even caring, or knowing that resource requests and limits exist at all. But if you do know that they exist, and you start using them, then you have much better control over the actual resource usage of your clusters.

XANDER GRZYWINSKI: Yeah, and just running things, it works great until you start to pack your nodes pretty full. And then things can really start to fall apart.

KASLIN FIELDS: And the whole point of Kubernetes is managing workloads at scale.

XANDER GRZYWINSKI: Right.

KASLIN FIELDS: So it's really important to manage those resources at the kind of scale that Kubernetes often reaches.

XANDER GRZYWINSKI: Mm-hmm.

KASLIN FIELDS: So there was also a KEP, I hear, that was rather interesting in the 1.27 release, KEP 753. And it has something to do with sidecar containers? I'm honestly not familiar. Would you tell me a little bit about it?

XANDER GRZYWINSKI: Yeah, I can talk a little bit about what I know about that one. So one of the consequences of being the, like, all-up release fleet is that we had 60 KEPs, that made it to the actual release in this one. So my familiarity for all of them is only surface-level. I'm more herding cats throughout the process, really.

But this one, defining like sidecars as a specific type of container that the kubelet would handle in a different way, and prioritize in a different way. Sidecars, I think, have always been kind of like a contentious-- like, there are a lot of people that don't like the pattern, and the ways that it's used. But I know that there are a lot of folks that are excited for this one.

And maybe going forward, if you can get the kubelet to treat sidecar containers in a specific way, that might actually help support the pattern a little bit better, and make it a little more flexible.

KASLIN FIELDS: Yeah, sidecar containers are definitely a very popular pattern that I see. I often give Intro to Kubernetes talks, and one of the things I talk about in those is just explaining what a pod is. And so I always explain that you can have multiple containers in one pod, but the most common way that folks do that is with a sidecar container. So it is very common. I'm curious to see how this would impact things, especially because Kubernetes doesn't usually care about things at the container level too much.

XANDER GRZYWINSKI: Right.

KASLIN FIELDS: We usually keep things at the pod level. So I can see why that might be a little contentious.

XANDER GRZYWINSKI: Yeah. Yeah, and you do see things, like some service mesh tools moving away from the sidecar pattern and such. But it's certainly not going anywhere. Like, it's always going to be a requirement, to some degree.

KASLIN FIELDS: Mm-hmm. There are a lot of benefits that sidecar containers can get you. So I'll be curious to check that one out. And we'll include a link to that Kubernetes Enhancement Proposal, KEP, in the show notes, if you want to read more of the details and the discussion that's going on around that. So what is the relation between that KEP and the release? Is there something going on with it in the release, or just a discussion that's been very prominent lately?

XANDER GRZYWINSKI: So that one, I know it had a graduation, this release. And I think--

KASLIN FIELDS: What does a graduation for a KEP mean?

XANDER GRZYWINSKI: So when a feature is introduced to Kubernetes, it is introduced at the alpha stage for the API. And then at some point, after some number of release cycles-- sometimes one, sometimes more-- it will graduate from alpha to beta, and then from beta to stable. Currently, there are three stages for those APIs.

And so the sidecar KEP had a graduation this release. I don't remember off the top of my head-- I think it was to beta, but I'd have to double check. But the significance there is that, with alpha features and APIs, those have to be enabled by a feature flag when running Kubernetes. And then beta features are on by default. So you don't have to utilize them, but they are on. And so that ends up being kind of a significant milestone, because then for all of the managed Kubernetes services through cloud providers, you then have access to the beta APIs.

KASLIN FIELDS: OK. All right. I guess folks will start learning a lot more about that one, potentially, then, in the 1.27 release. So as the release lead, what were you most excited about in this release?

XANDER GRZYWINSKI: Yeah, I think the vertical pod auto-scaling, which we mentioned, was one big point of excitement for me. Two, just the theme in general. Like, it made my job a lot easier, really, to not have to worry about all the exception requests and such.

And then I think working with the team was just really wonderful. We have a awesome group of people that volunteer to help out with the release. And so just getting the chance to work with a global community of amazing people is just a really cool opportunity. And that-- as long as I've been involved with the release, continues to be one of the best things about being a part of it.

KASLIN FIELDS: And we're already past the point that the release came out. 1.27 came out on April--

XANDER GRZYWINSKI: 11th. So it was last Tuesday, yeah.

KASLIN FIELDS: From when we're recording.

XANDER GRZYWINSKI: Yeah.

[LAUGHTER]

KASLIN FIELDS: So we're already past that point, which means that the next release cycle is already starting up. So it might be a good opportunity for us to talk a little bit more about the release team, and how that's structured. And I know shadow applications for 1.28, I believe, are already open.

XANDER GRZYWINSKI: They are, yeah. So we have a structured shadow program as a way to onboard new contributors to the project. So applications for that are open. It's a really great way to get involved in the project and the community. And I think-- I don't know, what's special about it to me is people of all backgrounds and skill sets can really bring something to that team.

So on the release team, you know, we're not really writing code. It's more process-oriented. And so it provides a great opportunity for folks that maybe were put off from open source, thinking that it was exclusively code contribution, to really be able to step in and add valuable contributions. If you are someone that is more oriented towards like deep into the code, the release engineering team, as part of SIG Release, is always looking for help, too.

And I mention it because the shadow program, at this point, has grown so much that it receives hundreds upon hundreds of applications every time we open it. And it's really hard to choose people, because we just get so many awesome applicants. And so if you are, yeah, code-oriented, I'd love to steer folks towards the release engineering team as well. Because they always need help, and it's a great group of folks to work with, too.

KASLIN FIELDS: Yeah. If you are going to try to apply to the shadow program, it's probably pretty helpful to have a rough understanding of what the roles are that you could be shadowing for.

XANDER GRZYWINSKI: Yeah. Yeah, so we have the enhancements team, which they are the ones that are kind of tracking all of the features that are going into the release, and making sure that everything is in place, in terms of just, like, documentation and organization for the release process. And then there is the CI signal team, which is watching all of the dashboards throughout the release to make sure that we're seeing mostly green colors, and things are working as expected.

And they're the ones that kind of give the go and no-go on each release throughout the process. There's also Docs and Release Notes, which are two teams, but they work quite closely together to make sure that the website documentation is ready to go for all of the new features, as well as just the actual release notes themselves. And then communications, which-- what we're doing right now.

The communications team ends up, making sure that all of the blogs are in place, and then organizing interviews like this to kind of communicate some of the new features. And then last, but certainly not least, would be the Bug Triage team. And so they are tracking any items that may prevent the release from getting out the door on time. Like, things that would be release-blocking bugs-- getting those things cleaned up, and taken care of even passcode freeze.

KASLIN FIELDS: So the bug tracking team must work with the other SIGs quite a bit.

XANDER GRZYWINSKI: Yes. They are working very closely with all of the SIG leads.

KASLIN FIELDS: Excellent. And an interesting thing about the Release team is, you see a lot of folks in Release who are not super deep Kubernetes contributors. It's a great way for folks, even who are not deep into the community, to get involved.

XANDER GRZYWINSKI: I myself have never, never opened a PR against Kubernetes, but I've been doing the Release team for over three years now. And, I don't know, it just goes to show that folks of all different backgrounds and skill sets can bring valuable contributions to the project.

KASLIN FIELDS: It's amazing how we can distribute that work. It's so valuable to be able to have so many different people working on such important parts of the project.

XANDER GRZYWINSKI: Yeah. I mean, at this point, there are, I would say, almost no people that have a full understanding of the entire project. Like, it is very distributed, and by necessity at this point.

KASLIN FIELDS: Yeah. So that's a little bit about the Release team. We've talked a little bit about 1.27. For our listeners out there who maybe have to administrate to Kubernetes clusters for their own work, or for their home-- whatever they may use Kubernetes for-- why should they upgrade to 1.27?

XANDER GRZYWINSKI: I know upgrading is a tough point for cluster operators. Especially with Kubernetes releasing three times a year now, it can be tough to keep up on that. I think the more that you stay up-to-date on that, the easier the upgrades are. And so, I think it's worth putting in the time up front to try and stay up-to-date.

But it gets particularly important when you get past that one-year support window, where patch releases are no longer included. So folks that are on, like, 1.23 especially, and before, would really want to consider upgrading so that they maintain those patch releases.

KASLIN FIELDS: Excellent. Thank you so much for being with me today, Xander.

XANDER GRZYWINSKI: Yeah, thanks for having me.

[UPBEAT JINGLE]

ABDEL SGHIOUAR: Well, thank you, Kaslin, for the interview. That was actually pretty cool.

KASLIN FIELDS: Yeah, I had a lot of fun talking with Xander at KubeCon.

ABDEL SGHIOUAR: Yeah. And I know, actually, that it was kind of hard to schedule the time, because it was going to happen, and it was not going to happen. But at the end, you managed to get your hands on Xander at KubeCon?

KASLIN FIELDS: Yeah, we ended up finding some time. And it's also kind of hard to find a good space for recording an interview at KubeCon because, of course, we're all very busy while we're at KubeCon doing talks, and connecting with people that we work with in the open source community, and things like that. So it took until the last day of the conference, but we managed to work it out.

ABDEL SGHIOUAR: Yeah, yeah. By the way, I learned something at KubeCon, because we are using the same set of microphones to do the recording when we do events recording, right? It's like the wireless Rode microphone. I figured that there is actually a way where you can turn it into a handheld microphone, so you can buy like an attachment--

KASLIN FIELDS: Yeah, I really want to buy that.

ABDEL SGHIOUAR: Yeah, I want to buy that as well, because it makes you look like a real journalist.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: All right. So there was actually a bunch of things at the interview with Xander that I want to dive into. And I think-- like, starting with that theme, Chill Vibes. That's a pretty interesting name.

KASLIN FIELDS: Chill Vibes. It is pretty great. A lot of folks I talked to on-site thought it was particularly relevant with KubeCon being during the week that it was in Amsterdam.

ABDEL SGHIOUAR: I mean, I have to admit that this probably was one of my favorite KubeCons since I've been going to KubeCon. The venue was great. I love Amsterdam, so it was pretty nice.

KASLIN FIELDS: Yeah. And so I talked with Xander about that, and he didn't really give much insight into where the name came from.

ABDEL SGHIOUAR: OK. The logo is pretty cool. It's like a little bear-- hugging bear, I think. Or panda, or whatever.

KASLIN FIELDS: His story for it was just like-- I felt like what I got out of his explanation of it was that that was his wish for the release, was to focus on the chill vibes, rather than those stressful parts of the release.

ABDEL SGHIOUAR: OK.

KASLIN FIELDS: So I feel like that was his wish for the release.

ABDEL SGHIOUAR: It is the Release lead that chooses the name, right?

KASLIN FIELDS: Exactly, yeah.

ABDEL SGHIOUAR: OK. Choose the theme, basically. And--

KASLIN FIELDS: Yeah, they have so much fun with it.

ABDEL SGHIOUAR: Yeah.

KASLIN FIELDS: It's always great to ask them about it.

ABDEL SGHIOUAR: Yeah. I'm looking forward to have the sticker of the logo on my laptop.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: Because now we have the 1.26, which is Electrify. We got that from Leonard.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: So I think we have to keep this as a habit. Each time we get a Release lead, we have to get them to get a sticker. That's how it works-- that's the deal.

KASLIN FIELDS: I'm going to need a special book or something to keep those in.

[LAUGHTER]

ABDEL SGHIOUAR: Or I'm going to need a bigger laptop. My laptop is getting full of stickers. There was a bunch of other interesting things that came out in this-- I mean, it doesn't feel to me like it was very chilled. There is a lot of changes-- there have been a lot of changes, actually, in 1.27, right?

KASLIN FIELDS: It's an interesting name for a release. I think that it had one of the highest numbers of changes of any release, actually.

ABDEL SGHIOUAR: Yeah, 60 enhancements or something?

KASLIN FIELDS: Yeah, it was a lot.

ABDEL SGHIOUAR: Which is crazy, yeah. And so one of the ones that I wanted to dive into a little bit, because I have a little bit of context, is KEP, which stands for Kubernetes Enhancements Proposals, 753, for sidecars containers.

KASLIN FIELDS: Right. We talked about that a little bit with Xander, but he wasn't super familiar. I think you have more context to add. Right, Abdel?

ABDEL SGHIOUAR: Yeah, pretty much. So essentially, what that improvement proposal was about is to make the kubelet handle sidecars different than they handle regular containers. And I can give you a few examples. But the main one, for example, is service mesh. Xander's talked about service mesh very briefly.

KASLIN FIELDS: He did.

ABDEL SGHIOUAR: It is described in the enhancement proposal, but here is the use case. So when you use a service mesh which uses sidecars, like issue, for example, if your container is starting with a sidecar, before the sidecar, which is the proxy, is available-- so before the sidecar is fully ready, your application cannot make any network calls. So basically, your application is stuck until the sidecar is available.

Which means if, for example, as part of your bootstrap process for your application, you are talking to an external API, or you're doing some sort of authentication, you can't make those network calls, right? And so that proposal has been to say if the kubelet can handle the sidecar different than the container, then it could potentially, for example, define or start the containers in a specific order. So start the sidecar first, wait until it's available, and then start the actual application container.

KASLIN FIELDS: So it's all about the ordering of how the containers come up, which makes a lot of sense for the kubelet to have some hand in.

ABDEL SGHIOUAR: Exactly.

KASLIN FIELDS: And it also sounds like it might have some relationship to health checks, because you want to make sure that your health checks are passing by, making sure that those are up in the right order-- and they can actually communicate.

ABDEL SGHIOUAR: That's also another one, yes.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: Yeah.

KASLIN FIELDS: Because health checks-- a big part of it is making sure that the application is ready to communicate.

ABDEL SGHIOUAR: Correct, yeah.

KASLIN FIELDS: So--

ABDEL SGHIOUAR: It's basically making sure that the sidecar does not break network calls for the application itself. And it's not only for service mesh, by the way. It works for-- sometimes you have a use case where your sidecar is just like a log aggregator, or like a log router. So your sidecar is just collecting the logs from the container of the app, and then sending them somewhere.

So in all these cases, that's essentially what you want to do. You want to be able to say my sidecar starts, first get to a ready stage-- whatever ready stage means for that specific sidecar. And then you can start the application itself-- the container of the application.

KASLIN FIELDS: OK, cool. So that sounds like a pretty useful update, which there's more information on in the blog, right?

ABDEL SGHIOUAR: Yes.

KASLIN FIELDS: And of course, we'll have more information in our show notes.

ABDEL SGHIOUAR: All right. So that's pretty cool. I think the other one, obviously, is like the k8s.gcr.io freeze, which continues to be a thing. We have talked about it many times.

KASLIN FIELDS: It's going to be a thing until something else is in place, and everything goes haywire. But hopefully, folks out there are listening to the news. Please change all of your references.

ABDEL SGHIOUAR: Yeah. So one more thing I wanted to get your opinion about, which came up, actually, in the news that we recorded for this episode, it's that Microsoft announced an LTS support for Kubernetes, where they will be announcing-- or supporting, I think, older versions of certain versions for two years, instead of the standard one year that the community offers.

KASLIN FIELDS: Right. That was a big thing for the contributor community to be talking about at KubeCon this year, at the contributor summit. There was a-- kind of an unconference session. We have an unconference at the contributor summit, where folks can submit their own topics. So that was one of the ones that was talked about there, since it was a very new thing, and folks are still kind of trying to process it.

Definitely, folks had very strong opinions. The community is paying attention, and looking at what this might mean for users, and also for the Open Source Project, and things like that. Personally, my favorite take that I've heard about it is one from one of our colleagues, Nick Eberts. I was talking with him about it, and he made a really good point that upgrades are just so painful.

ABDEL SGHIOUAR: Yes.

KASLIN FIELDS: The point of a long-term support would be we've got all of these releases coming out all the time. It's way too much for anyone to realistically keep up with.

ABDEL SGHIOUAR: Yes.

KASLIN FIELDS: I think we can all admit. [LAUGHTER] And also, what I see realistically in the field is that folks don't upgrade their clusters in-place. They create a new cluster, and they move everything. And when I was talking with Nick, he was saying that he thinks that should be the regular, best practice kind of pattern.

Because everything running in your cluster should be configured such that it's not dependent on the cluster that it's running on. Kubernetes is the platform, but all of the stuff running on top of it, you should be able to move it to another Kubernetes cluster without much hassle-- theoretically.

ABDEL SGHIOUAR: Yeah, yeah.

KASLIN FIELDS: So he thinks that the process for upgrades should be that you create a whole new cluster, and you transition things over to the new cluster-- instead of upgrading in place, which I think makes a lot of sense. Rather than long-term support, which, he argued, kind of puts a Band-Aid on the problem of we're going to support you a little bit longer, which just gets you more out-of-date.

ABDEL SGHIOUAR: Exactly.

KASLIN FIELDS: And that's one thing the community was talking about. If you are on these long-term support channels, then you're on this one version for longer. What is then your upgrade path? What is that going to look like you? So that's something folks are still figuring out.

ABDEL SGHIOUAR: Yeah. And whether, for example, you would be able to skip version--

KASLIN FIELDS: Skip upgrades.

ABDEL SGHIOUAR: --go from 1.27 to 1.29 directly, or do you have to go to the 1.28?

KASLIN FIELDS: Yeah, that was one of the major conversations happening at the contributor summit.

ABDEL SGHIOUAR: Yeah. I remember we had a brief conversation about it. We were heading out for dinner at KubeCon, and we talked about it with Nick. I took some time-- a little bit-- to reflect on it. And I think, first of all, I'm not surprised, right? Microsoft have a long history in supporting software. They know how to do it.

So announcing LTS is not really something new for them, I guess. The way they would describe it in the blog is we're going to clone or fork, actually, the version. So they're just going to keep an internal version of the code base. So, that's going to be interesting.

I think, personally, the right way to do this is to have people focus on making upgrades and non-events.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: Because the core of the problem is that upgrades is an issue for people, because it's disruptive, and it could potentially make your workload go down, and stuff like that. And if I put, just briefly, my ex-consultant's hat on, I would say, well, just focus on making upgrades not an issue for you as a customer, as people using the platform-- which is hard to implement, because not everybody is skilled, and have time, and have engineering resources, and stuff.

And so that's why I think cloud providers and the community should actually think about how can we make upgrades a non-problem. Either, as you say, making the application-- I still disagree on the fact that application should not be dependent. It is, because API version changes all the time. And that's one of the major problems with upgrades, right, is if an API goes from alpha to beta, from beta to J, like, you should be able to catch that, and update your deployments, right?

So I don't know how this could be implemented. We do it on GKE. When you're about to upgrade your clusters, we have like a little interface that tells you, hey, like these APIs are going to be deprecated, or deprecated, or they're moving from one stage to another, right?

KASLIN FIELDS: Yeah, it's nice to find that out through checks rather than through I tested it, and now it doesn't work. [LAUGHTER]

ABDEL SGHIOUAR: Exactly, exactly.

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: So I see where Nick's opinion, and your opinion, about trying new cluster comes from, because essentially you're going to be able to catch issues before they become an issue-- by trying to deploy your application on your version. And then if it doesn't work, you can go check documentation, and update your application accordingly.

Yeah, I think it's a very hard problem to solve. I think making upgrades not an issue is probably the right direction. What that means, how it's going to manifest itself, is TBD, I guess.

KASLIN FIELDS: Yeah, that's probably too big of a problem for anyone to tackle head-on. Everyone's tackling it from different aspects of the upgrade process. And maybe eventually, it'll be better than it is today. But we just got to kind of work with it in the meantime.

ABDEL SGHIOUAR: Exactly, exactly. We'll see.

KASLIN FIELDS: So one other thing that I wanted to go over was the conversation that I had with Xander about vertical pod auto-scaler. So the vertical pod auto-scaler is something that I've taught some like workshops on and stuff, because understanding all of the different ways that things can auto-scale in Kubernetes is very important, since it is all about running things at scale.

But the VPA is very interesting because it is open source, but it comes from a lot of the stuff that Google has done for our own managed Kubernetes service, for GKE. So it came out of those efforts. And so for quite a while, it's been something that's in open source, but none of the other providers have really supported it.

ABDEL SGHIOUAR: Yeah.

KASLIN FIELDS: And it also has had some challenges, one of which is getting addressed in the 1.27 release. It's not in the release blog.

ABDEL SGHIOUAR: Yeah, we find out. We find that the hard way.

KASLIN FIELDS: So it's something that Xander talked about a lot as being something that's really interesting. Hopefully other folks are actually using the VPA now, because Xander is at Microsoft. So anyway, the vertical pod auto-scaler, traditionally it's been kind of difficult to use the vertical pod auto-scaler as an auto-scaler.

ABDEL SGHIOUAR: Yes, in auto mode.

KASLIN FIELDS: Yeah. The recommendation has been to use it in manual mode, which, basically, it then gives you recommendations of here's how you should be setting your resource requests and limits. It also has the ability to set those resource requests and limits for you, but that's been disruptive. And what's changing in 1.27 is that that is now not so disruptive, right?

ABDEL SGHIOUAR: Yeah. So it's supporting this thing called in-place upgrade, or updates, where basically it modifies the resources without restarting your pod-- because you do not want to turn VPA on a pod which runs the database, obviously, right? Because then if it's auto mode, it will just restart your database.

KASLIN FIELDS: Yes. So if you have the opportunity to use the vertical pod auto-scaler, I like it because it puts some of the challenge of setting those resource requests and limits on something else. [LAUGHTER] On tooling that's going to monitor your workloads, and tell you what you need to do.

It can do that in manual mode, as has been the recommendation in the past. Or now with this in-place pod resource request and limit setting, it should be less disruptive. It won't be turning your pods off all the time to change their resource requests and limits, and restarting them. So it might be something to look into if you haven't in the past, or if you have in the past, and you didn't want to use it because of that disruption. The new changes in 1.27 might make the vertical pod auto-scaler more interesting to you.

ABDEL SGHIOUAR: Yeah. And if you are on GKE, we have actually used the vertical pod auto-scaler. We turn them on automatically now, and we collect the recommendations-- we turn them on in recommendation mode, and we collect the recommendation, and we make it available in the interface.

So if you go to the GKE interface, you can go to like-- I don't remember what's the name of the tab. It gives you recommendations.

KASLIN FIELDS: It's like the Insights tab, or something like--

ABDEL SGHIOUAR: Yes. Yes, exactly. So if you're on GKE, you don't have to do anything special. It already works, right?

KASLIN FIELDS: Yeah.

ABDEL SGHIOUAR: Yeah. Well, that's pretty cool. Thank you very much for the interview, Kaslin.

KASLIN FIELDS: Yeah. Excited to get to talk about 1.27. And I hope folks will check out all the new changes.

ABDEL SGHIOUAR: Yeah.

[UPBEAT JINGLE]

KASLIN FIELDS: That brings us to the end of another episode. If you enjoyed this show, please help us spread the word, and tell a friend. If you have any feedback for us, you can find us on Twitter @kubernetespod, or reach out to us by email <kubernetespodcast@google.com>. You can also check out the website at kubernetespodcast.com, where you'll find transcripts, show notes, and links to subscribe. Please consider rating us in your podcast player so we can help more people find and enjoy the show. Thanks for listening, and we'll see you next time.

[ELECTRONIC MUSIC]