KASLIN FIELDS: Hello, and welcome to the "Kubernetes Podcast" from Google I'm your host, Kaslin Fields.

ABDEL SGHIOUAR: And I'm Abdel Sghiouar.

[MUSIC PLAYING]

In this episode, we speak to the Kubernetes 1.34 release lead, Vyom Yadav. Vyom is a software engineer in the security team at Canonical and a member of the Kubernetes Security Response Committee. We talked about the new release team and the major updates, deprecations, and removals to expect in this version.

KASLIN FIELDS: But first, let's get to the news. GKE is celebrating 10 years of its existence with a hackathon. You can join online and help build the future of agentic AI orchestrated by GKE for a chance to win $50,000 in prizes and Google Cloud credits. The submissions are already open and due on September 22. Check the link in the description for details on how to sign up.

ABDEL SGHIOUAR: The Golden Kubestronauts program celebrated 100 members. Golden Kubestronauts are certified professionals who manage to obtain all CNCF certifications, as well as the Linux Foundation Certified Sysadmin. The program was launched in 2024 following the success of the Kubestronauts program, which today accounts for over 2,000 members.

KASLIN FIELDS: The creators of the Open Policy Agent, or OPA, are joining Apple. The news was published in a blog post by the maintainers. The project will remain under the CNCF, and there are no plans to change the governance or licensing model.

DocumentDB joined the Linux Foundation. Announced at the Open Source Summit Europe 2025, the open source project is joining under the MIT license. DocumentDB is a NoSQL document store. The project started in 2024 as a pair of Postgres extensions built by Microsoft, and rapidly evolved into a popular document store solution with 2,000 stars on GitHub and hundreds of contributions.

KASLIN FIELDS: Solo.io announced that they are donating the agent gateway to the Linux Foundation. The agent gateway is a cloud-native proxy designed to secure, observe, and enforce policies for agents communication. It supports the A2A, or agent-to-agent, framework, and MCP, or the Model Context Protocol.

ABDEL SGHIOUAR: KubeCrash is a community led virtual event happening on September 23, 2025. Whether you want to tackle your organization's hurdle or stay updated on the latest platform, engineering trends, KubeCrash provides the knowledge you need to level up your team's platform engineering capabilities. Check out the schedule and register for the event on kubecrash.io or check the show notes for more information.

KASLIN FIELDS: And that's the news.

ABDEL SGHIOUAR: Today, we're talking to Vyom Yadav. Vyom is a software engineer in the security team at Canonical. He is also the release lead for Kubernetes 1.34 and have recently joined the Kubernetes Security Response Committee. Vyom has been contributing to open source since his freshman year in college. Welcome to the show, Vyom.

VYOM YADAV: Hey, everyone. I'm glad to be here. Thank you for having me, first of all, and yeah, I'm really excited to be on the Kubernetes Google podcast and discuss about what's happening in v1.34.

ABDEL SGHIOUAR: Thank you. This is tradition we do every time there is a new release. So we open the interview or we open the discussion with the question that we usually ask all the time, knowing that we usually know what the theme of the release ahead of time, but as of the recording, I don't know. So this is going to be a scoop for me at least. I don't know what's going on. So what is the theme this year?

VYOM YADAV: Yeah, so before I tell you the name of the release, I would set a narrative of why the theme is what it is. Every release cycle, we inherit a lot of winds from previous release cycles, which can be a tooling, the documentation we have, the historical quirks of the project. So all of these winds that we inherent, we don't really have much control over it.

But as the release cycle progresses, sometimes, speaking in nautical terms, sometimes this wind fills our sails, sometimes it destabilizes the ship, or sometimes this wind just dies down. So it's not really the wind that matters here because the wind is cold. The wind actually moves the ship. But the ship wouldn't be able to steer properly or in the proper direction without the sailors on board that get on board the helm, control the sails, and stabilize the ship, and so on.

Getting into that narrative and that nautical theme-- because it's Kubernetes, and Kubernetes' logo is a ship wheel helm-- so getting into that, I thought that this release should be we should honor the sailors or the contributors that we have that actually make this ship go through the ocean and actually ship a release. So with that theme and with that nautical theme in mind, I have called this release Of Wind and Will. So it's short form as O' WaW so O-W-A-W kind of thing.

And this release, actually, I think the line that I have for this release is that this is a release that is powered by the wind around us and the will within us. So this is the release theme and the release title that I have decided to go with.

And why this is very special to me is that Kubernetes as a project is not just what in github.com/kubernetes/kubernetes. There is a lot more that happens behind the scenes just to ship the binary that you download of the release, cuts that happen during a release cycle, and there is a lot of cat herding, and there is a lot of work that goes on in project management around enhancements, documentation, communications.

So all this included, it's a very big and tremendous effort that takes place in order to sail this ship across the ocean. So keeping that thing in mind, I think this is why I decided it's Of Wind and Will because it's sailing a ship across the ocean and a ship cannot sail without the wind and the will of the sailors.

ABDEL SGHIOUAR: Awesome. Awesome. That's actually a great theme. I really like the narrative around how you got inspired from Kubernetes and the nautical theme. So I do have a question, actually. I think we never probably asked in these kind of interviews. Between the two cycles, so the two releases-- sorry, so there's a release in April, and this one has come in late-ish, in August-- how early does the work of the release team starts? Is it, like, four months? Is it two months? How early does that go?

VYOM YADAV: So it depends, actually, on which subtheme you are on. And as a release lead, it actually starts, I think, just a few days after the previous release. So it depends on in which role you are in the next release cycle. So I was the release lead, and it's the duty of the release lead to help in assembling the release team as well.

So as soon as the shadow application closes, we filter all the applications we receive to form teams, and its priority based. So because enhancements work is more front-focused, we focus on getting the enhancements shadows sorted first, and then we move towards getting the other shadows onboarded. So it depends on the role you are in. But for the release lead, it starts pretty much after the previous cycle ends.

ABDEL SGHIOUAR: OK, that's interesting. So it's basically a four months-ish amount of work to get the release out?

VYOM YADAV: Yeah, it's similar to that.

ABDEL SGHIOUAR: Awesome. Before we get into the details of the release, if you would, to describe it-- and I know we are a few weeks ahead of the release itself, but if you would describe it in few words, what would that be?

VYOM YADAV: If I really think about it, I think I have described it in a few words, and that's the title of my release. So Of Wind and Will, I don't think I can shorten or elaborate this further to underline that this is a release powered by the wind around us and the will within us. When I was thinking about the release theme, these were the quotes that inspired me the most. And I was thinking about how would I name a release? Because everyone has their personal preferences. And Octarine was the previous release. Before that, it was Penelope. So when I was thinking, I wanted to honor the contributors. So I think my few words over here would be the release name only, which is Of Wind and Will.

ABDEL SGHIOUAR: Awesome, great, I like that. All right, so let's get a little bit into details. There was a sneak peak article blog, if you want, published, which talks a little bit about some, as the title says, sneak peak. So that, I think, features that a lot of people will be interested in. But you shared with me a preview of the actual release blog, if you want, that has all the details. And there is a lot of things going on. There's a lot of new things. There's a lot of deprecations. Can you give us a little bit of a recap? What are we looking at, what to expect in this release?

VYOM YADAV: So this release, we have 58 enhancements, which is a little lower than the last release cycle, but I think it's a very close number. Last time, it was 63 or 64.

ABDEL SGHIOUAR: 64, I think, yeah.

VYOM YADAV: So it's pretty close to what we had the last cycle. And breaking this down into the stages that those gaps are in, we have 23 gaps that are going stable, 22 that are going beta, and 13 gaps in alpha. So in the sneak peek blog, we did not mention that we have deprecations, but we actually have a few deprecations. So now this is the wind that I was referring to. So this is a change that we require in our tooling, in project management, in how we categorize gaps. So this is something that we're going to discuss in the retrospective to fix this, but we have deprecations.

Getting the cgroup driver correct has been a pain point for many Kubernetes cluster administrators. So this KEP 4033 that solves this problem, which basically has the feature of discovering the secret driver from the container runtime. But to support this feature, container runtimes also have to support this. So not all container runtime versions support this.

And the major thing to note here is that the containerd version v1.x does not really support that, but still in the deprecation phase, containerd version v1.7 and forwards would be supported, and you can specify the cgroup driver using the kubelet flag. But if you are looking on to using this KEP and the auto discovery feature so you don't have that pain point, you should upgrade to containerd version 2.x. So this is one of the deprecation, and this also deprecates the kubelet command line or the configuration parameter. So this is one of the deprecation.

The other one is merely an alias. So whenever your service is routing traffic to pods, you want to make sure that how that traffic is routed. So whether you want to prefer it in the same zone-- so this was previously called PreferLocal. Now PreferLocal is a bit ambiguous, and it does not really convey the meaning properly. So this is more of a variable name change, where we are calling it now PreferSameZone. The meaning remains the same. The logic remains the same. It's just a name change and PreferClose would be supported as a deprecated alias for this. Note that this KEP, I believe the number was--

ABDEL SGHIOUAR: 3015.

VYOM YADAV: Yeah, 3015. And this KEP actually mentions that, in non-Google status, removing PreferClose from the API. So PreferClose is not going to be removed from the API, but it is deprecated. Alongside this, it also adds PreferSameNode if you want to route the traffic in the same node with the appropriate fallbacks to same zone, and then cluster-wide. These are the deprecations in this cycle, but I can jump into the interesting stable and beta.

ABDEL SGHIOUAR: Yeah, we'll talk a little bit about the enhancements. I have some few of them that I want to cover, but I do actually-- I was just looking at the blog for this PreferClose. It's true that PreferClose is quite ambiguous. It doesn't mean anything and PreferSameZone, PreferSameNode makes a lot of sense, especially if you're in a cloud provider because in cloud, we talk about the concept of zone and availability zone or whatever. So it's quite interesting.

All right, so we'll go into the new stuff. And I think, in my opinion, the big one is KYAML Yeah. Are we getting rid of YAML? What's going on there?

VYOM YADAV: No, we are not getting rid of YAML. KYAML is actually YAML. It is YAML and all YAML parsers can work with KYAML. So YAML is not going anywhere. We are not getting rid of YAML. You can still get whatever minus or YAML. It just has now a new option called KYAML, which is an alpha feature that you can try. KYAML is supposed to-- so YAML as a language has a really wide grammar, and not a whole of that grammar is useful or is the one that we really want when we want to see the output of a long manifest when we are scrolling down the terminal. So that's not really ideal.

So KYAML aims to bridge a few of that gaps. For example, whitespaces in YAML can be very painful, and especially when you are deep into that YAML and you don't know how deep you have to nest, and then you have to do space, and then if you are on just the terminal, it can be very painful. For example, IDEs have that feature of showing how deep you are inside a YAML and the parent of this node and then parent of that node, and so on.

ABDEL SGHIOUAR: Yeah, and autoindentation sometimes, that would just solve this for you.

VYOM YADAV: How this solves this, spacing does not make any change because it's similar to how JSON looks. And we have used a subset of grammar in the YAML definition itself to create KYAML. This also solves the famous [? NOVI ?] bug problem, which is like, some string literals in YAML, like no, n, capital No, vi, yes, they actually mean Boolean values, true or false. But if you have them as string literals, then your YAML can get-- you won't understand what's happening behind the scenes when you apply that manifest. So it also codes keys that are ambiguous so that there is no ambiguity left in the YAML.

Alongside that, another big problem with YAML is that when you are templating YAML, for example Helm, when you are creating the value of something specific, and then you have to exactly have the same spaces in your particular low value file that you are going to apply so that the final YAML that is created is in the proper format.

Now getting those spaces right in templating can be very painful. So KYAML also solves that problem because spacing is not an issue anymore because spacing is not an issue. You can specify it however you want. And it would be rendered correctly. So that is another thing KYAML solves.

ABDEL SGHIOUAR: So is this new alpha feature KYAML only as an output, or is it also as an input? Can you write KYAML to the Kube API?

VYOM YADAV: Yeah, KYAML is YAML. KYAML is YAML. You can use KYAML anywhere that where YAML is accepted. So it's just simple YAML. It's just a particular subset in the YAML grammar that is being used to produce output. It's still pure YAML.

ABDEL SGHIOUAR: Got it. So YAML is not going away. We're just introducing a new better thing called KYAML. I looked at it. I looked at the blog. I looked also at the KEP that Tim Hockin have opened a while ago to introduce KYAML, and I do have to admit, it's slightly more clean than just like simply YAML. Now I have to get my hands on very large, as you said, very large manifests because that's where it will start becoming more interesting or useful. Cool.

And the next one, I think, this is quite an interesting one. So typically, when you have-- and just to explain to the audience-- I'm going to take two seconds to explain it, and then you can add more stuff if you want. Typically, in a cluster where you need to pull images, there are two ways that your node can authenticate to the registry. It's either the service account of the node or the identity of the node itself. Or sometimes you can add a secret that contains the password of the registry or whatever.

And so now they're adding service account tokens that can be set at the pod level, which means you can set a pod with a token, an OIDC token, to say, pull the image from this particular registry. And this is how you authenticate. And this, for me, is solving a very particular problem of multitenancy among other things. Did I describe this correctly? Do you have anything to add?

VYOM YADAV: Yes, pretty much correctly. Projected service account tokens, what this is being called as, I think this is a really great feature, and especially for multitenant environments. You specify the audience you want this to be scoped to for a particular service account token. And you can have that in the service account definition. The audience you want this to scope to-- you can have the annotations, and then in the-- what's that? The credential provider config, you can actually specify the audience you want to bind to.

And it's not just that. You can actually have authentication over this. So you can actually control, based on roles, what kind of audience is permitted for kubelet to fetch tokens. So kubelet cannot fetch tokens for any audience-scoped token. It has to follow a particular set. So you define that in the configuration. And then the kubelet would fetch a token which would be ephemeral, and then it would fetch the image using that token. So this kind of isolates multitenant environments more than what they were used to be.

And this, actually, is a very good step. Of course, you have the root, which is the credential provider. That would still have to be controlled using traditional ways that you have been setting up a credential provider. But it's still you have pod-level isolation of the tokens that you share. And a really good aspect of this is that you don't have any secrets that are bound to the node, which means there is no overpermissive fetching, and a particular pod does not have access to the images of some other pod that it shouldn't have access of.

So I think this is a really great feature, and this is what I like about this release cycle. This release cycle are a lot of KEPs that I have been reading, they have been covering a lot of gaps that there were in implementations that we had. So filling those gaps to follow best practices-- so I have noticed a handful of gaps in this particular release cycle. These gaps have been-- the work in on these gaps have been going on for quite some time. But the stages that they are being promoted to, that's the, really, shift that I'd like to see going forward in Kubernetes release cycles.

ABDEL SGHIOUAR: Yeah, no, that's something that, I think, I noticed over the last few releases, which is introducing things that solve friction that exist already, just like solving problems that people already have, whether it's multitenancy or AI or other-- just very particular gaps, very particular features, very targeted.

Speaking of AI-- DRA, so Dynamic Resource Allocation. We've been hearing about this thing for a very long time. And from what I understood, it's like some core resources are going GA. So basically, my understanding is that they made DRA function the same way as the volume stuff, where you have volume claims and stuff like that, right? So what's going on there?

VYOM YADAV: So core of DRA is going stable. That's the big thing. So to give a bit of context here, there was another implementation of DRA before this, which was the classic implementation. And I've been reading more and more about the classic implementation versus the current implementation. So to knit the whole story, I'll cover from where DRA started.

DRA actually started from the ability to have pods consume resources like GPU, FPGAs, like NICs, and they are getting assigned to a-- getting bound to a particular node. Now all devices wouldn't be bound to a particular node, but the devices that are bound to a particular node, consuming that.

So the original implementation started in a way in which there was the device driver, and opaque parameters were being sent to the device drivers. For example, I want a GPU which has these following properties, and that resource request was sent to the device driver. And then the device driver would give you the nodes that support that, and then the scheduler would go back and forth with the device driver to finally figure out which nodes support this kind of device. And the pod would be scheduled onto that node.

Now this approach, the drawback that it had was with cluster scaling because these were opaque parameters. For example, I specified the key as foo with value 1,000. Now Kubernetes has no context what foo 1,000 means here. The device driver could be processing that foo 1,000 as maybe 1,000 divided by 1,000 equal to 1. And whatever node has that value as 1, using that, or maybe by 10, 20, or whatever.

So there was no context that cluster autoscalers had to perform a simulation of, hey, would this resource claim request be satisfied if I add a pod or a node of this kind? So this was the drawback with the classic DRA implementation, and this is why this was dropped in favor of the approach that we have right now, which is structured parameters for DRA.

So this kind of gap has been divided in a-- has been created in a better way in which the scheduler can actually predict whether adding a new node would satisfy this resource claim. So how this works is that there are resource slices that the device driver publishes on a per-node-level basis. So resource slices basically tells you, hey, this node has this kind of device. This node has this kind of device. And then there are your resource claims.

Now the thing about resource claims here is that resource claims are now based on CEL, the common expression language. So it's a condition that can be evaluated, that can be simulated. So that condition is evaluated, and the existing resource claims are checked, and you can actually have things like priority on which resource claim you want to match first. That's another gap separate from this one. But you can actually then, from that condition, match on which node supports that kind of device. And then that pod can be scheduled on that node.

So this is going stable now, which is the core of DRA. I think Patrick and Kevin actually have a talk at KubeCon about DRA is going stable.

ABDEL SGHIOUAR: We'll try to find it and add it to the show notes.

VYOM YADAV: Yeah, what's next and what's up for v1.35 as well. But this going stable actually means that a lot of real-world use cases were considered because this gap has already gone through getting reverted I think it was 1.32 when this was reverted because this did not suit the autoscaler constraints. And then, going through this phase, so I think this is a major milestone for DRA because it has gone through its ups and downs and now it is finally going stable.

ABDEL SGHIOUAR: Yeah, I do like the approach, the whole structured approach with proper resources that allows you to match pods to nodes and also what you said about autoscaler. It's pretty interesting. And then one more thing that actually attracted my attention. I never knew how useful this could be, but then when I thought about it, tracing for the kubelet and API server.

VYOM YADAV: Yeah, I think tracing for the API server, that's a pretty old KEP. I think the number is 647. So we are right now in 5,000-ish range for KEPs, and this is a really old KEP And the beautiful thing here is that both of these KEPs got into beta into the same release, and now are going GA in the same release as well. So this is something I found very amusing.

So this is tracing for the kube API server, which is basically you would have-- you would be exposing the spans, traces, and the trace context, and you can have a autocollector, which would be a sidecar, or you can have a central thing that kind of collects all these metrics, and it would be wrapped around the HTTP server in the API server for both client and server. So this is the first step I think, which is 647, which is tracing in API server grace.

Tracing in kubelet I think is 2831, which basically instruments the gRPC calls that you would make to different kind of interfaces, like CRI, CSI, so instruments that collects all the spans and then exposes them, and they can be-- so both of these, I think, can be configured in the same way in which you can define the amount of traces you want to basically input or basically read sampling. Yeah, that's the word I'm looking for. So it defines sampling rate for the traces you want. And it also defines the endpoint of the autocollector that you want these traces to be collected at. Both of these KEPs, I think, going GA at the same time improve the observability aspect of Kubernetes by quite a lot.

ABDEL SGHIOUAR: Yeah, just to be very clear, when we say tracing here, we mean specifically the control plane tracing. So essentially, allowing people to know exactly what's happened from the moment you issued an API call to the API server through all the components. So it's not the data path in a way. It's the control path, essentially, the control calls.

VYOM YADAV: Yeah, this KEP actually lists nongoals as replacing the existing logging and metrics and all that portion of tracing. So when you are sending a request, it's going through an admission controller. So having the traces over there, having the how long did it take to go through that process, so when the request finally reaches the HTTP server of the API server, so that complete trace would be there in the KEP that is 647.

The other is the tracing for kubelet, which is different from the former KEP, which is for the gRPC requests that you are sending to various endpoints-- also, the reconciliation mechanism. So, for example, creating a reconciliation mechanism and fetching the image, creating the pod sandbox, and then creating the container, so having traces over all of that as well.

ABDEL SGHIOUAR: Yeah, so potentially adding a volume, adding anything that needs to be added. So having that end-to-end trace so you know if there is any bottlenecks or anything that slows down the control plane of the cluster, essentially. Cool, cool. And there is one more thing, actually, I was reading just this morning. I don't know if you have any idea about it. I've seen it when sidecars were introduced. I've seen some KEPs floating around, which is like pod-level resource requests and limits. So this is going to beta.

VYOM YADAV: Yeah, that's a pretty popular KEP, pod-level resource requests and limit. I think this was 2837, if I'm not wrong.

ABDEL SGHIOUAR: Yeah, 2837. It was opened in 2021, so it's quite old as a KEP.

VYOM YADAV: Yeah, I think this was basically to stop overprovisioning resources. So when you are basically overprovisioning resources for each container and you have to specify, for example, you specify requests and limits at a container level, you also want to specify them at a pod level. So this cap goes towards that. I have not checked out what the code was merged for this KEP for this particular release cycle. That is something I will do because there are 58 KEPs, and tracking all these is a bit too much. So yeah, I haven't checked the actual code changes that were merged for this KEP, but yeah, that's what I know about pod-level resource requests.

ABDEL SGHIOUAR: Yeah, it's interesting because I think everybody is used to the idea of setting up the resource requests at the container level, and at the pod, but being able to have a pod-level resource limits is quite interesting because, then, you can potentially-- it opens up futures where you can potentially have quotas at the pod level, and then the containers can share whatever resources the pod has and other use cases. It's quite interesting.

VYOM YADAV: Yeah, so having it at the top level of pod and not caring about how the containers split it.

ABDEL SGHIOUAR: Exactly. Exactly. Cool. Awesome. This was pretty cool, actually. I think that, first of all, I like the theme, Wind and Will. I like the narrative behind it. So kudos on that.

VYOM YADAV: Thank you.

ABDEL SGHIOUAR: And yeah, I think this release is also going to be exciting because there's a lot of new interesting things. I like, also, the fact that not everything is focused on AI because it feels like there was like a little bit of an AI overload in the last couple of releases.

VYOM YADAV: Yeah, I feel this is a very well-rounded release on the KEPs that I've been going through, and it covers a lot of different aspects of the project. So I really like this, the things that are coming out in this release cycle. And yeah, I look forward to finally getting the 1.34 out on time. And that's the goal that I have in my head right now.

ABDEL SGHIOUAR: And getting some sleep.

VYOM YADAV: Yeah, I'm actually going on a vacation after the release cycle.

ABDEL SGHIOUAR: Awesome.

VYOM YADAV: It has been a very long grind, and I want to take a break.

ABDEL SGHIOUAR: Awesome. Thank you so much, Vyom. Thanks for coming on the show.

VYOM YADAV: No worries. Thank you. Thank you for having me.

ABDEL SGHIOUAR: That brings us to the end of another episode. If you enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on social media @kubernetespod or reach us by email at <kubernetes@google.com>. You can also check out the website at kubernetespodcast.com, where you will find transcripts and show notes and links to subscribe. Please consider rating us in your podcast player so we can help more people find and enjoy the show. Thanks for listening, and we'll see you next time.

[MUSIC PLAYING]