#21 September 19, 2018

Kubernetes SIG-PM, with Ihor Dvoretskyi

Hosts: Craig Box, Adam Glick

This week, your hosts talk to Ihor Dvoretskyi, Developer Advocate at the Cloud Native Computing Foundation, about SIG-PM, the Special Interest Group for Kubernetes Program, Product and Project Management.

Do you have something cool to share? Some questions? Let us know:

Chatter

News of the week

ADAM GLICK: Hi, and welcome to the "Kubernetes Podcast from Google." I'm Adam Glick.

CRAIG BOX: And I'm Craig Box.

[THEME MUSIC]

ADAM GLICK: How's it going, Craig?

CRAIG BOX: You and I are finally in the same city. Although, it turns out, we're not actually in the same hotel.

[LAUGHS]

ADAM GLICK: Yeah, so we're still in recording digitally over the net, as it turns out.

CRAIG BOX: We'll just have to pretend that we're enjoying each other's company. But we will be seeing each other later this week when, by the time you hear this, Google Cloud Next in Tokyo will be live.

ADAM GLICK: Indeed, and hopefully, if you're hearing this in the first few days that this is up, please stop by and say hi to us. As an added bonus, we got some stickers printed. We finally have stickers--

CRAIG BOX: Hooray!

ADAM GLICK: --for the podcast. So stop by and pick one up. We have plenty of them.

CRAIG BOX: Did you have them printed in time for the Seattle Summit last week?

ADAM GLICK: I did not. I just got them before I got on the plane. So I literally threw the box into my gear here. So, actually, folks here in Japan will be the first ones to be able to get their hands on the stickers.

CRAIG BOX: Brilliant. Why don't you tell us how the Seattle Summit was?

ADAM GLICK: I see the lineup you did there. Nice pitch. The Seattle Summit was great. It was about 1,400 people or so came through.

CRAIG BOX: Really?

ADAM GLICK: And it was a great chance to meet and talk to a lot of people doing some really interesting stuff, especially around the serverless space. There were a bunch of folks that were building really interesting applications, and looking for how they're doing it, and really seeing a growth in the startup community of a lot of really interesting startups in the area interested in jumping in and building, in both serverless stuff and the Kubernetes stuff.

CRAIG BOX: Brilliant. Well, it's not a competition, of course, but we had over 3,000 people at the Singapore Summit last week. As a financial capital of the world, there are a lot of people running on premises Kubernetes installation, so a lot of people coming and asking us about the upcoming GKE On-Prem.

A lot of people with very interesting workloads running on Kubernetes, and a full house for my talk, which was fantastic. Hopefully, there'll be a video posted later on. But if you didn't have a chance to see it there, I will be in Sydney with a very similar talk, and then two weeks after that in Hong Kong with my colleague Mete, where you can come along, and learn about Kubernetes, GKE and Istio.

ADAM GLICK: Cool. Speaking of news and things showing up, did you see that we broke the top 100 tech podcasts on iTunes?

CRAIG BOX: Yes, so if you slice and dice it the right way, if you turn your head the right direction, and limit things to the right categories, you can actually call us top 63 as of last week, so--

ADAM GLICK: Nice, so thank you to all of our listeners. You folks all made that possible, so thank you.

CRAIG BOX: Absolutely. How are you enjoying Japan so far?

ADAM GLICK: Japan has been great. It is an incredible place. There is really nice people. It is amazing, the technology that is everywhere. Have you had a chance to try the loo, my friend?

CRAIG BOX: Yes, yes, that was the first topic of discussion. Being that we're at different hotels, we had to trade discussion points. It's like, well, what's it like when you open the door? I have a little panel on the wall, which gives me several different flushing options. And I think that even just the fact that when I walk in the door, the lid opens automatically, that warms my heart.

ADAM GLICK: The lid opens automatically. There's a light that comes on. There is an entire control panel. It blew my mind. I'm like-- well, I feel like, in the states, I am in a water closet dark age.

CRAIG BOX: Yes, and, of course, they're made by a company named TOTO, so the song "Africa" just plays in my head, every time I go to the washroom.

[LAUGHTER]

ADAM GLICK: Shall we get to the news?

CRAIG BOX: Let's get to the news.

[MUSIC PLAYING]

ADAM GLICK: For those running machine learning in the cloud, Google has announced that their Tensor Processing Units, or TPUs for short, are now available in beta from Google Kubernetes engine. Google posted about TPU availability and GKE in a post walking through their open source Minigo project, which is an open source implementation of the ideas in the human-beating AlphaGo paper. Using 640 TPUs, the Minigo team were able to use 115 petaflops of TPU power playing 20 to 30 games per second. Go, go Minigo.

CRAIG BOX: Tom Gallacher from development company YLD came up with a great way to demonstrate the concept of dynamic admission control in Kubernetes by connecting it to a heart rate monitor. If Tom's heart rate isn't over 130 beats per minute, his pods won't be admitted. So he can't use Minikube unless he's on his treadmill. Tom demonstrated this at a recent London meetup but no word on if the presentation itself was stressful enough to allow his pods to run.

ADAM GLICK: Did he say if that was a no sweat exercise?

[CRAIG SIGHS]

[RIMSHOT] [LAUGHS]

The CNCF posted a new case study last week, showing how Northwest Mutual, which is a bank located in the Midwest United States, switched to cloud native infrastructure to help them increase their number of deployments from 24 a year to over 500 while lowering their downtime-- another great example of Kubernetes in action in the financial services

CRAIG BOX: How is the Northwest different to the Midwest?

ADAM GLICK: It's a little further north.

CRAIG BOX: Pulumi, a lesser known cheese and also a Seattle startup with $5 million in funding and a dozen ex-Microsofties, this week released their cloud native SDK for configuring Kubernetes. Pulumi is an SDK for actually using code to describe infrastructure, bringing code back to infrastructure as code. Confused? Their screencasts and videos explain it all very well, and you can find them linked in the show notes. Pulumi also announced that they've joined the Cloud Native Computing Foundation.

ADAM GLICK: If you're interested in joining the Kubernetes 1.13 release team, a call went out last week looking for people interested in being a part of the process. If you want to know more about the process, please check out show 11. Speaking of Kubernetes releases, beta 2 for 1.12 is now out. And by the time you hear this, a release candidate for 1.12 should also be out. Barring anything unforeseen, we expect the release of 1.12 to launch on September 25.

CRAIG BOX: Does the thought of running your own Akka cluster give you a heart attack-ack-ack-ack-ack-ack? If so you ought to know by now, that the Akka framework, with two k's, now supports Kubernetes clustering. Akka is a popular framework for Java and Scala, which underpins Apache Spark. We're told one of the k's now stands for Kubernetes. But at the time of recording, Lightbend, the vendor, have not confirmed which one.

ADAM GLICK: If you're interested in taking the acronym challenged Certified Kubernetes Application Developer exam, that's CKAD with a K and a C, you may be interested in a new blog series from Elliot Forbes. He started a three part series on his studies for the exam. In his first posting, he covers about 40% of the content that's identified for the exam. So if you're studying for the CKAD test, or thinking about it, it might be a good place to read and compare notes.

CRAIG BOX: Are you ready for this? My colleague Ahmet Alp Balkan has written a blog post on advanced Kubernetes health checking. Above regular liveness and readiness, Ahmet's blog post will inform you, with trademark hand-drawn diagrams, how to do health checks with a sidecar server, headless clients, or secondary health ports. For example, did you know that if a machine learning job heats up a GPU, after a period, your computation can slow down? You can detect this and move the job to another node.

ADAM GLICK: Finally, Sysdig raised an additional $68.5 million last week to expand their work on security and performance of containers and cloud native architecture. Just another example of how the growth of Kubernetes-based infrastructure is catching the eye of venture capitalists as cloud native infrastructure grows in the enterprise.

CRAIG BOX: And that's the news.

[MUSIC PLAYING]

Our guest today is Ihor Dvoretskyi, a developer advocate with the CNCF and a co-chair of the Kubernetes SIG-PM. Welcome to the show, Ihor.

IHOR DVORETSKYI: Thank you, Craig. Thank you, Adam.

ADAM GLICK: Hey, Ihor. Recently, we talked to the release leads for the 1.11 release. You were the feature lead for 1.11. What does a future lead do?

IHOR DVORETSKYI: The feature lead role is actually acting as a product manager for the upcoming release. Feature lead person collects features from the community, from the end users, from companies that are contributing to Kubernetes.

And we are collecting them in a specific place. It's called features repo under Kubernetes GitHub organization, where we're tracking the progress of them. As a feature lead, I'm responsible for interacting with feature owners, people, and teams, who are developing these features and should deliver them or not deliver them.

CRAIG BOX: As the case may be.

IHOR DVORETSKYI: Yeah, based on multiple criteria to the upcoming release. So just a brief insight about release process of Kubernetes. The typical Kubernetes release is around three months. We have four releases per year. And it consists of three main stages.

So the first month is the features discussion, negotiation, and so on. So we have a formal deadline. It is called feature freeze until the end of the first month of the release cycle, when the special interest groups that are developing some components, they have to provide their ideas about what is the specific feature they are going to work on in this release.

It can be a brand new feature. It can be an existing feature but with enhanced functionality, like when some feature was existent in a previous release as an alpha feature or as a beta feature, and in the next release it will be beta or stable respectively. So before the feature freeze, the development teams have time to negotiate the discussions about what we'd like to see in the upcoming release.

They are working closely, also, with us, with SIG-PM, on the scope of the features for this release. At the same time at feature freeze, we have to have the clear list of the features that will be available in the upcoming release. This list can be changed because some development teams would find that they're not able to deliver this feature in time.

And we really appreciate it because we don't want to have any low quality features in the releases. So it's better to bump this feature to the next release and have it ready in the next one, than deliver a low quality code for this, just to see this formal feature in a change log.

So after the feature freeze happens, development teams have time to start working on the actual coding. And feature lead starts interacting with the documentation team, with marketing team, with release notes team on other items that are related to the features to the release process.

Documentation team is responsible for having all these features documented in official Kubernetes documentation on Kubernetes.io. Marketing team started working on preparing the announcements, blog posts, and so on, and any other marketing activities that are related to the upcoming release.

The same with the release notes process, where the release notes lead have to work with the features lead on defining that features that have to be specifically highlighted, for example, in the change log. That features that have to have some specific mention there, and so on.

And the last part of the release cycle is the code freeze. It's before the beginning of the final months of the release cycle when the active code and the active development of the feature has to be finished and starting from the code freeze until the end of the release cycle when their Kubernetes release has been cut, we have around one month for working on the bug fixes, and so on.

And the feature lead in this period of time is actively working mostly with marketing team on the upcoming marketing promotions on the upcoming blog posts. We're also working with vendor companies on the announcements around Kubernetes, what they can get from Kubernetes upstream for their commercial products, and so on.

CRAIG BOX: What's the bar for something to be considered a feature? If I make a change to something that already exists, that might be a bug fix. It might merit a release note. But when does it become a feature that needs to be tracked by you and your team?

IHOR DVORETSKYI: Initially, we had an informal agreement that a feature is some piece of code that should have some kind of documentation and is, like, the atomic unit of Kubernetes codebase. So we may have some bug fixes, right? And it's fixing a few lines of code. But something significantly big and that brings some new functionality for Kubernetes is exactly a feature.

Speaking about the features, themselves, we have three stages. It's alpha, beta, and stable, or GA. Alpha features are mostly data features that are in the active development. They're features that not stable. They're definitely not recommended for any kind of production usage.

They are mostly like the development snapshot of what is being done by the development teams, and what are our common goals, what we'd like to see in Kubernetes in production-ready Kubernetes in the nearest time.

Beta features, well, they're usually production-ready enough. But they may still have some bugs and issues. But the main difference between a alpha feature and a beta feature is that everything that was bundled into the feature at a beta stage can not be changed except for the bug fixes.

So if you develop some component that is related to some feature in Kubernetes-- for example, that is related to Kubernetes API. On the alpha stage of API, API can be changed. And everything that you developed before can be broken. But when you're developing something for the beta functionality of Kubernetes, the API is stable. Even the name of the feature will not change without any big reason for that.

So yeah, and stable features, GA features, actually the same beta features, but much more polished, including so many bug fixes as possible. And usually Kubernetes vendors, companies that offer some Kubernetes services can provide CLA on that functionality.

CRAIG BOX: There's a lot of process now around releasing features and a special interest group dedicated solely to product management. But you've been working on this since, I believe, Kubernetes 1.3, so before there was a SIG-PM and before features lead was a role. And so I assume you've helped put a lot of those things in place. And tell us about the journey and how we got from the 1.3 stage and your early involvement to the process that we have which turns out reliable releases every three months now.

IHOR DVORETSKYI: Yeah, so it was the big journey for entire community, not only in the features area. When Kubernetes 1.0 was released, it was way, way smaller codebase than today, right? As far as I remember, it was the single repo on GitHub or maybe just a few repos on GitHub including documentation that was a separate repo and so on. For example, today we have more than 60 or 70 reports under multiple organizations on GitHub.

CRAIG BOX: Wow.

IHOR DVORETSKYI: So that period of time, it was pretty easy to track what is the basic functionality of Kubernetes. That period of time, we even haven't had any definition of Kubernetes core. So defacto Kubernetes core was everything that is landed into kubernetes/kubernetes repo on GitHub.

At the same time, Kubernetes became to be much more complex product and much more complex open source project. Multiple companies joined Kubernetes community to contribute some new functionality. Multiple companies joined Kubernetes community to consume that or to build some of their commercial products on top of the vanilla Kubernetes.

So the process of tracking everything, every piece of functionality for Kubernetes, it came to be much more difficult with every release. Approximately that period of time, a group of people was in a Kubernetes community proposed the way how can we track the features was in Kubernetes.

We've started with definition of the feature. So it was, again, it was more informal definition. As I mentioned before, it was mostly about having some, more or less, big piece of code that can deliver some new functionality. So it was, more or less, informal. So we've decided to create a separate process for tracking the features. We created a separate repo on GitHub. It's called features repo.

CRAIG BOX: It's a good name.

IHOR DVORETSKYI: Yeah, pretty clear, yeah? We also started tracking the efforts around the features in the tracking spreadsheets. And so that was the bootstrap of the process. Around the same period of time, there was a proposal within a community to establish the PM group.

It was not a special interest group at that period of time. It was just like the working group, something like that, so just a group of people who can discuss the best practices of what can we do for Kubernetes to make Kubernetes real product. It was in middle 2016, as far as I remember, and at that period of time, it was a period when Kubernetes 1.3, Kubernetes 1.4, Kubernetes 1.5 being released.

Also at that period of time, the release process was a bit different. So today, we have the dedicated release team that consists of multiple individuals, more than 10 individuals, from multiple companies who are contributing to different areas to the release process. At that period of time, it being a single person responsible for release and a group of informal volunteers who can help in that.

So with release 1.3 I've been exactly that volunteer who had been helping the release lead with the features tracking and the features work. So that was the process, so exactly at that period of time bootstrapped to you big pieces was in Kubernetes community around features and products. So we bootstrapped the features process. And we bootstrapped the product management group.

So Kubernetes was much more mature with every release, with every month of development. Kubernetes became to be much more complex. And at the beginning of 2017, again, a group of us has decided that it's a good time to start working on Kubernetes as a product, as a solid market product, despite the fact that it's an open source project, despite the fact it's uncommercial. But it is solid enough to be clarified as a product.

And at the beginning of 2016, we've established the SIG-PM, the Special Interest Group. We had a well-defined group of people who've been responsible for that. We've started working on different procedures and processes. How can we make Kubernetes a real product?

We also had a few people from commercial companies, for example, from Google, Aparna Sinha, who is a group product manager at Google. She was one of the people who started working on SIG-PM and on the process of making Kubernetes a real product. And it was really good because person with an exceptional experience in making commercial products came to Kubernetes community, to the open source community, to make open source project so solid.

And approximately at the same period of time, we had another big change in Kubernetes community world. Starting from release 1.6, it was the first quarter of 2017, and the first release team was formed. And exactly at that period of time, these informal volunteers who'd been helping the release lead with different activities had defined roles, have their names, for example, features lead, or documentation lead, or marketing lead. And it was the solid process of running their Kubernetes release, release process.

ADAM GLICK: You're a co-chair for SIG-PM. What does SIG-PM do besides approve releases?

IHOR DVORETSKYI: We are mostly working within the release process. But except of the release process itself, we're also gathering the market voice, the end user voice. For example, CNCF has an end user community wherein there are multiple companies that are not Kubernetes vendors. They're not offering any services on top of Kubernetes or with Kubernetes. But they are end users of Kubernetes itself, so that at a big financial companies, or entertainment companies, and newspapers, and so on.

So these companies are using Kubernetes in their everyday activities. And they definitely have enough words to say for us about what they would like to see in Kubernetes, what they would like to see in Kubernetes in two years. Because SIG-PM is also working, not only on the current scope of the features, on the technical features for the upcoming one or two releases, we are also working on the roadmap for what do we want to see in Kubernetes in the next few years, for example.

CRAIG BOX: Do you take that to the SIGs, and say, hey, we've got a lot of customers who want this feature, and try and find people to build it?

IHOR DVORETSKYI: Yeah, so it's been made in a bit different way, but yes. So where we have some negotiations with the SIGs exactly at that period of time when SIGs have the planning process.

So we're working with the SIGs because SIG-PM doesn't deliver any technical code. We have development teams within Kubernetes community who are delivering that. And SIG-PM is acting as a bridge between the end user community and the development teams to see in Kubernetes, in an open source Kubernetes, exactly what the end users would like to see there.

CRAIG BOX: What you describe there is, obviously, very similar to what I do at Google as a developer advocate. And you are a developer advocate at the CNCF.

IHOR DVORETSKYI: Right.

CRAIG BOX: How did your contributions lead to you getting that role?

IHOR DVORETSKYI: It was a long history. So I've been a Kubernetes contributor since late 2015. I'm actually, like, to provide a disclaimer, I'm not a developer. I'm not a software coder. My experience before was mostly operational, been working as a DevOps engineer, as an SRE engineer, and so on. So I've been a person who'd been consuming someone's code, not writing it.

But this experience made me to Kubernetes as a project when I'd been looking for some solutions. How can I run my Docker containers on a big scale, on a multiple hosts, and so on. So I found Kubernetes as a project, and I started working, and I started looking inside of it.

But as we remember, late 2015 was in the first days of Kubernetes, right? And for example, documentation was not so solid as today. And my first contribution to Kubernetes was just a few lines of code that I changed in the documentation.

But I excited the process of margin debt, a small contribution, into the Kubernetes codebase so much. So I've started working on looking for some bugs in the documentation because, again, I'm not a real software engineer. So I'm not the best person to find some bugs in a code.

But I know how should good documentation for a good technical project look like because I've been using Kubernetes in my everyday experience in my day to day activities. And I've tried multiple tutorials, multiple articles from Kubernetes.io documentation. And definitely, you can see any issues there. So the process of contributing was pretty easy.

And I have to mention that today Kubernetes is one of the most welcome projects to the new contributors and to existing contributors to make a contribution to the open source world. So if you're looking for the best project in the world to get started with the open source contributions, Kubernetes can be one of them.

Coming back to 2015, I started looking at the Kubernetes codebase, and Kubernetes community, Kubernetes documentation, and I also notice that Kubernetes is not only the codebase, right? It's real community. And I notice that there are multiple SIGs. They have their regular weekly, bi-weekly meetings.

There is the big community meeting that happens every Thursday at 10:00 AM Pacific time. And it still happens every Thursday at 10:00 AM Pacific time where most people from the Kubernetes community are sitting together and discussing some ideas. So it was really amazing to learn. I've been really excited to feel as a part of this big community.

Also, the meetup ecosystem started growing so the local communities who then started working, who'd been discussing Kubernetes, and established them, and so. But my first big step within a Kubernetes community was to start being a co-lead or co-chair of SIG-OpenStack.

So I've been working in Kubernetes at that period of time on my spare time because my company, my previous company, was an OpenStack vendor and in my billable time I'd been working on OpenStack, not on Kubernetes, but as a person who's been working on OpenStack.

And I've been passionate about OpenStack, about movement of OpenStack as a open source platform, but with Kubernetes in mind as a bit different platform, and then a bit different set of technologies that can easily complement each other. I've been looking at the areas where I can contribute my experience of OpenStack and to receive new experience with Kubernetes.

So SIG-OpenStack is a special interest group that is focused on the close collaboration efforts between Kubernetes and OpenStack and being a co-lead of that group was the biggest step in moving from the regular contributor to Kubernetes to some kind of management role. So I've been working for SIG-OpenStack for some period of time.

But at some point, I left my previous company. I've joined CNCF as a developer advocate. But when I've joined CNCF, I've been working on SIG-PM also. And I've been acting as a product manager, being a SIG-PM co-lead for around 1 and 1/2 years.

CRAIG BOX: Is it fair to say that the recognition of that work is what helped you get hired into that role?

IHOR DVORETSKYI: Yeah, probably, yes. So CNCF is the noncommercial foundation. We are mostly helping projects to receive some vision for them. So we offer them marketing services for our projects. We also offer legal services to our projects. But developer experience and developer advocacy is one of the important areas.

You can see how many talks are worldwide and multiple meetups or big conferences. And on conferences of any scale, for example of KubeCon, how many developer advocates are there that are advocating for this technology and to related technologies. But these people are vendor representatives. They are presenting their commercial companies.

So at CNCF, CNCF decided to hire a developer advocate who will work in the total vendor-neutral environment, and will work on specifically the open source project. With my experience, speaking at meetups, speaking at the conferences, with my experience of being a person who'd been involved into the Kubernetes community itself, I found that it's an amazing place for me. And CNCF found it, and probably I'm the best person, at least from those who have applied. [LAUGHS]

ADAM GLICK: Ihor, you've worked on a ton of features that have come out in Kubernetes going back several releases. What was your favorite one that you thought was most impactful?

IHOR DVORETSKYI: The development pace for new features in Kubernetes today is much lower than previously. There are two reasons for that. First of all, because Kubernetes is much more mature product today than one or two years ago, and all the major and important functionalities all implemented there.

The second reason is that Kubernetes, as I mentioned before, it was the solid single repo on GitHub. Today, we are working on splitting Kubernetes to Kubernetes core itself and to some extra functionality that is a part of Kubernetes ecosystem, and part of Kubernetes GitHub organization, for example, but not Kubernetes core itself. So the development base of some new functionality for Kubernetes core is much lower today than previously.

But that was one of the biggest achievements of Kubernetes community. And that was one of the biggest set of umbrella features in Kubernetes world that we've been able to remove all the non-critical components from Kubernetes itself to their own place. They are still extremely important for Kubernetes as a solid product. But they are outside of the main codebase.

CRAIG BOX: Your path to contribution through Kubernetes was a documentation change in the beginning. I think mine was probably as well. What would you recommend for someone who wants to get involved with some of the non-coding processes around product management and documentation? How would you recommend someone join the community?

IHOR DVORETSKYI: That's a great question. And I'm really appreciated that people are asking this question much more often today than two years ago because two years ago we had-- the Kubernetes community two years ago was around 95% people who'd been contributing code, or would like to contribute code, and only around 5% of people who were delivering some non-coding items.

Today, we have much more people with product management experience, program management experience, project management experience, with documentation, with technical writing experiences, with marketing experience. All these people who'd like to contribute to be a part of this big project. At Kubernetes community we have a few special interest groups that are focused specifically on these non-coding errors.

At SIG-PM, we are covering the product management error, project management, and program management. Even officially today-- so initially, it was the SIG product management. Today, we are officially called SIG-PM only. We have the product management subgroup for people who are specifically interested in product management.

We also have to program management subgroup for people who are interested in program management, which are kind of different from each other. Besides SIG-PM, we also have SIG-docs, a group for people who would like to contribute documentation. We have SIG-release, a group of people who are specifically interested in long-time working on anything release related, like the release process, release cycles, best practices of how to release software, and so on.

We have multiple ways how can you be engaged with Kubernetes community. First of all, we have a few mentoring initiatives within the community. And again, most of this mentoring initiatives are targeted for the coding contributions, but we still open for the non-coding contributions. We're still open for people who'd like to bring their best practices in a product management area.

And for example, at KubeCon we had a SIG-PM intro talk together with Aparna Sinha. And we received lots of questions from people from the community who are defining themselves as I'm a product manager at company X. We are interested in Kubernetes. We are going to have, or we already have some Kubernetes solution. But I would like to work as a product manager for vanilla Kubernetes, for open source Kubernetes, so how can we do it?

The same is around documentation. So we have lots of technical writers in this world. And technical documentation is a truly important area because your codebase can be amazing, but if nobody will have a way how to use it, if nobody will understand what should they do with this codebase, you will fail as a project. So technical documentation is a great area where you, as a technical writer or as a person who would like to be involved with the community, is a great place to start.

ADAM GLICK: Ihor, it's been wonderful having you on the show. Thanks for coming on.

IHOR DVORETSKYI: Thank you.

CRAIG BOX: Ihor can be found at a meetup or a conference near you or @idvoretskyi on Twitter or GitHub. You can find links to those pages in the show notes.

[MUSIC PLAYING]

ADAM GLICK: Thank you for listening, sharing us with your friends, and giving us a five star rating on iTunes. We continue to be really thankful for your support, so if there's someone you know who would enjoy the podcast, please share it with them. If you have any feedback for us, you can find us on Twitter @KubernetesPod or reach us by email at kubernetespodcast@google.com.

CRAIG BOX: We're very close to a milestone for Twitter followers. So if you don't follow us, please check us out. And we'll give you the updates for each episode as it's released. You can also check out our website at kubernetespodcast.com, where you'll find all of our show notes. Until next time, take care.

ADAM GLICK: Catch you next week.

[THEME MUSIC]