ADAM GLICK: Hi, and welcome to the Kubernetes Podcast from Google. I'm Adam Glick.

CRAIG BOX: And I'm Craig Box.

[MUSIC PLAYING]

Our apologies to the most ardent fans of our show who were wondering, why is it a day late? Why is it a day late, Adam?

ADAM GLICK: Ah, it's because at least 50% of us is located in the United States. And Monday this week, when we normally record, was Labor Day.

CRAIG BOX: Ah. How was Labor Day?

ADAM GLICK: It was lovely.

CRAIG BOX: Laborious?

ADAM GLICK: Very little labor. It's the magic of Labor Day is the celebration of not doing labor.

CRAIG BOX: Excellent. Put your feet up.

ADAM GLICK: So we went camping. Unfortunately because some of the fires that people may have heard of in the Western United States and Canada, there was a burn ban, so we were camping minus a campfire.

CRAIG BOX: Oh.

ADAM GLICK: Although I have to say the most amazing thing that I discovered is that we went camping at a campground. It was a lovely place. And there were people in the campsite next to us and they had a propane campfire-- that they literally brought, like, a propane fireplace and were just using that. And I was just, I was amazed.

CRAIG BOX: That was allowed?

ADAM GLICK: Yes, propane was allowed. No charcoal, no wood.

CRAIG BOX: Ah, I thought you'd to have to have some crazy USB system with your marshmallows over a tiny Peltier cooler or something like that.

ADAM GLICK: Give it some time, give it some time.

CRAIG BOX: You could rip the cover off your Pentium 2 from the '90s and then see if it puts out enough heat to melt a s'more.

ADAM GLICK: Excellent. How about you, Craig?

CRAIG BOX: I'm gearing up for part two of Craig's world tour. Those of you who were listening earlier in the run of this podcast will remember that I did a few dates from Asia-Pacific. And so I'm going to be at the Google Cloud Summit in Singapore next week and then at Google Cloud Next in Tokyo the week after that. The subsequent week, I'll be at the summit in Sydney. And then two weeks after that, in Hong Kong.

So if you're in any of those locations and you're interested in having me come and talk to you about Kubernetes or are interested in coming to one of those events, please do reach out. Feel free to tweet at us. Tweet at me explicitly, if you want me to come and do anything. But I look forward to seeing some more of the world.

I had an interesting encounter, I should say. I gave a talk at a meetup in New Zealand on my last tour. And I was just having a chat with a few people there beforehand. And there was a guy who was like, oh, yeah, I learned about this meetup because I listen to your podcast. That was like-- we were about three or four weeks in then. I was thoroughly impressed. So Leo, if you're listening.

ADAM GLICK: That person deserves a sticker. We need to make a sticker and send it to that person.

CRAIG BOX: We shall indeed. We need to make stickers and give them to everybody, but we are yet to do so.

ADAM GLICK: Shall we get to the news?

CRAIG BOX: Let's. At the Open Source Summit last week, Google Cloud announced it was moving the testing and hosting infrastructure for Kubernetes to the CNCF, along with a $9 million donation to cover its cost for the next three years. This gives the Kubernetes community full control of the container serving, build pipeline, and CI servers, as well as other infrastructure pieces of the project. These are some of the last parts of the project that were only accessible to Googlers, due to the history of their creation.

Dan Kohn-- with a K-- executive director of the CNCF, said of the donation, "we're thrilled to see Google Cloud transfer management of Kubernetes testing and infrastructure projects into contributor's hands, making the project not just open source, but openly managed by an open community."

ADAM GLICK: I've heard a lot of people asking how exactly does Kubernetes spend $9 million. Do you know?

CRAIG BOX: Yeah, the Kubernetes project runs around 5,000 virtual machines, more or less 100% of the time. If we're talking about just a one-core virtual machine, even with Google Cloud's generous sustained use and committed use discounts, that's $120,000 per month. Why so many? Well, every pull request to the project causes clusters to be spun up and down to run tests, which number in the hundreds per day. Then there's the bandwidth. The container images that constitute the various Kubernetes components are served from Google Cloud Storage. And last month, the project served almost 130 million containers. That's almost four million per day.

ADAM GLICK: That certainly is a lot of infrastructure.

CRAIG BOX: Well, if you're interested in that infrastructure, Aaron Crickenberger and Benjamin Elder from Google wrote an overview to the Kubernetes CI/CD infrastructure, including the nautically named tool Prow, which is an "if this, then that" for GitHub events. The Kubernetes instance of Prow runs over 10,000 CI jobs per day, most of which are standing up an entire Kubernetes cluster and exercising it using real-world scenarios. Each new piece of code has to be checked against all supported releases, cloud providers, container engines, and networking plugins. Prow itself runs on Kubernetes and has been adopted by a number of other open-source projects.

ADAM GLICK: Last week, the CNCF welcomed TiKV-- an open-source distributed transactional key-value database-- into the CNCF sandbox, home to early stage and evolving cloud-native projects. TiKV-- a name that they've probably never tried to say out loud-- offers simplified scheduling and auto balancing without dependency on any distributed file system and is inspired by the Google Spanner database. The project serves as an open-source unifying distributed storage layer that supports strong data consistency, distributed transactions, horizontal scalability, and cloud-native architecture.

CRAIG BOX: The CNCF was founded with a mere 28 members back in 2015. And at the Open Source Summit, it announced it added 38 more, bringing it to a total of 284. New members include large companies, consultancies, startups, organizations using Kubernetes, and even a university.

ADAM GLICK: Rounding out our CNCF news, the results from the sixth CNCF survey were released. This survey takes a pulse on the community to better understand the adoption of cloud native technologies. The key takeaways were that production use of CNCF projects has grown more than 200% on average since December 2017 and evaluation has jumped almost 4x. The use of serverless technologies continues to grow, up 22% since December, with the majority of respondents using hosted platforms.

Survey respondents indicated the top three benefits of cloud-native technologies are faster deployment time, improved scalability, and cloud portability. 40% of respondents from enterprise companies-- those were defined as companies having over 5,000 employees-- claimed to be running Kubernetes in production. You can find a link to the full report in the show notes.

CRAIG BOX: Istio 1.0.1 was released last week. This dot release brings updates and bug fixes, primarily around installation and configuration validation, including the ability to run the pilot control plane stand alone for people who only want to use Istio's traffic management capabilities without its monitoring and metrics.

ADAM GLICK: "Forbes" this week published an article talking about five cloud-native technologies to watch beyond Kubernetes. Most of these won't be a surprise to listeners of the show, but they chose Istio, Prometheus, Helm, Spinnaker, and Kubeless.

CRAIG BOX: A side note-- the founder of Kubeless has recently left Bitnami and is now working on a new startup called Triggermesh, billed as your knative platform in the cloud and on premises.

ADAM GLICK: Amazon has added support for horizontal pod autoscaling to their elastic container service for Kubernetes. Amazon's version of Kubernetes was not previously compatible with API server aggregation, but they've patched this upstream. Other components which use aggregation, such as the Kubernetes service catalog, will also work when Amazon's batch flows through.

CRAIG BOX: Finally, Kontena-- with a K and some questionable vowels-- announced version 1.3.0 of their Pharos-- with a P-H-- Kubernetes distribution. This release updates the base to Kubernetes 1.11 and includes CoreDNS and an updated CRI-O runtime, as well as a new command-line tool for installing and managing clusters.

ADAM GLICK: And that's the news.

[MUSIC PLAYING]

ADAM GLICK: Liz Rice is a technology evangelist with Aqua Security and program co-chair of the KubeCon and CloudNativeCon events in 2018. Welcome to the show, Liz.

LIZ RICE: Thank you very much. Pleasure to be here.

ADAM GLICK: For those listening, perhaps you can give them a little taste of your background and how you've gotten to be co-chair and working at Aqua. Wow, I could take like a 15-minute version of this, but I will try and keep it less than that. So I was a software engineer by trade, originally. Spent a long time working on network protocols, which were-- ultimately, it was a great kind of foundation for what I'm currently doing, but terrible to sort of talk about at parties.

So after a while, I left that and got involved in more consumer-facing things. I worked for Skype for a couple of years. And that was a hugely exciting time. I went from there to a music recommendation company called Last.fm. Then went and did some consultancy for a bit, did some really interesting startup things, spent some time in the US.

Again, there's been a few a startups that have not gone brilliantly. And one of them was just coming to an end. It was a thing in TV recommendations. And a friend of mine from back in the network protocol days was saying she and a friend were looking at founding a startup in the container technology space. And I'd only really at that point heard of containers through her. And this is Anne Currie and Ross Fairbanks, who was our third co-founder.

And so container technology was completely new to me at that point. But Anne had said, you will really like this. This is getting back to the kind of heartland of hard-core technology that you really enjoyed back in the network protocol days. And I thought, yea, I kind of did.

So that startup-- Microscaling Systems-- did basically run to its conclusion. But it got me really hooked on containers. I met the folks at Aqua, really liked what they were doing in terms of a product that not only was really interesting technology, but also was something that-- it's a startup, but they have a product that customers are prepared to pay for because enterprises care about the security of their deployments.

And we really hit it off very well and that's how I got involved in Aqua, and how I'm currently doing this sort of technology evangelism role, where I get to talk about interesting things and work on some open-source projects, which is great.

CRAIG BOX: What sort of projects are Aqua currently involved in?

LIZ RICE: Our main product is not open source. It's the Aqua Container Security Platform. And that-- it's what enterprises can use to secure their container deployments, whether they're running on Kubernetes or other container orchestrators, whether they're on a cloud platform, like GCP or AWS, or whether they're hosting it themselves. So that's kind of the main thing-- covering the lifetime of containers through from image scanning at the build stage right through to runtime protection when you're actually running the containers.

And then kind of to complement that, we started doing some open-source projects. The first of them was kube-bench, which has to do with implementing the Center for Internet Security's guidelines if you like their recommendations. And then the latest one that we just released is kube-hunter, which is about penetration testing for Kubernetes deployments. And it's really fun.

CRAIG BOX: How did the kube-hunter project come about?

LIZ RICE: I was trying to rack my brains on how we came up with the idea. I think it was really the idea that actually a lot of these configuration settings are pretty complicated. And how would you know whether you had actually set things up to be secure or not? And you know, there was that really high-profile situation with Tesla and the open dashboard. And we thought, well, you should just have a tool that you can check. You can run it somewhere outside your deployment and see is my dashboard exposed to the elements or not.

And then from there, we thought, well, we could build a more general-purpose tool that tries all sorts of different, essentially, attacks to see what's open, but make it easy to use for a Kubernetes operator.

ADAM GLICK: So it's kind of a penetration testing tool?

LIZ RICE: It's exactly that, yeah.

ADAM GLICK: For folks that are more familiar with the traditional VMs or a bare-metal infrastructure and are now looking at kind of the containerized Kubernetes world, what's the difference in how they should be thinking about things like penetration testing from the more traditional environments to the cloud native?

LIZ RICE: I think that the principles are all exactly the same, whether you are running Kubernetes or traditional deployments. What we've done with kube-hunter is we're really testing specifically for the ports that Kubernetes will typically be communicating on. And we're using some knowledge of what the APIs would look like to give warnings that explain what might be exposed, given that it's Kubernetes. We can give much more descriptive alerts and messages to describe what might be possible if port X or Y is found to be open.

CRAIG BOX: One of the interesting features of kube-hunter is what you call an active mode, where it can actively look for a problem in the cluster and then exploit that to possibly gain further privileges.

LIZ RICE: Yeah, so that was something we didn't want to put in as the default because by default, we don't want to affect anybody's cluster. But we wanted to have the option to run tests that would prove whether or not escalation is possible in a given environment. So we don't do anything super destructive. We don't try to take any pods down or anything. But we might, for example, try to run an executable to demonstrate whether or not an executable can be run in a particular pod.

It's not supposed to be super destructive. It's not supposed to be actually doing what a hacker would do if they were trying to exploit your cluster, but it is supposed to be illustrating the kind of things that they could do, were they to attempt to attack your cluster.

CRAIG BOX: When you do hear about maybe the nation-state exploits against the traditional infrastructure, quite often there are a lot of different exploits that are chained together. Do you think that there will come a time where people are actually looking for a Kubernetes dashboard exploit, and then connecting that to a local root exploit running in the container, and then maybe getting remote root from-- do you see a world where people are chaining those kind of things together?

LIZ RICE: I'm sure it's happening. And you know, hackers are pretty creative and they will be doing that kind of thing. And actually, one of the interesting things about the way kube-hunter is structured is every time we find something, it publishes an event internally, to which you could subscribe and then run additional tests-- additional attacks, if you like. So as we grow the set of tests in kube-hunter, it will be pretty easy to chain them together.

CRAIG BOX: Are you worried that people might take this list of exploits and build it into something like Metasploit, for example?

LIZ RICE: So even publishing this tool in the first place, we did kind of think long and hard about, is this potentially putting tools into the hands of attackers? But the reality is they already have these tools. They have comprehensive port scanning and exploitation tools available to them. So a determined hacker, a determined expert in that field-- they already know how to do this.

The people we're trying to help are the people who actually maybe don't have a huge amount of security background. And by running this kind of test and by giving them really clear information about what we found, we're hoping that will enable them to improve their security posture.

ADAM GLICK: When you look out on the roadmap, what are you most excited about that's coming in the future as part of this?

LIZ RICE: So we haven't formalized a roadmap of exactly what tests will come next. We have got some ideas internally, but we're also really interested to hear what the community wants to add.

CRAIG BOX: What has the response been from the community since you've released kube-hunter?

LIZ RICE: Yeah, we were really pleased with how quickly people started to come to the project. We actually have two mechanisms for using kube-hunter. You can either go to the GitHub repository, which is at aquasecurity/kube-hunter-- and you can go there and run the tests directly from the code that's there. And we also have a website we've put together to make it really easy for people to run it. They don't even have to run a Git clone.

And both of those paths into using kube-hunter, we've been really excited by. So the GitHub-- I should look and check how many stars it has right now-- was rapidly into the hundreds, which I was really excited about.

CRAIG BOX: Nice.

LIZ RICE: And then also we found we've had a really good response to the site where you can go and get this kind of nicely formatted report. I was actually quite surprised by that personally because I thought everyone will come to the Git repo and run it from there.

CRAIG BOX: Oh, no. Everyone loves a nicely formatted report.

LIZ RICE: That was exactly the thing. Security people that I've spoken to have said, you know, the thing is I can put this pretty report and I can just give that to my CSO. And that gives-- maybe there's something that I need resources to fix and I get the resources. Very happy about.

CRAIG BOX: Or alternatively, all responsibility has been absolved. Let's move on now and talk about your other high-profile role this year, which is the program co-chair the KubeCon and CloudNativeCon. What exactly does that make you responsible for?

LIZ RICE: So myself and my co-chair-- it's Janet Kuo from Google coming up for the next two and it was Kelsey Hightower for the one that happened already in Copenhagen-- and as co-chairs, we are really responsible for the program and as far as we get the kind of final say on which of the CFPs get scheduled. We get to pick the keynotes. We have a huge program review committee to help us with that and to help us with the scoring. But eventually, we choose what gets discussed, which is kind of a huge responsibility.

CRAIG BOX: What's the going rate for a keynote?

[LAUGHTER]

LIZ RICE: Yeah, it's like one really, really great submission-- a really good idea.

ADAM GLICK: Awesome. How did you get involved with KubeCon, becoming a co-chair for the program?

LIZ RICE: So they actually approached me. Dan Kohn from the CNCF called me up one day and asked if I would be interested. I think I do now have an idea who it was who kind of suggested me as a possibility for the role, which I'm actually really grateful for because it's really good fun. It's a lot of work putting the agenda together. But the excitement, actually being in Copenhagen, and sort of getting to be on the keynote stage, and host the event, and get to see what makes people really excited, and get involved a little bit in the sort of backstage of what are people really wanting to talk about in the cloud-native world-- it's a real privilege, so I'm really glad to have been invited.

CRAIG BOX: We've closed submissions now for all of the KubeCons that are happening this year. What are the things that you're looking at in people's submissions and what makes a really good talk?

LIZ RICE: I think the first thing that people spot when they're reviewing is does it have a catchy title and does it immediately make them think, yeah, if I saw this on a program, that's something I would want to attend? We really like to see people laying out, in a bit of detail-- not super verbose, but a bit of detail-- what they're actually planning to cover. Because it's one thing to say, I'm going to talk about a really interesting topic. But to stand out amongst all the submissions on that same topic, if you say, I am going to cover these three points and I'm going to illustrate it through this particular demo or I'm going to reveal what happened in our user experience of-- this three things that went wrong-- that kind of level of detail really helps us establish, yeah, this is going to be a talk with interesting points being made.

ADAM GLICK: Do you know in advance what topics you want to focus on for a particular KubeCon? Or is it something that you review the ideas that come in and kind of let that inspire the direction that you take the conference?

LIZ RICE: For me, I think it's really important to reflect what the community wants to talk about. And the best measure that I have for that is looking at the numbers of submissions that come in on different topics. So for example, at Copenhagen, we had quite a lot of service mesh talks. And that reflected the fact that we have something like 100 submissions on the topic. And that tells you that there's a lot of people with something to say, something to learn, something to share on that subject.

CRAIG BOX: You've mentioned program committees before. I was involved with a couple of the events, helping in various tracks. But if someone's out there and they want to get involved and become part of one of those committees or help put together the program for a KubeCon, what opportunities are there and how would you recommend someone-- what community things can people do to get on the radar?

LIZ RICE: Yeah, I think we try to invite people who we are aware of that have been working in Kubernetes, in the cloud-native projects, or who we know from previous KubeCon events or other cloud-native events-- that they're interested, involved, and that they're not just going to push a vendor. It's very important that we're not allowing vendor agendas to be promoted too strongly. So that's kind of what we're looking for. We welcome people to volunteer if they are interested. They're more than welcome to volunteer. I hope that isn't going to result in like 5,000 emails to the CNCF.

ADAM GLICK: And what's the email address to send those to?

LIZ RICE: Yeah.

CRAIG BOX: It didn't result in anyone baking cookies based on last week's podcast episode, so you might be all right.

LIZ RICE: Yeah, well, in that case, I think its speakers@cncf.io.

CRAIG BOX: When you make your submission to the CFP process, you do have to put in information about your company and various categories you identify with. Have you given consideration to a more blind judging process?

LIZ RICE: Yeah, we have. And this has been raised a few times. It's something we definitely want to pay attention to, ensure that we're giving everyone a fair shot. The reason why we don't judge blind is because you need so much information to choose between similar-looking submissions. And we want to know whether people are a subject matter expert. And a talk from somebody who's been a core contributor to a particular project is probably going to be more interesting than somebody who's come from-- really not had any experience with it at all.

So that's really helpful information for judging the quality of the actual proposal. And you can't do that unless you know who the person is. And as soon as you know who the person is, the reality is we know where they are in terms of if they're a member of a minority group or not. We obviously try very hard to balance diversity in all senses. And I think is one of the responsibilities that we have as co-chairs.

So while we're going to be looking at things like gender diversity, we're also looking at things like company diversity. It's a real challenge actually, that we have a small number of companies who have a lot of contributors to these projects. But we don't want the conversation to be entirely dominated by those companies. So trying to balance, let's call it a diversity of viewpoint, it's a really hard challenge. I can't say that we're always going to make the right call, but we do try really hard to balance things out the best we can.

CRAIG BOX: So what advice would you give to someone who wants to get started at presenting at an event like this, but possibly doesn't have the experience or the brand to get selected off the bat?

LIZ RICE: So it's not just about being that subject matter expert. We do try to look-- and as part of this kind of diversity of viewpoint, we do want to get a mix of experienced speakers, and new speakers, and people who are new to the project. If they have something interesting to say, it's kind of incumbent on them to have a really clear proposal because that's almost the only information we have to go on. But if somebody has exciting things to talk about and they can express that through the proposal, then we're really excited to read those and we want to get those on the agenda.

CRAIG BOX: Moving on, we have the first KubeCon coming up in China in November. It's very exciting.

LIZ RICE: It really is, yeah, yeah. It's actually going to be the first time that I've been to China personally, so that makes it a real adventure for me. And I think it's going to be a real experience for the whole cloud-native community really because there are all these massive companies, these massive cloud providers in China who, I think for a lot of us in the rest of the world, we're not particularly exposed to what they're doing. And I think they're going to have some really interesting stories to share. So yeah, it's going to be really exciting.

CRAIG BOX: What considerations have we made to the fact that this is the first KubeCon presented in two languages simultaneously?

LIZ RICE: Right. So the good thing is the CNCF being part of the Linux Foundation can build on-- they've already run conferences in China, so they've done things like having the simultaneous translation. They know who to work with on that kind of thing. And they know something about how to make that successful. So it's fantastic there's a huge team of specialists who know how to run conferences in the Linux Foundation and we get to leverage that.

ADAM GLICK: There's also a KubeCon coming up in Seattle later this year. Can you give us any hints as to what sessions have been selected for that?

LIZ RICE: Well, nothing as yet has been selected.

CRAIG BOX: You've still got a chance, Adam.

[LAUGHTER]

LIZ RICE: Well, it's too late to get into the process now. We've literally, this weekend, received our massive spreadsheets as co-chairs, which gives us all the information about all the submissions and how they've been scored by the program committee. So yeah, Janet and I have a few days of burying ourselves in that spreadsheet. And when we emerge like a kind of butterfly out of the cocoon, we will come with this beautiful agenda for Seattle.

ADAM GLICK: Lovely.

CRAIG BOX: Do you have a talk of your own in the CFP process for Seattle?

LIZ RICE: I don't actually. But I do get to-- one of the privileges is I do get to give a keynote. So I will be presenting some content. The other advantage that you get as a co-chair is you don't actually have to decide what you're going to talk about, unlike everybody else in the CFP process. So yeah, I do have an advantage there.

ADAM GLICK: Thanks, Liz. It was great chatting with you.

LIZ RICE: Thank you for having me. My pleasure.

ADAM GLICK: If you want to get in touch with Liz, you can find her on Twitter @lizrice, or on her homepage lizrice.com, and Aqua Security at aquasec.com.

CRAIG BOX: (With a C.)

ADAM GLICK: Thanks for listening. As always, if you've enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on Twitter @KubernetesPod or reach us by email at kubernetespodcast@google.com.

CRAIG BOX: You can also check out our website with our show notes at kubernetespodcast.com. Until next week, take care.

ADAM GLICK: Catch you next week.

[MUSIC PLAYING]