#180 May 19, 2022

KubeCon EU 2022, with Ricardo Rocha

Hosts: Craig Box

Live from Valencia, it’s KubeCon EU! Craig talks to conference co-chair and CERN computer scientist Ricardo Rocha about the event, and what it’s like to be in a room full of people again.

Do you have something cool to share? Some questions? Let us know:

Chatter of the week

News of the week

Lightning Round

CRAIG BOX: Hi, and welcome to the Kubernetes Podcast from Google. I'm your host, Craig Box.


CRAIG BOX: Well, here we are. We managed to make it halfway around the world with all our suitcases before a lovely airline misplaced them on the hour-long flight from Barcelona to Valencia. Thankfully, 24 hours later, we were reunited. It was a bit touch and go there, given I checked in my recording gear.

My first official duty at KubeCon EU was waking up a group of jet-lagged ServiceMeshCon attendees with deliberately bad 9:00 AM karaoke. If there is such a thing as good karaoke, it definitely doesn't happen at 9:00 AM with no water and no fallback speakers. So I simply channeled my inner William Shatner, and the ice was well and truly broken for the conference.

The highlight of events like this for me is always meeting listeners of the show. The modern creator often works alone, recording or streaming from wherever they happen to be, with no idea if anyone will ever see or hear their work. By the end of this conference, I hope to have met as many of you as I can, but I want to share one story from that first day.

A listener of the show came up to me and told me how transformative the show and the things he learned by listening to it had been for his career. Thank you for sharing that story, Hamza. It really makes it worthwhile, and melts away the stress of losing one's luggage.

Another listener told me that he misses me talking about Christmas trees. Only a few months to go, my friend. Let's get to the news.


CRAIG BOX: A survey commissioned by SlashData for the CNCF suggests that there are now over 7 million Cloud Native developers worldwide, and 7,000 of them have descended on Valencia this week. Here at KubeCon, the CNCF announced it has hit 800 members, with Boeing joining as a Platinum Member and Coinbase joining at the Gold Level. Other announcements include the launch of a Prometheus Certified Associate Exam.

Google Cloud has released new tools to make configuration a first-class citizen for Cloud Native developers. Porch, a new package orchestrator tool, has been open sourced as part of the "kept" Project, K-P-T, and an open source plugin for Backstage provides a WYSIWYG GUI experience. Google's Config Sync Tool has also been open sourced, meaning the Anthos Configuration Management Suite is now entirely based on open source tools.

Another project open sourced out of its commercial base is Tetragon, an ePPF-based security, observability, and runtime enforcement platform. The tool, previously part of Isovalent's Cilium Enterprise, detects and responds to security significant events. Those events, including processes, privilege changes, and IO, can be mapped to Kubernetes identities, like namespaces, and pods.

The Envoy Team announced Envoy Gateway, a new project to provide a unified ingress layer for Kubernetes based on their popular proxy. Built on the Gateway API, extensions will be available to directly access Envoy's XDS APIs. The initial sponsors include the creators of Contour and Emissary Ingress. And the new project aims to replace those tools with a common core over time.

A new tool for managing access to cloud infrastructure has been launched this week. Infra is created by the authors of Kitematic, which was acquired by Docker and turned into Docker Desktop. If the name seems familiar, they used it first for a desktop app in 2020.

The team has now pivoted to security. The new tool controls access using the principles of least privilege, saving users from having to manage certificates, keys, or integrations with identity providers. Infra is open source, with a hosted and paid version planned for later.

The Cloud Foundry Foundation has announced the beta of a new Platform as a Service based on top of Kubernetes. Korifi is an experimental implementation of the Cloud Foundry V3 API backed entirely by Kubernetes custom resources. It builds on learnings from several previous integrations, aiming to eventually replace cf-for-K8s and KubeCF.

After its acquisition of security vendor NeuVector in October, SUSE has announced the first open source release. NeuVictor, with a U, includes DevOps vulnerability protection, automated runtime security, and a layer 7 container firewall. The upstream project for Neu Vector, which is to be called Open Zero Trust, has been proposed to the CNCF Sandbox.

In case you thought there weren't enough options for running Postgres on Kubernetes, vendor EnterpriseDB has added another one. CloudNativePG is a new open source operator, joining those from Zalando and Crunchy Data, among others. Aside from its vendor heritage, its unique proposition is that it also intends to join the CNCF Sandbox.

Google Cloud has announced a new Assured Open Source Program, allowing customers access to curated, Fuzz-tested, and SLSA-compliant packages for open source projects used by Google itself or submitted by the community. The announcement also includes a partnership with security vendor Snyk, integrating Assured OSS into Snyk’s solutions. The program will be available in preview in Q3, and you can register your interest via a form.

Finally, in recent guest news, Argo vendor Akuity, from Episode 172, announced a $20 million series A round. Troubleshooting vendor Komodor, from Episode 153, raised a $42 million series B. And security vendor Deepfence, from Episode 173, launched a Deepfence Cloud, a fully managed security observability offering. And that's the news, or at least that which we have time to read. For all the other announcements from KubeCon, please check the show notes in your podcast player or on kubernetespodcast.com.


CRAIG BOX: Ricardo Rocha is a computing engineer at CERN, a member of the CNCF Technical Oversight Committee, head of the CNCF's Research User Group, co-chair of this week's KubeCon EU, but most famous as a guest on Episode 62 of the show. Welcome to the show, Ricardo.

RICARDO ROCHA: Hi. Thanks again for hosting me.

CRAIG BOX: It's lovely to see you again. I think this is the first in-person interview that we've done for this podcast since December 2019. Is it strange for you to see so many people?

RICARDO ROCHA: It is a bit strange, but it's mostly--

CRAIG BOX: A relief?

RICARDO ROCHA: It's been a long time of Zoom calls and virtual conferences. So I think it's well on time to get everyone together again.

CRAIG BOX: How did Switzerland fare in the last couple of years?

RICARDO ROCHA: Actually, Switzerland has not been that bad. CERN is kind of split between France and Switzerland. People living on the French side had a bit of a tougher time. Switzerland, after a couple of months of lockdown at the start, has been gradually opening. And we kind of went back to normal in the last two months.

CRAIG BOX: It kept a neutral policy towards the virus?

RICARDO ROCHA: Yeah, yeah.

CRAIG BOX: Very timely.

RICARDO ROCHA: [CHUCKLES] Pretty much, yeah.

CRAIG BOX: We met in May 2019, at the last proper KubeCon EU. When you were at that event, did you think that, three years later, you'd be hosting the next event in person?

RICARDO ROCHA: Absolutely not, yeah. No, I was pretty happy to participate and to have an active role there. But I never thought I would be here now.

CRAIG BOX: We spoke to you alongside your colleagues, Clemens and Lukas. How are they doing?

RICARDO ROCHA: Lukas has been very involved in the same areas, but he's a proper physicist, so he's been taking physics quite seriously.

CRAIG BOX: I hear he's now a professor for data science.


CRAIG BOX: Technical University of Munich.

RICARDO ROCHA: Yeah. So he just recently moved back to Germany. He's still involved in this area, but more on the physics side now.

CRAIG BOX: OK. And Clemens?

RICARDO ROCHA: Clemens is actually here this week as well. He's still involved also with the CMS Experiment, but also very much interested in the computing side. So he is actually here with us as well, yeah.

CRAIG BOX: Not everyone who's listening to this show will have heard Episode 62, although it was a fantastic conversation, and I do encourage people to go back and listen if they can. Can you give me a quick recap for the listeners of the work that is done at CERN?

RICARDO ROCHA: Right. So very quickly, we run very large scientific experiments. The most famous is obviously the Large Hadron Collider, which is a 27-kilometer particle accelerator.

And we accelerate protons to very close to the speed of light. We make them collide in very precise points where we've built these massive experiments. And we try to understand a bit the nature and the origins of the universe, and a bit better the particles that we circulate in this accelerator.

But in reality, from my perspective, which is an IT perspective, what we do is generate a lot of data. So we need these massive computation systems and big data centers to analyze all this data we generate.

CRAIG BOX: Have you found any new bosons in the last couple of years, or did the collider have to go into lockdown too?

RICARDO ROCHA: We actually were in a long shutdown period in the last couple of years. That has been a challenge also during the pandemic, because we had to do a lot of work on site, and they were kind of limited for some periods. We're just about to restart. We now have beam again for the last two months.

CRAIG BOX: Can't the scientists just pack the collider up and take it home for the weekend?

RICARDO ROCHA: Yeah, I wish. But for now we are not there yet. We still have to build these big machines.

CRAIG BOX: How did the lockdown change the kind of research you were doing? Obviously, you didn't have access to the physical equipment, and the collider has been shut down. What have you been able to do over that period?

RICARDO ROCHA: Actually, for us this period is also a very good period to do upgrades, both of the machine but also of the systems themselves. Without having a run going and the beam, we have a bit more flexibility on upgrades. So this is what has kept us busy.

It didn't really affect us from the IT point of view. We could still do the systems upgrades we needed, and it's been kind of smooth. I think, for the experiments themselves, it was a bit trickier.

CRAIG BOX: Now, am I right in saying that you've been working to get CERN to support more machine learning infrastructure?

RICARDO ROCHA: Right. So in the last couple of years I have been helping a lot. Machine learning is something that is really picking up in our community.

And I've been trying to help on the infrastructure side to make sure that we offer all the resources people need. Machine learning is really something that is very hard to do if you don't have access to proper accelerators. And in this case, it means having them on premises, but also offering public cloud resources.

CRAIG BOX: Just checking — that's GPUs and so on, not particle accelerators?

RICARDO ROCHA: [CHUCKLES] Exactly. Yeah, yeah, yeah. So those are the accelerators that are interesting in this case.

There's a lot of applications for it, from simulation to different ways of doing data analysis. So there has been two parts. Here is offering a good centralized machine learning solution that our users can rely on, and at the same time to make sure that we can integrate and burst into public cloud resources. So I've been investing quite a bit of work there as well.

CRAIG BOX: What, if anything, have you had to do differently to support this kind of work, on top of Kubernetes, versus on the more traditional infrastructure?

RICARDO ROCHA: I think Kubernetes is actually a good opportunity we have here, the fact that we already had quite a bit of our infrastructure migrated there. The access to public cloud resources specifically is made quite a lot easier when we have an interface like Kubernetes, where you can go to GCP and have a managed Kubernetes, and go to Azure and have a managed Kubernetes. So these were things that facilitated a lot. And on the other side also there's quite a lot of frameworks that allow you to do machine learning-- things like Kubeflow that build on Kubernetes. And we had the expertise on Kubernetes so we could more easily deploy a service like this.

CRAIG BOX: There are now data on Kubernetes communities. Do you think we'll get to the point where either all data is in Kubernetes or where it stops being strange to run storage in ML in this fashion?

RICARDO ROCHA: There are challenges that are related to the tools you use, and there are challenges that are related to physical restrictions. One thing we have problems with is when we start talking about petabytes of data. It's not really the access to it necessarily that is the only challenge. It's really moving data around.

So when you start having to cross boundaries, the data movement, and the data gravity, is the biggest challenge. We have a lot of experience with this, from the great days, so it's something we know how to handle. But integrating this with modern technologies, public clouds, it's something we have to work on still.

CRAIG BOX: You are now on the CNCF Technical Oversight Committee, and you are the leader of the CNCF Research User Group. I didn't even know we had one of those. What does that group do?

RICARDO ROCHA: Yeah. So the Research and User Group, it actually was an idea that came out of Barcelona as well in 2019. I was having lunch with Bob Killen, who is now at Google as well, and a couple of other people. We thought, there's a community that is very interested in making use of Kubernetes, but has some specific requirements.

So if we think research here, we are thinking about batch workloads, about running notebooks, and integrating with the traditional storage systems in-house, things like this. So we decided to form a group that focuses on this. We meet biweekly, and we cover all these topics that I just mentioned, and we get a lot of end user reports. So it's very much end user-focused.

CRAIG BOX: Is there a particular difference in the kind of batch work that is done for research versus the kind of batch work that's done for other applications?

RICARDO ROCHA: If you ask researchers, they will very often say yes.

CRAIG BOX: And they'll twiddle their glasses while they'll do it, and say, I think you'll find that they're very different.

RICARDO ROCHA: Yeah, exactly. They will say, yeah, that's enterprise IT, we cannot rely on it, or something like that. But when we start looking at what modern IT uses batch for — and in the Research User Group, we have companies from financial institutions that do exactly the same as we do, yeah. So I don't think there is that much difference.

CRAIG BOX: Do you think there is a chance for interoperability between your group and other groups, looking at the problem space for financial or other enterprise use cases?

RICARDO ROCHA: Absolutely. I lead this group, together with Jamie Poole. And he's from G-Research in the UK. So we already had them coming to CERN and discussing a bit more in detail different topics. So there is a lot of commonality there.

CRAIG BOX: And we mentioned you joined the TOC. Have you been involved with any particular projects since joining that group?

RICARDO ROCHA: Yeah. So one thing I'm very keen is, again, to make sure that research, the use case goes through in all the projects in the CNCF, not only Kubernetes. So I've been involved in incubation of different projects. I actually helped with the Volcano one as well. I sponsored that one.

And I try to be quite close to these kind of projects. Of course, I help in other areas as well. But it's something that I'm very keen on giving a contribution in that area.

CRAIG BOX: I'm in the process of submitting a project to the CNCF myself. What can you tell me about what you did with Volcano, for example? That's a batch framework for Kubernetes that came out of China, as I understand. What was the process like for that submission as it went through the stages to become an incubating project?

RICARDO ROCHA: The first step is really this sandbox, which is a low barrier entry to the CNCF. And I think that's a really nice way to get more exposure to the projects, but also to go through the minimal requirements to build a community, basically, and to make sure that the project has a good path for sustainability.

Then, incubation is really the first big step. And this involves due diligence that has a couple of steps to check, in terms of contributions to the project, diversity in terms of those contributions. But also, there's a step to check with end users.

So one of the things we do during the due diligence is to actually reach out to end users of that project, and understand which ways they rely on the project, which phase they have — if it's just testing, or if they are about to go to production, or are already in production. And it's really to have a good view of the project, and not only to see if they already are at the incubation stage, but to summarize where the next steps for the project can be, and how they can better fit with other projects in the ecosystem as well. So it's more a summary of what should come next for the project.

CRAIG BOX: As an end user yourself who is working on batch workloads on top of Kubernetes, were you able to give any specific advice to the Volcano Project, or was there anything that you were able to learn from how they do things that you were able to apply to your particular use case?

RICARDO ROCHA: I think the most interesting part in that specific project is that we get a lot of exposure to use cases, in Europe, in my case, and in the US. In that case, they have a very, very strong community in China. And it was incredibly interesting to see the different types of uses they have of a project like that.

I think that there is a lot to learn from enlarging our view of where and how things are being used. I think that's where I learned the most there. And then it's a thriving community as well. So it was an interesting process to go through.

CRAIG BOX: For Istio, I'm working with two of your colleagues on the CNCF. They will be doing the due diligence. Will you just be taking their recommendation, or is there something else that you and the other members will want to do, looking at projects that you're not actively sponsoring yourself?

RICARDO ROCHA: We do have one person assigned, or a couple of people, depending on the actual project, to go through more in detail the due diligence. But then, even before it opens for public comments, where everyone in the community can give feedback not only to the TOC, we do have some discussions.

Also, I haven't been in the TOC for very long. So the first couple of times I've done due diligence, I always go back and ask for recommendations and suggestions, look back into other previous processes to try to learn a bit. So it is a shared effort, but there's clearly someone that has to guide it.

CRAIG BOX: Another shared effort is the co-chairing of the conference, which we're at this week. How did you become a co-chair for this and subsequent KubeCon events?

RICARDO ROCHA: That's an interesting question. I'm not sure I know exactly. I think it came out of being active specifically in the end user community. I think there was a will, maybe, to have more end users represented, not only at KubeCon but even as chairs. And I think the current lineup, it kind of shows that.

CRAIG BOX: Well, there used to be two chairs for each event. I think there are three now. It feels like they wanted to have someone representing end users, which — let's be honest — are probably the majority of the people paying to attend the event.

RICARDO ROCHA: Yeah. I think it's quite important. Because in the end, the role of the chairs is also quite a lot in the program committee. And having end users represented there will ensure that the end user overall representation will also be improved.

CRAIG BOX: We spoke with your colleague Jasmine James, who is the program co-chair of the last event in North America, and carrying on to this one. We talked to her in Episode 165. What guidance has Jasmine given you throughout the process?

RICARDO ROCHA: I think, for me, I had the opportunity to shadow North America already, KubeCon North America. So I got involved early. So that was incredibly nice, because I could join and listen to how things are structured.

Things are overwhelming at the start, the amount of work and the amount of people involved. But it was very interesting to see not only Jasmine, but at the time, Stephen and Constance as well. And I think that that really paid off, to be able to shadow and see how they were working, how they were doing things, to learn how we could best structure ourselves. I think one thing we have constantly in the calls is, when we had doubts, we always asked Jasmine, how was this done before? And that's constant.

CRAIG BOX: Do you have someone shadowing you for this event?

RICARDO ROCHA: Not yet, not yet. But hopefully soon, yeah.

CRAIG BOX: Now, you mentioned the main responsibility being leading the program committee and going through the session review process. What can you tell me about the number of submissions for the event and the ratio of accepted submissions and so on?

RICARDO ROCHA: I did China before this one, which was virtual only. It's a much bigger event this time.


RICARDO ROCHA: I think the ratio is pretty hard. I was involved in more scientific conferences before, and I always found that those were pretty tough. And I don't think here is any different.

CRAIG BOX: I'm told that this is getting up there, in terms of the number of attendees and submissions and so on, to one of the top 10 events in the world, in the tech world at least.

RICARDO ROCHA: Yeah, I'm pretty sure. It's incredible, the amount of submissions we get, the quality of the submissions as well. I think it's been really, really hard to go through it and select. But that's a good problem to have, I guess.

CRAIG BOX: Were you able to define the buckets that you wanted to see submissions to come in? You mentioned, obviously, representing end users. Was there an active goal to try and say, these are the kind of submissions that we want, and communicate that out to people before the event?

RICARDO ROCHA: Yeah, I think that there is a couple of things that are quite important. The end user representation is one of them. The other one was that, as this is KubeCon Europe, we also wanted to have good representation of the more local communities.

So this is something that we also try to be proactively selecting in case of doubts. So I think those are important to build these communities around, to have this kind of representation. Again, coming back to the previous topic, this was the first time we have a dedicated research tract.

So that's something I'm really happy about. We were able to push this one for the first time. Let's see how it goes. The first round we had pretty good submissions, but we hope to grow this as well.

CRAIG BOX: Were you reaching out to either the Research User Group or the broader community to try and actively solicit submissions in that space?

RICARDO ROCHA: Yeah, absolutely. So the CNCF Research User Group was the easy one. But what I also did is, from the contacts that I've built in the last 10, 15 years, was to make sure that all the people that could be interested had submissions sent. So that was pretty good. We also have a co-located event for Batch and HPC.

CRAIG BOX: Were you able to identify any themes across submissions from the general public for this event and perhaps group them together in any way?

RICARDO ROCHA: Yeah. So there are tracts that are very popular. Obviously, security has been hugely popular, because of recent events. We saw a lot of interest also in the business value.

And of course, tracts like operations, and configuring, and extending Kubernetes, they've always been popular. There is some effort for the CFP, for North America, in regrouping the tracts, to get a more homogeneous submission across tracts. I think this is something we have to work on to improve the event, is to make sure that we don't have tracts that are really overloaded with submissions, and to make it more clear to people where they can contribute the best, in which area, so that not everyone ends up submitting to the same place.

CRAIG BOX: Do they give you data on what sessions people attend? If you are someone who's interested in a particular topic, are you more likely to go to all of the sessions in that tract, or do you see people pick and select things that are interesting to them across many different tracts?

RICARDO ROCHA: That's an interesting question. I am not completely sure we have individual data. We probably have what people expressed they would like to attend.

I'm not sure that corresponds always to where people actually go. From personal experience, I know I very often select multiple talks at the same time, and then make a decision at the last moment. But I wouldn't expect that people stay in one tract. Especially for end users, they will definitely be jumping around.

CRAIG BOX: I'm told that there are 7,000 people here in person this week, and of course many thousands more online. Did the idea that this event would be in person change how you planned it?

RICARDO ROCHA: Yeah, the fact that this was getting everyone back together kind of opened up this. For example, for the Contributor Summit, I think that's something that having it back live makes a huge difference. And you need to make sure that there's enough time and space for people to network as well, all the side events that have to happen.

So it's a very different game from just doing a virtual event. You can see this in that there's a ton of smaller events happening on the side. I think that that's a pretty big challenge for the events team as well.

CRAIG BOX: Yeah, you mentioned not knowing what sessions you want to attend. I've had problems where there are whole co-located days that I'd love to attend that are on at the same time. And then I think a lot of people who are more involved in the contribution side of things, they see the event as being everyone in the same place, and there are people who don't go to any sessions at all. You can go and watch the videos of these when you get home, but you can't have the face-to-face interaction again.

RICARDO ROCHA: Yeah. That was another change from, for example, North America. That was a change in terms of organization for the virtual sessions.

We now keep big rooms where people just can sit and follow the virtual sessions in one place and choose whatever they want. In North America, there were dedicated rooms for those virtual sessions. So that was also opening up quite a lot of possibilities, in terms of logistics, to host additional sessions.

CRAIG BOX: I remember at the last North America KubeCon I went to, there were puppies that were brought out so that people could pet, and enjoy them, and possibly adopt them, I'm sure. Is there anything cool that's happening at this event that you know of, anything different, or anything with a local flavor?

RICARDO ROCHA: I actually don't know. That sounds pretty cool. I wasn't aware of this.

CRAIG BOX: Keep an eye out for puppies. See if there are any around.

RICARDO ROCHA: Yeah, I think that's a good idea for Detroit.

CRAIG BOX: I may have to disinfect them every time. It may not sound so nice. Finally, you summarize your interests on your website as code, mountains, and flying. We've talked about the code part. What about the mountains and the flying?

RICARDO ROCHA: I think the mountains and flying kind of go together as well. I've been passionate about aviation for a long time. I'm a very keen glider pilot as well.

And I live in the Alps, so I basically fly in the Alps in big mountains. And this built into me also a passion for mountains themselves. So I do a lot of mountaineering as well. I spend a lot of time in the mountains.

CRAIG BOX: Are those two things related? Do you fly yourself up to the top of a mountain and then ski down it?

RICARDO ROCHA: Not quite like that.

CRAIG BOX: Probably hard to land a glider at the top of a mountain.

RICARDO ROCHA: Yeah, yeah, yeah. It's been done, but it's a very good way to lose a glider as well, so no. But I was born close to the ocean. But when I moved to CERN, I kind of developed this passion. I even merge it with some code. Some of the pet projects I have for code as well are related to flying.

CRAIG BOX: I hear the US Department of Defense put Kubernetes on an F-16. Have you managed to put it onto your glider?

RICARDO ROCHA: Yeah, no. That would be probably too much. I think, when I'm up there, I like to kind of break from the rest. So it's something I've been able to keep separate until now, yeah.

CRAIG BOX: All right. Well, thank you for putting on a great conference this week. And thank you very much for joining us, Ricardo.

RICARDO ROCHA: Yeah, thank you very much.

CRAIG BOX: You can find Ricardo on Twitter by spelling his last name backwards and then putting Porto on the end, or you can find him on the web, at RicardoRocha.io.


CRAIG BOX: Don't forget to check the show notes at KubernetesPodcast.com, to read all the rest of the KubeCon news. You will also find transcripts there a few days after each episode.

Follow us on Twitter, @kubernetespod, or share the show with a friend. If you would like to rate us in your podcast player, it will help the algorithm know we're there. If you'd like to suggest a guest, drop us an email at KubernetesPodcast@google.com. Thanks for listening, and we'll see you next week.