CRAIG BOX: Hi, and welcome to the "Kubernetes Podcast from Google." I'm your host, Craig Box.

[MUSIC PLAYING]

Well, I've been telegraphing it for a little while now. But it's time for the down low. I'm leaving Google. You heard it here first. Unless, of course, I told you in person.

Next week, you'll learn more about the future of the show. But let me just posit that Trevor Noah announced he was leaving "The Daily Show" this week. Coincidence? You decide.

I'd like to share an anecdote I wrote in my departure email. My first work trip was to Seattle, in January, 2014. I remember writing down two actions on a Post-It note.

"Make a Settlers of Catan clone with all these hexagon icons," and "Go learn about this new Project Seven thing." I never got round to the first one. I'm going to miss working at Google.

But the thing I'm going to miss the most is talking to you all each week. You're a good-looking bunch. I like to think you're a self-selected group of the perfect intersection between interested in the thing I know about and enjoys my unique take on things. Given that, if you haven't already, please follow me on Twitter.

Very soon you will be able to learn about the incredibly exciting new thing I'm moving onto. If you find my ideas intriguing, you can subscribe to my newsletter. And Adam and I have been quietly threatening to host a parenting podcast for years now. So please share your enthusiasm with us to get that over the line.

This show, of course, has a long history of guest hosts. So you might hear from me again. Or you might find I pop up in some other interesting places. Either way, it's been a privilege to talk to you for almost 200 episodes. And I want to thank each and every one of you, for your support. Let's get to the news.

[SYNTH MUSIC]

Google Cloud continues to close the gap between classic and autopilot modes on GKE, this week adding support for GPUs in Preview. The NVIDIA T4 and A100 GPUs are now available with no extra configuration or drivers required. Just specify your GPU requirements and your Pod configuration, and Autopilot handles the rest.

As with all Autopilot pods, you're only billed for running time. So you don't have to pay for setup, tear down, or wasted resources on a GPU node. Prices are published, and start at $0.12 per pod hour, on top of the regular vCPU and memory cost.

A CVE has been published describing an issue where Rancher stored sensitive values in plain text on various Kubernetes objects. Passwords, API keys, and Rancher’s service account token were stored on custom objects like Cluster, and the management.cattle.io API group. New versions of Rancher fix this error, with a one-time migration at cluster start up. And a script has been published to rotate tokens.

The results are in for this year's Accelerate State of DevOps report, published by Google Cloud's DevOps Research and Assessment team. This year's report focused on security, with findings including the biggest predictor of an organization's software security practice was cultural, not technical. High-trust, low-blame cultures were significantly more likely to adopt emerging security practices. And teams who established security practices have reduced developer burnout and are more likely to recommend their team to someone else. Security, eh? Sounds interesting. As always, a copy can be yours for the low, low cost of your email address.

Congratulations to the 27 Summer LFX Program CNCF interns. The group worked on 15 projects, including Kubernetes, Pixie, and Chaos Mesh. And the stories of seven of them are published on the CNCF Blog this week. We hope this is the start of a lifelong love of open source.

Finally, the Kubernetes Project is preparing for the release of a third-party security audit later this month and is using the opportunity to look back at the outcomes from the last one. Craig Ingram from Google's Kubernetes hardening team has been tracking 37 issues disclosed in the 2019 audit. A blog post on kubernetes.io expands on that work and points out which issues are closed or whether further design work is required. Twelve items are marked as requiring attention. But all are deemed of medium importance or lower. And that's the news.

[MUSIC PLAYING]

Dan Stein is Head of Engineering at General Bioinformatics, leading a team of data scientists working on AI technology for DNA sequencing. Dan Stein is also DJ Fresh, a multimillion-selling artist with two UK number-one records, and co-founder of Breakbeat Kaos Records. Welcome to the show, Dan.

DAN STEIN: Hi, Craig. How's it going?

CRAIG BOX: Great, thanks. How are you doing?

DAN STEIN: Very good. Very good now that we've managed to get our technical recording issues out of the way. I guess you probably didn't think for a second that you'd have problems with somebody who's got a history as a recording engineer and music producer getting the Zoom microphone set up working. But hey-ho.

CRAIG BOX: I have been reliably informed that you're talking to us on 10,000 pounds worth of microphone.

DAN STEIN: [LAUGHS] Now that I've finally got my equipment set up properly, you're going to be hearing me very well, I'm hoping.

CRAIG BOX: If Dan sounds like a potato, it's entirely his own fault.

DAN STEIN: That's very true.

CRAIG BOX: So my first question to you is obviously, do your cool friends know you're a nerd?

DAN STEIN: I don't have any cool friends that I know of. I mean, they might pretend to be cool. But they're definitely not cool.

I mean, as far as I'm aware, I think everybody that I know is a bit of a geek. Although they would argue otherwise, I'm sure. I've always been really into technology. Obviously, with my sort of history in drum, and bass, and music production, even the genre of music that I've been wrapped up in mostly all these years is very high tech and it's all about being on the cutting edge of music production.

So most of the people that I spend time with are geeks, in my opinion, whether they realize it or not, including you, Craig. Sorry.

CRAIG BOX: Absolutely. First to hold up my hand. We've had classically-trained musicians on the show. We've had people who had to choose between computer science and attending conservatory. I think there's probably a clearer path between electronic music and tech, like you say. Tell me about how you got the bug.

DAN STEIN: I remember being five or six years old. And I don't want to give my age away here, but I'm not a Z Gen, or whatever they're called, person. I'm actually quite old.

So my dad had an Atari 800. And he brought this thing home. And he would play the game with the paddles. Do you remember the paddle game, where you have the two paddles?

CRAIG BOX: "Pong," maybe?

DAN STEIN: "Pong," in my opinion, is still far better than any games that are released at the moment. So yeah, he had a couple of Atari games. And I was messing around with his computer.

There was a cartridge that was the basic programming language, Atari BASIC. And I was like, what's this? My dad had absolutely no interest in programming at all.

But somehow, I managed to get this cartridge working. And I think I went and bought a programming magazine that had some Atari code in the back.

Back in those days, that was what you did. You'd buy a magazine. There'd be like some code in. You'd type it in and then you'd learn from that.

So I did that quite a bit. I always thought that I was going to be — we're talking sort of like, early even '90s here. I guess I grew up in the sort of mid '90s. I was in my teens. So I thought I was going to be a quote, unquote, "programmer." That's what I was going for.

At uni I met a guy who played on the local pirate radio station. And I had just been messing around with OctaMED on the Amiga, making music with that. I played him some of the music that I'd made. And he was like, man, this is like, really cool. You should come on this pirate station. And you know, can you DJ? And I'm like, yeah, kind of.

CRAIG BOX: Why not?

DAN STEIN: [LAUGHING] Next thing I knew, I was on pirate radio. And they were like, what are you going to call yourself? And I'm like, I don't know. I'm so OCD that that could have taken 20 years to get an answer to that question.

Luckily, because I used to scratch a record that said fresh, somebody said, why don't you call yourself fresh? Because you're always scratching that fresh record, dig-a-dig-dig-dig-fresh. That was it. So I was named.

And then not long after that, somebody else heard some music that I was doing. And I got signed to a small underground record label called Kickin' Underground Sound. This is, like, back in '98 or something, 'round about when I was at uni, as I said. So the rabbit hole split in two, and I went off down one fork.

CRAIG BOX: I know Fatboy Slim controlled many of his hits on an Atari ST. You went from the Atari to the Amiga. That's a good choice. As a Commodore kid, I can definitely agree with that.

But were you using the software to do chip-tune stuff and the kind of thing you could do from the actual computer? Or were you looking at using the computers that control other electronic instruments?

DAN STEIN: It was chip chains. And then, later on I brought a sampler. I actually brought a sampler with my uni loan. [LAUGHING]

CRAIG BOX: As opposed to textbooks or whatever you're meant to buy?

DAN STEIN: Well, you're supposed to spend it on accommodation, and food, and important stuff like that. But I just went out and blew the whole thing on a sampler. But to be fair, I paid for the sampler within six months.

Then my career took off pretty quickly. So I was using this sampler and sequencing the sampler from an Amiga, initially. Then I got an Atari ST, actually, because Ataris have slightly different software. So OctaMED on the Amiga was a tracker, which is a really cool type of sequencing software. But it's not really what most people were using at the time, which was an Atari ST. Then later, when things went to audio, I moved to like, a Mac.

CRAIG BOX: What did you actually go to university to study?

DAN STEIN: I went to study business IT. I mean, that was the only course that was available to me where I wanted to go that was remotely in the lane of what I wanted to do. My parents weren't keen on me doing IT, strictly.

Back at that point, it was something that people didn't really understand. This is even before the dotcom boom. So there was really no awareness of this being something that had all this amazing potential future to it that we see today.

I kind of settled on business IT as a bit of a compromise with my parents. And actually, really hated most of it, to be honest. I was a bit put off.

I went and got a job, a programmer for I won't say which band, but for a small high street band, helping with their website, initially. Then my music stuff was just taking off really, in the background. I decided to go a different path, yeah.

CRAIG BOX: There were a few different ways that tech and music came together for you here. You were an early internet pioneer, setting up an online music forum in 2001, which is still operating today. What were the mechanics of doing that, at the time? Was that something that you had to write the code for yourself?

DAN STEIN: When I decided to focus full time on music, I set up a record label. We also set up a forum called dogsonacid.com. It was originally the forum that was part of our label website, and it became quite popular.

I decided to try and separate it from our label so that it was a bit less opinionated and a bit more inclusive. Came up with this name, Dogs on Acid, because I thought, I want a name that you don't forget and that worked.

We had about 100,000 registered members within the first couple of years, which, at the time, was quite a big deal. I think it was the biggest music-forum-based platform on the internet at the time.

I was working, I would say, as a founder and as a product manager, really. So I wasn't writing code at that point. Although I grew up writing code and loved writing code, for some reason I guess that's just what happens, isn't it? You're caught up in the day-to-day presentation of the product, and the platform, and networking, and building it, and doing all the advocate stuff.

So even then, I wasn't really writing code at that point. I also had another startup with a guy called Chris Parry, who was the founder of XFM. Don't know if you've heard of XFM. They're a pretty big radio station in the UK.

CRAIG BOX: Yep.

DAN STEIN: And Fiction Records, who signed The Cure, who were one of my favorite bands. So Chris invested in a company that I started with another guy called Chris McNabb, who's quite a well-known security expert and network security.

We set up this company that was protecting music copyright on the peer-to-peer networks. We had the only solution at the time for being able to remove copyright material from BitTorrent, which was massive. So we could basically protect your content on BitTorrent.

We had about 15 engineers working for us at the time. Unfortunately, that didn't work out because major record labels were tightening their budgets. And we were trying to sell them a very expensive product that needed a lot of servers behind it, so our running costs were really high.

But we had a couple of really big clients. We worked with EMI, Buena Vista, Disney for a couple of years. So I've had like a really mixed career, straddling both tech and music, I guess. Yeah.

CRAIG BOX: Is a record label like a tech startup?

DAN STEIN: I think there's a lot of skills, if that's the right word, that I would lean on managing a record label that is similar to the ones that are involved with managing a team of engineers. One of the things that I kind of draw a parallel between, but I hope that people that I work with find positive about my influence over our process, is that in music, you're taught to look for the best in people. Music is actually, when you think about it, incredibly inclusive like at a formative level.

Nobody knows what the next big thing is going to be. So it could be anything. As a result of that, as somebody who's a good A&R and somebody running a record label, you have to be very open minded to who might come up with the next big idea, or sound, or genre.

I think that encourages you to think quite inclusively and to be quite sort of forgiving. I definitely feel like there's a parallel there with music. There's a sense of like, we're in this together, and we're all going to extend what we each have to offer, and try and find a way to work together as a team.

CRAIG BOX: It feels a lot less like a zero-sum game and that you have people collaborating with each other as a matter of course. And success breeds more success. And see people like, I've been watching the Taylor Hawkins tribute concerts over the last couple of weeks. And seeing the collaborations and people you wouldn't expect, they're all sort of touched by this one guy, is amazing.

And I do feel, in technology, and maybe especially in our branch of it, there's a little bit of like, well, I'm working on a project and your project has to fail for mine to succeed because you're all kind of filling in one niche. And that's something that I think is a real shame. I don't quite know what we need to do about that as a community. But you do see that with a lot of open-source projects, for example.

DAN STEIN: I know what you mean about that, with open source, but a lot of that's just human nature, right? And you can't really do anything about human nature. I think really, overall, from what I've seen, there's so many things in the tech industry that a lot of the world can learn from. I was just thinking yesterday, if only there was a Stack Overflow for politicians. Our politicians could really use that right now. Do you know what I mean? Because I know if you've been watching what's been going on. [LAUGHING]

CRAIG BOX: I've been following it very closely. What kind of questions and answers, do you think?

DAN STEIN: If I was to think of abolishing the high tax rate in the UK, would that be a good idea? Imagine if you could just have an army of people that just came back and go, no, well, actually, that was tried in this year, and that didn't work because of this, and actually, if you look at, you know what I mean? So I think there's a lot of resources that we have. And I think open source, obviously being the most obvious of them, but Stack Overflow, which is like open-source knowledge sharing, I think there's a lot of really positive stuff that comes from the tech community, at least from my perspective.

I'm constantly telling people in the music industry about Jira, about Agile and Stack Overflow, all those tools that we just take for granted that we use every day, obviously not used so much in other fields, like music.

CRAIG BOX: One of the things that you find with people as they move to more senior roles in technology is that they're not so much writing code anymore but they're helping other people. They're throwing in the odd suggestion here and there. I do wonder if there's a parallel there to production, to running a label and so on. And I also wonder if people who do that sometimes — in tech, you get people who say, I've done being a manager for a while, I want to go back to writing code, I want to get closer to it. And I wonder if that's a way to pick back up your story as to when you started actually making music rather than perhaps the facilitating that you've been doing to this point.

DAN STEIN: I was always making music as I was running record labels. And I've always been somebody that's had a hand in lots of pies. At the same time, I'm always kind of taking on too many things, really. Throughout the early 2000s, I set up a record label called Breakbeat Kaos with a guy called Adam F, who I was a big fan of. He'd had some success producing some really big hip-hop stars in the US, like LL Cool J, Method Man, Redman.

We set up Breakbeat Kaos together. And at the time, the mission was we wanted to take drum and bass, which was a massive underground genre in the UK but we felt didn't get enough support from the mainstream in the UK, we wanted to try and get that onto mainstream radio and onto MTV. Gives you an idea of how old I am, MTV. That was central to the problem at the time.

CRAIG BOX: Back when they used to play music videos?

DAN STEIN: Exactly, yeah.

CRAIG BOX: I've done my research. You're only a couple of years older than me.

DAN STEIN: Yeah, so you're also quite old. [LAUGHING]

Or we're both quite young, depending on how you look at it. Let's go with that. We wanted to try and expose this music that we had so much love for and the community around the music as well because there's a real strong community within drum and bass. It's like a lifestyle. And we wanted just to try and get that out there.

So we decided to hire top-end PR teams and really approach drum and bass the way that major labels approached other forms of music. We signed a band called Pendulum. And we brought them over from Australia. And they had a multi-platinum album with us called, "Hold Your Colour." Although there were other previous commercial launches of drum and bass, for sure, in the charts, I think that was probably one of the key moments for the explosion of drum and bass into the mainstream that we've seen over the last 12, 15 years or so.

We were working with Pendulum, Nero, Chase and Status, artists like that pretty early on. And I basically was planning on actually moving away from music and had written an album called "Kryptonite." I had, on that album, a track called "Gold Dust." At that point, got ill. It wasn't cancer, but I had another tumor. I was quite sick for a couple of years. I didn't know what the problem was. And eventually, they found out that I had this really rare tumor called an insulinoma.

And I was in hospital having that operated on when "Gold Dust" came out. It just smashed up the chart. So basically, it didn't get into the top 10. But it was the highest-selling track that year not to get into the top 10. And it just consistently stayed in the top 20 like week after week after week after week. And then I went on after that to have two number ones — technically, three number ones. There's one that's the number one on downloads, "Gravity."

That track sold more than all of them put together. It was obviously like a really amazing time for all of us, especially for me, but also for the genre because suddenly we were getting this mainstream recognition. And suddenly all the biggest in the UK were putting drum and bass in their tracks, Tinie Tempah, loads of people. So it was a really, really exciting time. Felt kind of like the culmination of 15 years of hard graft with the record label.

CRAIG BOX: Tell me then, while you're making this music, are you still programming software or are you finding a way to scratch that technical itch?

DAN STEIN: Not during that period. There was a period where I was using things like Kyma, which is a sound-programming language, effectively, but it's using a UI. Or, like, MaxMSP, I would use. There was a period during the 2000s where I picked up C++ and started learning C++ for a couple of years. I'd hit a point and think, well, what am I going to do with this?

And it's like, we had our website. We had Dogs on Acid. And we had people working for us, doing that. So I always felt like I'm going into meetings saying, right, this is how we're going to redesign the front page, or we're going to have this new feature, we're going to do X-Y-Z. I felt like I was kind of too far detached at that point, which is kind of touches on what you're saying.

In 2015, I had cancer, which I've totally recovered from. I had thyroid cancer. And I had to lose my thyroid and my parathyroid glands, which meant that I have to be on some pretty crazy medication. So there was a period for a couple of years where I was very chemically all over the place whilst I was adjusting to this medication. And I decided to stop touring because it was just impossible to tour. I mean, there were days where I literally could not get off the couch because of my thyroid levels. I was just so exhausted. I just couldn't move.

I had this really grueling touring schedule at the time. And in the end, I had to just basically put a sword through the whole thing and stop touring. And so at that point I thought, what do I do now? Well, maybe I'll do music production. Maybe when I stop touring, stop basically being an artist, maybe I'll focus more on the record label side.

And I tried a few things for about a year. And just wasn't really feeling fulfilled. I felt like I wanted a real change. And I think after having cancer, which I'm sure if anybody's listening to this has had any kind of life-changing illness, would appreciate that it can really change your perspective on things in a really big way, something that helped me to realize that being successful isn't the only thing that's important.

I was on a ladder where I was surrounded by people whose goal was just constantly to get records to number one, to get to the main stage, to headline this, to headline that. That's great. But you have to make a lot of sacrifices to keep that up. Like an athlete, right, if you want to be the number one boxer, you have to sacrifice pretty much everything because everybody you're going up against will sacrifice everything. And that's what you're going up against.

So for me it was the moment of sort of thinking, well, hang on a minute. There's so many things in my life that I've yet to experience and so many things that I've put to one side to try and chase all the things that I've been chasing. It gave me a chance to think back over what I'd done so far in my life and the areas of my life that I wanted to focus on more.

And obviously, family and friends was a really big one. I'd really missed almost every wedding. I was always away. I was always busy. And luckily, my family and my friends were very understanding about that. I managed to rekindle relationships with most of them.

But the other thing that I'd also neglected was this love for programming, just raw programming. And I think, it's like I was saying before, working on Dogs on Acid scratched that itch to some extent because I'm building a product with engineers. But it was actually the engineering part that I missed. It was the hands on that I missed. I thought, what am I going to do about this?

So I was messing around with the Spotify API. And I wrote a little program that would find unsigned popular tracks on Spotify that some friends of mine were interested in, in the music business. And then I wrote a couple of little tools like that. I wanted to get back into it but not as a founder. I wanted to get back into it as an engineer and to be taken seriously as an engineer and work my way up the chain that way.

I basically found out about boot camps and went and did a three-month Ruby boot camp, which was depressing because I wasn't really that keen on Ruby and I wanted to do Node. But I couldn't find a Node boot camp near me. So I went and enrolled in this We Got Coders [INAUDIBLE].

And I went and did this course for three months. And then at the end of it, they had a work-placement program. So everybody was being placed into jobs. And they were like, do you want to do this? Because obviously, everybody on the course kind of knew who I was, which was a bit awkward. And it was kind of surreal. I was just like, well, yeah, why not? Everybody else is, so I'll do that as well.

So I went on the work placement thing. And I was offered a job as an engineer at a company called HT2 Labs in Oxford who were building analytics platforms for learning. I went in and took this job and started working for them. The first day that I went in, I was walking upstairs and the lead engineer, Ryan, walked down the stairs past me and he said, DJ Fresh, it's going to get louder. And I was like, oh, man, this is going to be really hard.

So it was really weird, you know? It's like everybody kind of knew my music. Everybody knew who I was. And I think people were just tripping out, like, what is going on here? Why is he here doing this relatively junior job in this company? But I just absolutely loved it. I was just machining through.

I helped them build some pretty cool stuff. And there was a data scientist there who was supposed to be building a machine-learning model for them. I guess because he was heavier on the theoretical side, he was struggling with the practical side. He left, and then I basically built it for them in my free time. And they gave me his job.

They won some awards for that platform that we built together. Then I was, I guess, poached by another bigger company called GLG Expert Networks, which is a pretty interesting vertical that I'd never heard of before, you know, 250 engineers, big multinational corporation. I went to work for those guys as a principal engineer for a while. And then I landed up at General Bioinformatics, where I am now. I'm managing a team of seven data scientists, engineers, some of whom are bioinformaticians and genomicists as well.

So it's pretty big data, tens of billions of rows in some of our databases. But yeah, just absolutely really loving it. And I guess lots of people say to me, including people that I work with, are like, are you going to start your own company? You're obviously going to start your own company.

But the thing is, as anybody that's a founder or that runs a company knows, it takes over your life, right? And you can't sleep at night. And the great thing about working for somebody else is that you can — as I say this, I was working until 11 o'clock last night on something. But generally, you can put it down at the end of the day. You don't feel guilty about it. You do your best work, and then you go home.

And as part of having a richer tapestry in my life, this, at the moment, fits really well with what I want to do. And I guess maybe, one day, I might come up with something and want to go and start my own company with all the things that I'm learning at the moment.

But I thought also, it was really good to just have a chance to be injected into the realm of UK tech companies, rather than coming at it as a founder, because I think as a founder, you tend to collect people to you that think the way that you do. And so in some ways, unless you're really looking, you don't learn best practices that you do from working in other companies and moving around a bit over time.

CRAIG BOX: Where did the Kubernetes piece come into it for you?

DAN STEIN: Strangely, I was working at a company. I won't say which one, just in case. But the CEO had built his own Kubernetes basically. I won't say what that's called either. But it's open source, like a containerization.

CRAIG BOX: Everyone was building one for a while.

DAN STEIN: Right, yes, like a container orchestration platform framework. And that was what we were using internally for like blue-green deployments and, we were working in a Kubernetes-like way with a Kubernetes-like technology that wasn't Kubernetes. And that was really what, I suppose, exposed me to that pattern for the first time just from a programming point of view.

I've always had this weird kind of obsession with just autonomy. And the concept of lots of nodes operating autonomously in groups is just something that really tickles my brain for some reason. So I am a geek, I guess, right? I've just basically proved that with that statement.

CRAIG BOX: So this company, which we shall not name, is running something that's Kubernetes-like but not Kubernetes. Where do we actually move on to the real thing?

DAN STEIN: When I started working at GB with a small team of scientists — predominantly, bioinformaticians and genomics, data scientists, and engineers. And we work with external groups ranging from small, scientific startups with requirements for NGS-sequencing analysis right up to really large, multinational enterprise agtech companies.

And as I was saying before, it's pretty big data. So we have tens of billions of rows in some of our databases because it's gene sequences, transcripts, transcriptomes, genomes. I don't know if you know much about gene sequencing, but every bit of information that you find in the human genome has to be stored as data. And as you can imagine that, it's quite a lot of data.

[LAUGHS]

CRAIG BOX: Well it's only like, the four letters though. Isn't a lot of Gs, As, Cs, and Ts? You must compress really well.

DAN STEIN: I mean, there are, I'm sure, technologies that will help with compressing gene sequences. But at the moment, it seems to me that most of the technology around gene sequencing is quite archaic. A lot of the work that's done is like individual bioinformaticians sitting at individual machines, running these open-source algorithms or maybe running them on a compute cluster.

So for us, we aggregate a lot of data, running big data ETLs that import data from the NCBI SWISS-PROT ensemble and small databases of toxicology and pathogens. And then we filter and analyze genetic variants. And we run gene-sequence analysis steps. So for us, one of the things that's been a real game-changer about Kubernetes, we had one particular NGS pipeline type.

So with all of the gene sequencing, NGS pipelines are like running a pipeline of different programs according to a rule set. If program X has a threshold of Y for a specific thing, then run the next program, otherwise something else, that kind of thing. And so people would largely do this themselves on a terminal and run these programs one after another. Or they might have a batch script that runs them together.

And there are some technologies that automate some of this stuff for you. There's something called PGAP, which is like a Python orchestrator for orchestrating some of these bioinformatics jobs. But what we've done is we've built a Kubernetes-based framework that allows you to use a UI to design any arbitrary DAG structure.

And then each node could be a big machine learning task that needs to run on a really powerful GPU machine on AWS. Or it might be something that's not such a big task. And then the orchestration — I don't know if you've come across Argo?

CRAIG BOX: Mm-hmm.

DAN STEIN: We use that for some of our ETL stuff. It's kind of similar what we've built to Argo, but it's specifically for being able to execute things in user-design graphs, whereas Argo, you have to be an engineer to be able to go and write Argo manifest, basically, right? And so we have like a UI, and we have an API for that.

CRAIG BOX: I was going to ask how your team decides on build versus buy. And when you started the project, whatever time that was will determine whether or not things like Kubeflow were available. There are things out there that you could use as baseline for this or you may have had to write some of it yourself. Where did you land on that spectrum?

DAN STEIN: I've looked at Kubeflow in the past. That was one of the things that got me interested in Kubernetes, because I was working as a machine learning engineer. And so obviously that seemed like a useful technology for deploying machine learning.

We've looked at quite a few different technologies. And obviously, we're trying to avoid reinventing the wheel and building things that you don't need to. But in the domain that we're in, there just aren't tools that exist to do what we need to do in the way that we need to do it. So we've kind of been forced to build things.

But we've also been using a technology that I've actually just been made a community champion for. I'm not sure what that means. So I'm feeling quite proud of myself at the moment. A company called Hasura. Hasura is effectively auto-generated APIs. So you give Hasura a Postgres connection string. There are actually other integrations for other database types, but Postgres is the one that they're most famous for. You basically give a docker container, running Hasura, a Postgres connection string.

And then it will create an API for you. And you can do all kinds of cool stuff with granular auth. You can have pagination out of the box, very detailed where clauses. And you have something a bit like GraphiQL. I don't know if you've come across GraphQL playgrounds for GraphQL. But they have their own dashboard that has a sort of database management UI that also allows you to run queries, and create joins inside your API, and join remote schemas, and do all kinds of cool stuff.

So basically, for us, a lot of the work that we do is very experimental. So a scientist is literally doing an experiment and wants to try something that hasn't been tried before, and so needs to look at some connection between data that isn't something that's immediately available.

And it's always been really amazing for us because we can import data into Postgres, and then we can use Hasura to spin up APIs on the fly. And then we created another tool which does the same kind of thing but as a UI. So you take like a YAML manifest, effectively. It describes your application. And you say I want it to have 10 pages.

And page one is going to be the result of some nested GraphQL query that is being executed against the Hasura. And then we basically unwrap the results of the query and generate a table from that. And we've got some features that allow you to export CSVs, and drill into nesting at different levels, or import lists of IDs, that kind of thing.

So if somebody comes to us and says, I want to look at this data from the NCBI, and I've got some custom data set, and I want to see how these things relate, but I don't want to use a database to do that, I don't want to write SQL queries, we can basically spin up an API with a Hasura in seconds.

And then we can write a manifest for our deep-space framework, also in a couple of minutes or whatever, and then just have a custom UI that is kind of ephemeral so we can have it when we need it and then spin up something else when we need it.

CRAIG BOX: Are your engineering team effectively building a product for the bioinformaticians? Do they need to know any of the things you've been talking about here? Or are they just clicking around and dealing with the UI that you've designed?

DAN STEIN: We work closely with scientists in our team. I hear from friends of mine who are in machine learning or data science companies that you have these issues with synchronizing workflows. You can have issues where you've got teams that are working in Jira on very different — so we've got science tickets and dev tickets. And it's quite challenging, that, in a mixed-science engineering company.

But the benefit of it is that we have really good communication with scientists. And we're working alongside the scientists. So the products that we're building are very much developed with scientists for their use cases.

CRAIG BOX: We talked about the idea of controlling music machines from a central controller and so on. People talk about Kubernetes as being orchestration. They do use all these musical metaphors. Do you see a direct connection between these things, having straddled both worlds? Or do you think of them as completely separate environments that just parts of your brain like different things?

DAN STEIN: I think music is code, basically. Music is code that we can hear. And so if you're somebody who's making or thinking about music, I think there's the scale with music because you have, on one side of the scale, people who believe that music is a personal thing, and should be completely thought about personally, and you shouldn't care what anybody else thinks. And then you go right up to the Max Martin end of the scale where everything is about a production line.

I'm probably somewhere in the middle. But I feel that music is an experiment, that there are lots of directions you can go. But there's a common thread to great music. To me, that is like code. There's algorithms and formulas inside it. I've got a friend, John Powell, who wrote a book called, "How Music Works," which is all about this topic, basically. There are lots of people that think quite scientifically about music.

So for me, the problems that you encounter as an engineer and with infrastructure as well are very similar to the problems that your brain encounters when you're thinking about arrangements, and sound design, and that kind of thing. So even though I think people find that surprising to think about it that way, I think there's a really strong common thread. And the proof in the pudding is that there's so many engineers that are into music and that also make music. I'm sure you must have seen that, right, Craig?

CRAIG BOX: Oh, absolutely.

DAN STEIN: It's incredible, the amount of people that seem to be into music that are into code. So we live in a world where so much of everything around us now is a tech platform of some description. And there's a lot of people who are finding that their areas that they were working in retail just are struggling. And there's all of these unfilled jobs in tech that need to be filled.

So I think it's important that people stop looking at coding and technology as something that a certain kind of person does, and start seeing it as just a canvas, just like anything else. It's like, if you do art, your canvas is paper. If you do music, your canvas is an instrument or a door. And code can be a canvas for almost anything that you can imagine, which to me, is just the most incredible thing.

And I think I kind of see code as an extension of music, where before I was writing music, I was limited to the sonic spectrum. And now I can do things involving music but with UIs or virtual reality. And really, the world is your oyster when you start feeling comfortable to use a language or to use languages and write code.

CRAIG BOX: It feels to me that how we're teaching coding to children, especially, is changing. You wouldn't think of someone doing music and say, all right, well, hey, only if you're going to do this seriously should you take it up at all. There are a lot of people who will just do music for fun. They have no interest in being professional.

That is something, I think, that people used to look at programming and say, well, hey, unless you're going to get a job doing this thing, there's no point knowing anything about it. And it feels like we're starting to teach people, this is a thing you need to be aware of how it works.

What advice would you give someone who's interested in coding technology, whose brain kind of works in the way that is the amalgam of those two things? Given your experience in the industry, given how you see people growing up, given that you're a father yourself, how would you advise people to get a start today?

DAN STEIN: There's never been so much free help available, from your podcast, right, Craig, to platforms like Treehouse or—

CRAIG BOX: Stack Overflow for politicians.

DAN STEIN: [LAUGHING] Stack Overflow for politicians. Obviously, things like Udemy, obviously YouTube. There's so many platforms to help you learn code. So have you come across 3Blue1Brown on YouTube?

CRAIG BOX: I have.

DAN STEIN: He does the most incredible videos on concepts from like maths through to machine learning, linear algebra, that kind of thing. I kind of lost interest with that stuff when I was at school. I used to ask difficult questions, and people used to get annoyed with me and think that I was trying to be smart or something.

So for me, going through that journey that I've been going through and learning or going back over these things again, obviously I've had to go back over linear algebra for the machine-learning stuff that I've been doing, and having access to those videos, with all of his passion that he puts into the way that he produces his videos, I think you can watch one of his videos and see a concept that before felt like a scary school thing. But it's presented in a way that feels friendly and passionate. And it's infectious.

So I think there's lots of infectious material like that that gives technical topics that are normally maybe a bit challenging for people a whole new lease of life. But in terms of code, specifically, some of the online platforms, like Treehouse, will give you a environment where you can actually type the code and then see the response or a kind of online shell. I think those kinds of things weren't available before.

But ultimately, there's a myth that people that are good at coding are a certain kind of person because people struggled with maths at school like — I mean, I did and so many people do. I'm sure you must know people that struggled with or hated maths at school that are amazing engineers, right? It's just the school system doesn't really match up to where we are now. And I think modern learning is such a different paradigm from the kind of learning that most of us had growing up at school.

So I think it's important for people to not think back to school when they think of coding and maths, and instead to go and look at what's out there, and have an open mind because there are some really incredible gamified platforms out there that make learning really fun now. The biggest thing that I find when I'm talking to people who are thinking about pivoting or are interested in getting into these things for the first time is that they just think back to how it was at school, and that puts them off.

Perhaps that's a generational thing. I certainly hope that younger people today are getting the kind of introduction that they would have got to art and music, where it's like something that's interesting and fun, not something that's arduous, that you have to learn in order to tick a box.

CRAIG BOX: There is so much more that you can do on computers today than when you and I were learning. Do you feel that the experience you had having to plug in the basic cartridge and teach yourself from the ground up is something that works for everyone? Or do you think that that was formative to you? Do you think that the modern approach, where there is so much more out there, but there's the paradox of choice that we all have these things that we can go to and learn. How do you focus in that kind of world where you don't just have one path to follow?

DAN STEIN: I feel like you've kind of crossed a few different topics there in terms of focusing with all the distractions online. That's a big problem, I'm sure, for everybody. It's like even when you are on a path—

CRAIG BOX: I'm scrolling Reddit as we speak.

DAN STEIN: [LAUGHING] Great to hear, man. [LAUGHING] I'm glad to hear that I'm keeping you that interesting, Craig.

CRAIG BOX: I'm not.

DAN STEIN: Let's say, for example, like, I've been learning Rust recently. At work, I mostly use Nodes, JavaScript, Python, and lots of Bash scripts and stuff. I've been learning Rust. And I've been finding it really, really hard to find the time to do that. That all comes into the distractions of life, and family, and all that kind of thing as well, the whole topic around time management as it exists in 2022 for most people and how you can learn new things.

But I guess where you're talking about the cartridge thing, there is definitely the ability to go and buy a course on Udemy for next to nothing and be guided through a course with lots of visual cues that make it a lot more conducive to fitting into an hour here or an hour there. It's hard with all the distractions. But at the same time, the resources are so plentiful. There's just so much out there. And there's a lot of people that are willing to help you.

I was on Stack Overflow with a Rust problem that I was having a few days ago. And somebody helped me get through something that I probably would have just been completely lost if I'd just been with my Rust book and doing what I was doing. Do you know what I mean? So I think that kind of thing didn't exist when I was growing up. It was all, you had to buy a book.

And there was nobody that you could really ask unless maybe you had a teacher at school that was interested in something. The learning side, in terms of resources, has never been more plentiful. There's so many free resources out there.

CRAIG BOX: Is that true of music production as well? Do you think that people can pick up the skills that they need to from online resources in the way that you can with tech?

DAN STEIN: One of the reasons that I've become less interested in music a bit is because there's so many presets and pre-made sounds available that if somebody writes a track this week, by the end of a few days, after it's been heard on TikTok, somebody's posted a YouTube video explaining how to make the lead sound or even just like, here, download this preset. That's a real challenge for music in terms of creativity.

I think you've also got this kind of globalization of Spotify where record labels want to release music that's globally and Spotify friendly, which is a problem, I think, as well. So ironically, it's been a bigger problem for music, whereas I think for tech, it's like an open doorway. I say to a lot of friends that I have who are musicians, that maybe don't feel like they fit into the modern music world with TikTok and all the rest of it, to have a look at codes.

I find myself, a lot of the time, talking about what I'm doing with work and with code and showing people things that I've built and things that I'm doing, trying to get them to be interested in it. I'm not sure if I'm succeeding with that. But I'm definitely trying every day. It's like definitely a certain kind of brain as well. In tech, you've got the very designery-focused, frontend-element design, just design in general, on one side of the spectrum. And then you go right to infrastructure and back end on the other side.

And across that spectrum, you're going to get people who aren't that technical that love design, that some front end might appeal to them. But then there's very much a kind of person that's interested in low-level algorithms, and infrastructure design, and that kind of thing. So it isn't for everybody. But I think that it's for a lot more people than realize. I think there's a lot more people out there that think that it isn't for them, that it could be.

CRAIG BOX: Good. Well, thank you for putting out the good word. Are we going to have a single out there in the future that you can download and play on your Commodore 64? Or are we going to have a spoken-word album from you extolling the virtues of Rust or something like that?

DAN STEIN: I have written down a few potential names for tracks in my phone that have come from machine-learning concepts. And I'm always trying to find a way to slip something in there. But I haven't managed to so far. I've got a new track called "Higher" that's out at the moment. It's out a few weeks ago.

If your listeners want to check that out, it's pretty cool, like a drum and bass track. And there's some new music that I'm working on, as well as building my crazy, Kubernetes-based DAG stuff that I'm doing for work at the moment. So there's more music coming as well.

CRAIG BOX: All right, thank you very much for joining us today, Dan.

DAN STEIN: Great. Yeah, no problem, Craig. It's been fun.

CRAIG BOX: You can find Dan on Twitter @DJfreshUK and on Facebook at thedjfresh.com.

[MUSIC PLAYING]

Thank you, as always, for listening. If you enjoyed the show, well, you pretty much know what to do by now. If you have any feedback, please send it to us on Twitter @KubernetesPod or by email at kubernetespodcast@google.com. The website, kubernetespodcast@google.com, is where you find transcripts, and show notes, as well as the links to subscribe. What's next for the show? Tune in next week to find out.

[MUSIC PLAYING]